[Numpy-discussion] Numpy 1.11.0b1 is out

Hi Chuck,

I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)

the tarball seems to be incomplete, hope that does not bode ill ;-)

adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources.
executing numpy/core/code_generators/generate_numpy_api.py
error: [Errno 2] No such file or directory: 'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'

tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes

-rw-rw-r-- charris/charris 62563 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h
-rw-rw-r-- charris/charris 981 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h

FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11
and 3.5.1 on Mac OS X 10.10.

Cheers,
Derek

Charles R Harris

2016-01-27 01:58:51 UTC

On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier <

Post by Derek Homeier
Hi Chuck,

I'm pleased to announce that Numpy 1.11.0b1 is now available on

sourceforge. This is a source release as the mingw32 toolchain is broken.
Please test it out and report any errors that you discover. Hopefully we
can do better with 1.11.0 than we did with 1.10.0 ;)
the tarball seems to be incomplete, hope that does not bode ill ;-)
adding
'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h'
to sources.
executing numpy/core/code_generators/generate_numpy_api.py
'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'

Grr, yes indeed, `paver sdist` doesn't do the right thing, none of the
`multiarray/*.c.src` files are included, but it works fine in 1.10.x. The
changes are minimal, the only thing that would seem to matter is the
removal of setupegg.py. Ralf, any ideas.

tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes

-rw-rw-r-- charris/charris 62563 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h
-rw-rw-r-- charris/charris 981 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds
and passes all tests with Python 2.7.11
and 3.5.1 on Mac OS X 10.10.

You probably didn't fetch the tags, if they can't be reached from the
branch head they don't download automatically. Try `git fetch --tags
upstream`

Chuck

Charles R Harris

2016-01-27 02:13:30 UTC

Post by Charles R Harris
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier <

Post by Derek Homeier
Hi Chuck,

I'm pleased to announce that Numpy 1.11.0b1 is now available on

tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes

-rw-rw-r-- charris/charris 62563 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h
-rw-rw-r-- charris/charris 981 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?)
builds and passes all tests with Python 2.7.11
and 3.5.1 on Mac OS X 10.10.

You probably didn't fetch the tags, if they can't be reached from the
branch head they don't download automatically. Try `git fetch --tags
upstream`

setupegg.py doesn't seem to matter...

Chuck

Charles R Harris

2016-01-27 02:45:11 UTC

On Tue, Jan 26, 2016 at 6:58 PM, Charles R Harris <

Post by Charles R Harris
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier <

Post by Derek Homeier
Hi Chuck,

I'm pleased to announce that Numpy 1.11.0b1 is now available on

tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes

-rw-rw-r-- charris/charris 62563 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h
-rw-rw-r-- charris/charris 981 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?)
builds and passes all tests with Python 2.7.11
and 3.5.1 on Mac OS X 10.10.

You probably didn't fetch the tags, if they can't be reached from the
branch head they don't download automatically. Try `git fetch --tags
upstream`

setupegg.py doesn't seem to matter...

OK, it is the changes in the root setup.py file, probably the switch to
setuptools.

Chuck

Charles R Harris

2016-01-27 03:47:42 UTC

On Tue, Jan 26, 2016 at 7:13 PM, Charles R Harris <

On Tue, Jan 26, 2016 at 6:58 PM, Charles R Harris <

Post by Charles R Harris
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier <

Post by Derek Homeier
Hi Chuck,

I'm pleased to announce that Numpy 1.11.0b1 is now available on

tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes

-rw-rw-r-- charris/charris 62563 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h
-rw-rw-r-- charris/charris 981 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?)
builds and passes all tests with Python 2.7.11
and 3.5.1 on Mac OS X 10.10.

You probably didn't fetch the tags, if they can't be reached from the
branch head they don't download automatically. Try `git fetch --tags
upstream`

setupegg.py doesn't seem to matter...

OK, it is the changes in the root setup.py file, probably the switch to
setuptools.

Yep, it's setuptools. If I import sdist from distutils instead, everything
works fine.

Chuck

Ralf Gommers

2016-01-27 09:23:31 UTC

On Tue, Jan 26, 2016 at 7:45 PM, Charles R Harris <

On Tue, Jan 26, 2016 at 7:13 PM, Charles R Harris <

On Tue, Jan 26, 2016 at 6:58 PM, Charles R Harris <

Post by Charles R Harris
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier <

Post by Derek Homeier
Hi Chuck,

I'm pleased to announce that Numpy 1.11.0b1 is now available on

tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes

-rw-rw-r-- charris/charris 62563 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h
-rw-rw-r-- charris/charris 981 2016-01-21 20:38
numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?)
builds and passes all tests with Python 2.7.11
and 3.5.1 on Mac OS X 10.10.

You probably didn't fetch the tags, if they can't be reached from the
branch head they don't download automatically. Try `git fetch --tags
upstream`

setupegg.py doesn't seem to matter...

OK, it is the changes in the root setup.py file, probably the switch to
setuptools.

Yep, it's setuptools. If I import sdist from distutils instead, everything
works fine.

Setuptools starts build_src, don't know why yet but it doesn't look to me
like it should be doing that. Issue for more detailed discussion:
https://github.com/numpy/numpy/issues/7127

Ralf

Derek Homeier

2016-01-27 15:37:12 UTC

Post by Derek Homeier
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11
and 3.5.1 on Mac OS X 10.10.
You probably didn't fetch the tags, if they can't be reached from the branch head they don't download automatically. Try `git fetch --tags upstream`

Thanks, that did it. Successfully tested v1.11.0b1 on 10.11 and with Python 2.7.8 and 3.4.1 on openSUSE 13.2 as well.

Derek

Nadav Horesh

2016-01-27 11:19:02 UTC

Why the dot function/method is slower than @ on python 3.5.1? Tested from the latest 1.11 maintenance branch.

np.__version__
Out[39]: '1.11.0.dev0+Unknown'

%timeit A @ c
10000 loops, best of 3: 185 µs per loop

%timeit A.dot(c)
1000 loops, best of 3: 526 µs per loop

%timeit np.dot(A,c)
1000 loops, best of 3: 527 µs per loop

A.dtype, A.shape, A.flags
Out[43]:
(dtype('float32'), (100, 100, 3), C_CONTIGUOUS : True
F_CONTIGUOUS : False
OWNDATA : True
WRITEABLE : True
ALIGNED : True
UPDATEIFCOPY : False)

c.dtype, c.shape, c.flags
Out[44]:
(dtype('float32'), (3, 3), C_CONTIGUOUS : True
F_CONTIGUOUS : False
OWNDATA : True
WRITEABLE : True
ALIGNED : True
UPDATEIFCOPY : False)

From: NumPy-Discussion <numpy-discussion-***@scipy.org> on behalf of Charles R Harris <***@gmail.com>
Sent: 26 January 2016 22:49
To: numpy-discussion; SciPy Developers List; SciPy Users List
Subject: [Numpy-discussion] Numpy 1.11.0b1 is out

Hi All,

I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)

Chuck

Sebastian Berg

2016-01-27 12:10:36 UTC

Post by Nadav Horesh
from the latest 1.11 maintenance branch.

The explanation I think is that you do not have a blas optimization. In
which case the fallback mode is probably faster in the @ case (since it
has SSE2 optimization by using einsum, while np.dot does not do that).

Btw. thanks for all the work getting this done Chuck!

- Sebastian

Post by Nadav Horesh
np.__version__
Out[39]: '1.11.0.dev0+Unknown'
10000 loops, best of 3: 185 Âµs per loop
%timeit A.dot(c)
1000 loops, best of 3: 526 Âµs per loop
%timeit np.dot(A,c)
1000 loops, best of 3: 527 Âµs per loop
A.dtype, A.shape, A.flags
(dtype('float32'), (100, 100, 3), C_CONTIGUOUS : True
F_CONTIGUOUS : False
OWNDATA : True
WRITEABLE : True
ALIGNED : True
UPDATEIFCOPY : False)
c.dtype, c.shape, c.flags
(dtype('float32'), (3, 3), C_CONTIGUOUS : True
F_CONTIGUOUS : False
OWNDATA : True
WRITEABLE : True
ALIGNED : True
UPDATEIFCOPY : False)
Sent: 26 January 2016 22:49
To: numpy-discussion; SciPy Developers List; SciPy Users List
Subject: [Numpy-discussion] Numpy 1.11.0b1 is out
Hi All,
I'm pleased to announce that Numpy 1.11.0b1 is now available on
sourceforge. This is a source release as the mingw32 toolchain is
broken. Please test it out and report any errors that you discover.
Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
Chuck
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Derek Homeier

2016-01-30 19:27:58 UTC

Post by Nadav Horesh
from the latest 1.11 maintenance branch.

The explanation I think is that you do not have a blas optimization. In
has SSE2 optimization by using einsum, while np.dot does not do that).

I am a bit confused now, as A @ c is just short for A.__matmul__(c) or equivalent
to np.matmul(A,c), so why would these not use the optimised blas?
Also, I am getting almost identical results on my Mac, yet I thought numpy would
by default build against the VecLib optimised BLAS. If I build explicitly against
ATLAS, I am actually seeing slightly slower results.
But I also saw these kind of warnings on the first timeit runs:

%timeit A.dot(c)
The slowest run took 6.91 times longer than the fastest. This could mean that an intermediate result is being cached

and when testing much larger arrays, the discrepancy between matmul and dot rather
increases, so perhaps this is more an issue of a less memory-efficient implementation
in np.dot?

Cheers,
Derek

Sebastian Berg

2016-01-31 08:48:02 UTC

On 27 Jan 2016, at 1:10 pm, Sebastian Berg <

Post by Nadav Horesh
from the latest 1.11 maintenance branch.

The explanation I think is that you do not have a blas
optimization. In
has SSE2 optimization by using einsum, while np.dot does not do that).

to np.matmul(A,c), so why would these not use the optimised blas?
Also, I am getting almost identical results on my Mac, yet I thought numpy would
by default build against the VecLib optimised BLAS. If I build
explicitly against
ATLAS, I am actually seeing slightly slower results.
%timeit A.dot(c)
The slowest run took 6.91 times longer than the fastest. This could
mean that an intermediate result is being cached
and when testing much larger arrays, the discrepancy between matmul and dot rather
increases, so perhaps this is more an issue of a less memory
-efficient implementation
in np.dot?

Sorry, I missed the fact that one of the arrays was 3D. In that case I
am not even sure which if the functions call into blas or what else
they have to do, would have to check. Note that `np.dot` uses a
different type of combinging high dimensional arrays. @/matmul
broadcasts extra axes, while np.dot will do the outer combination of
them, so that the result is:

As = A.shape
As.pop(-1)
cs = c.shape
cs.pop(-2) # if possible
result_shape = As + cs

which happens to be identical if only A.ndim > 2 and c.ndim <= 2.

- Sebastian

Cheers,
Derek
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Derek Homeier

2016-02-02 00:23:33 UTC

On 27 Jan 2016, at 1:10 pm, Sebastian Berg <