[Numpy-discussion] numpy-1.11.0.dev0 windows wheels compiled with mingwpy available

[Carl Kleffner]

I made numpy master (numpy-1.11.0.dev0 ,
https://github.com/numpy/numpy/commit/0243bce23383ff5e894b99e40df2f8fd806ad79f)
windows binary wheels available for testing.

pip install -i https://pypi.anaconda.org/carlkl/simple numpy

These builds are compiled with OPENBLAS trunk for BLAS/LAPACK support and
the mingwpy compiler toolchain.

OpenBLAS is deployed within the numpy wheels. To be performant on all usual
CPU architectures OpenBLAS is configured with it's 'dynamic architecture'
and automatic CPU detection.

This version of numpy fakes long double as double just like the MSVC builds.

Some test statistics:

win32 (32 bit)
numpy-1.11.0.dev0, python-2.6: errors=8, failures=1
numpy-1.11.0.dev0, python-2.7: errors=8, failures=1
numpy-1.11.0.dev0, python-3.3: errors=9
numpy-1.11.0.dev0, python-3.4: errors=9

amd64 (64bit)
numpy-1.11.0.dev0, python-2.6: errors=9, failures=6
numpy-1.11.0.dev0, python-2.7: errors=9, failures=6
numpy-1.11.0.dev0, python-3.3: errors=10, failures=6
numpy-1.11.0.dev0, python-3.4: errors=10, failures=6

Carl

[Robert McGibbon]

Hi,

Is it possible to test this with py35 as well? For MSVC, py35 requires a
new compiler toolchain (VS2015) -- is that something mingwpy/mingw-w64 can
handle?

-Robert

Post by Carl Kleffner
I made numpy master (numpy-1.11.0.dev0 ,
https://github.com/numpy/numpy/commit/0243bce23383ff5e894b99e40df2f8fd806ad79f)
windows binary wheels available for testing.

pip install -i https://pypi.anaconda.org/carlkl/simple numpy

These builds are compiled with OPENBLAS trunk for BLAS/LAPACK support and
the mingwpy compiler toolchain.
OpenBLAS is deployed within the numpy wheels. To be performant on all
usual CPU architectures OpenBLAS is configured with it's 'dynamic
architecture' and automatic CPU detection.
This version of numpy fakes long double as double just like the MSVC builds.
win32 (32 bit)
numpy-1.11.0.dev0, python-2.6: errors=8, failures=1
numpy-1.11.0.dev0, python-2.7: errors=8, failures=1
numpy-1.11.0.dev0, python-3.3: errors=9
numpy-1.11.0.dev0, python-3.4: errors=9
amd64 (64bit)
numpy-1.11.0.dev0, python-2.6: errors=9, failures=6
numpy-1.11.0.dev0, python-2.7: errors=9, failures=6
numpy-1.11.0.dev0, python-3.3: errors=10, failures=6
numpy-1.11.0.dev0, python-3.4: errors=10, failures=6
Carl
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Olivier Grisel]

Post by Robert McGibbon
Is it possible to test this with py35 as well?

Unfortunately not yet.

Post by Robert McGibbon
For MSVC, py35 requires a new compiler toolchain (VS2015) -- is that

something mingwpy/mingw-w64 can handle?

I am pretty sure that mingwpy does not support Python 3.5 yet.

I don't know the status of the interop of mingw-w64 w.r.t. VS2015 but as
far as I know it's not supported yet either. Once the issue is fixed at the
upstream level, I think mingwpy could be rebuilt to benefit from the fix.

[Nathaniel Smith]

On Mon, Oct 19, 2015 at 2:26 AM, Olivier Grisel

Post by Olivier Grisel

Post by Robert McGibbon
Is it possible to test this with py35 as well?

Unfortunately not yet.

Post by Robert McGibbon
For MSVC, py35 requires a new compiler toolchain (VS2015) -- is that
something mingwpy/mingw-w64 can handle?

I am pretty sure that mingwpy does not support Python 3.5 yet.

Correct.

Post by Olivier Grisel
I don't know the status of the interop of mingw-w64 w.r.t. VS2015 but as far
as I know it's not supported yet either. Once the issue is fixed at the
upstream level, I think mingwpy could be rebuilt to benefit from the fix.

Upstream mingw-w64 doesn't support interop with any version of visual
studio that was released this millennium -- all the interop stuff is
new in mingwpy.

VS2015 had a major reorganization of how it handles runtime libraries,
so it's not quite so trivial as just adding support the same way as
was done for VS2008 and VS2010. Or rather, IIUC: we *could* just add
support the same way as before, but there are undocumented rules about
which parts of the new runtime are considered stable and which are
not, so if we did this willy-nilly then we might end up using some of
the "unstable" parts. And then in 2017 the Windows team does some
internal refactoring and pushes it out through windows update and
suddenly NumPy / R / Julia / git / ... all start segfaulting at
startup on Windows, which would be a disaster from everyone's point of
view. We've pointed this out to the Python team at Microsoft and
they've promised to try and put Carl and the relevant mingw-w64 folks
in touch with the relevant internal folks at MS to hopefully tell us
how to do this correctly... fingers crossed :-).

Aside from that, the main challenge for mingwpy in general is exactly
the issue of upstream support: if we don't get the interop stuff
pushed upstream from mingwpy to mingw-w64, then it will rot and break.
And upstream would love to have this interoperability as an officially
supported feature... but upstream doesn't consider what we have right
now to be maintainable, so they won't take it as is. (And honestly,
this is a reasonable opinion.) So what I've been trying to do is to
scrounge up some funding to support Carl and upstream doing this right
(the rough estimate is ~3 person-months of work).

The original goal was to get MS to pay for this, on the theory that
they should be cleaning up their own messes, but after 6 months of
back-and-forth we've pretty much given up on that at this point, and
I'm in the process of emailing everyone I can think of who might be
convinced to donate some money to the cause. Maybe we should have a
kickstarter or something, I dunno :-).

-n

[Robert McGibbon]

Got it. Thanks, Nathaniel -- this is really good information.

-Robert

Post by Nathaniel Smith
On Mon, Oct 19, 2015 at 2:26 AM, Olivier Grisel

Post by Olivier Grisel

Post by Robert McGibbon
Is it possible to test this with py35 as well?

Unfortunately not yet.

Post by Robert McGibbon
For MSVC, py35 requires a new compiler toolchain (VS2015) -- is that
something mingwpy/mingw-w64 can handle?

I am pretty sure that mingwpy does not support Python 3.5 yet.

Correct.

Post by Olivier Grisel
I don't know the status of the interop of mingw-w64 w.r.t. VS2015 but as

far

Post by Olivier Grisel
as I know it's not supported yet either. Once the issue is fixed at the
upstream level, I think mingwpy could be rebuilt to benefit from the fix.

Upstream mingw-w64 doesn't support interop with any version of visual
studio that was released this millennium -- all the interop stuff is
new in mingwpy.
VS2015 had a major reorganization of how it handles runtime libraries,
so it's not quite so trivial as just adding support the same way as
was done for VS2008 and VS2010. Or rather, IIUC: we *could* just add
support the same way as before, but there are undocumented rules about
which parts of the new runtime are considered stable and which are
not, so if we did this willy-nilly then we might end up using some of
the "unstable" parts. And then in 2017 the Windows team does some
internal refactoring and pushes it out through windows update and
suddenly NumPy / R / Julia / git / ... all start segfaulting at
startup on Windows, which would be a disaster from everyone's point of
view. We've pointed this out to the Python team at Microsoft and
they've promised to try and put Carl and the relevant mingw-w64 folks
in touch with the relevant internal folks at MS to hopefully tell us
how to do this correctly... fingers crossed :-).
Aside from that, the main challenge for mingwpy in general is exactly
the issue of upstream support: if we don't get the interop stuff
pushed upstream from mingwpy to mingw-w64, then it will rot and break.
And upstream would love to have this interoperability as an officially
supported feature... but upstream doesn't consider what we have right
now to be maintainable, so they won't take it as is. (And honestly,
this is a reasonable opinion.) So what I've been trying to do is to
scrounge up some funding to support Carl and upstream doing this right
(the rough estimate is ~3 person-months of work).
The original goal was to get MS to pay for this, on the theory that
they should be cleaning up their own messes, but after 6 months of
back-and-forth we've pretty much given up on that at this point, and
I'm in the process of emailing everyone I can think of who might be
convinced to donate some money to the cause. Maybe we should have a
kickstarter or something, I dunno :-).
-n
--
Nathaniel J. Smith -- http://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Henry Gomersall]

Post by Nathaniel Smith
The original goal was to get MS to pay for this, on the theory that
they should be cleaning up their own messes, but after 6 months of
back-and-forth we've pretty much given up on that at this point, and
I'm in the process of emailing everyone I can think of who might be
convinced to donate some money to the cause. Maybe we should have a
kickstarter or something, I dunno :-).

Just noticed this. Yes a kickstarter would be good - I'd be willing to
contribute something.

What's the current estimated cost for 3 person months?

Cheers,

Henry

[Ralf Gommers]

Post by Henry Gomersall

Post by Nathaniel Smith
The original goal was to get MS to pay for this, on the theory that
they should be cleaning up their own messes, but after 6 months of
back-and-forth we've pretty much given up on that at this point, and
I'm in the process of emailing everyone I can think of who might be
convinced to donate some money to the cause. Maybe we should have a
kickstarter or something, I dunno :-).

Just noticed this. Yes a kickstarter would be good - I'd be willing to
contribute something.
What's the current estimated cost for 3 person months?

We're expecting to be able to announce some progress on this front during
the holiday season - please hold on for a little bit.

Cheers,
Ralf

[Henry Gomersall]

Post by Henry Gomersall

Post by Nathaniel Smith
The original goal was to get MS to pay for this, on the theory that
they should be cleaning up their own messes, but after 6 months of
back-and-forth we've pretty much given up on that at this point, and
I'm in the process of emailing everyone I can think of who might be
convinced to donate some money to the cause. Maybe we should have a
kickstarter or something, I dunno :-).

Just noticed this. Yes a kickstarter would be good - I'd be willing to
contribute something.
What's the current estimated cost for 3 person months?
We're expecting to be able to announce some progress on this front
during the holiday season - please hold on for a little bit.

Great, will do!

Henry (oddly excited about windows build systems)