[Numpy-discussion] guvectorize, a helper for writing generalized ufuncs

[Stephan Hoyer]

NumPy has the handy np.vectorize for turning Python code that operates on
scalars into a function that vectorizes works like a ufunc, but no helper
function for creating generalized ufuncs (
http://docs.scipy.org/doc/numpy/reference/c-api.generalized-ufuncs.html).

np.apply_along_axis accomplishes some of this, but it only allows a single
core dimension on a single argument.

So I propose adding a new object, np.guvectorize(pyfunc, signature, otypes,
...), where pyfunc is defined over the core dimensions only of any inputs
and signature is any valid gufunc signature (a string). Calling this object
would apply the gufunc. This is inspired by the similar numba.guvectorize,
which is currently the easiest way to write a gufunc in Python.

In addition to be handy like vectorize, such functionality would be
especially useful for with working libraries that build upon NumPy to
extend the capabilities of generalized ufuncs (e.g., xarray after
https://github.com/pydata/xarray/pull/964).

Cheers,
Stephan

[Nathan Goldbaum]

Post by Stephan Hoyer
NumPy has the handy np.vectorize for turning Python code that operates on
scalars into a function that vectorizes works like a ufunc, but no helper
function for creating generalized ufuncs (http://docs.scipy.org/doc/
numpy/reference/c-api.generalized-ufuncs.html).
np.apply_along_axis accomplishes some of this, but it only allows a single
core dimension on a single argument.
So I propose adding a new object, np.guvectorize(pyfunc, signature,
otypes, ...), where pyfunc is defined over the core dimensions only of any
inputs and signature is any valid gufunc signature (a string). Calling this
object would apply the gufunc. This is inspired by the similar
numba.guvectorize, which is currently the easiest way to write a gufunc in
Python.
In addition to be handy like vectorize, such functionality would be
especially useful for with working libraries that build upon NumPy to
extend the capabilities of generalized ufuncs (e.g., xarray after
https://github.com/pydata/xarray/pull/964).

First, this seems really cool. I hope it goes somewhere.

I'm curious whether you have a plan to deal with the python functional call
overhead. Numba gets around this by JIT-compiling python functions - is
there something analogous you can do in NumPy or will this always be
limited by the overhead of repeatedly calling a Python implementation of
the "core" operation?

-Nathan

Post by Stephan Hoyer
Cheers,
Stephan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Stephan Hoyer]

Post by Nathan Goldbaum
I'm curious whether you have a plan to deal with the python functional
call overhead. Numba gets around this by JIT-compiling python functions -
is there something analogous you can do in NumPy or will this always be
limited by the overhead of repeatedly calling a Python implementation of
the "core" operation?

I don't think there is any way to avoid this in NumPy proper, but that's OK
(it's similar to the existing overhead of vectorize).

Numba already has guvectorize (and it's own version of vectorize as well),
which already does exactly this.

[Travis Oliphant]

There has been some discussion on the Numba mailing list as well about a
version of guvectorize that doesn't compile for testing and flexibility.

Having this be inside NumPy itself seems ideal.

-Travis

Post by Stephan Hoyer

Post by Nathan Goldbaum
I'm curious whether you have a plan to deal with the python functional
call overhead. Numba gets around this by JIT-compiling python functions -
is there something analogous you can do in NumPy or will this always be
limited by the overhead of repeatedly calling a Python implementation of
the "core" operation?

I don't think there is any way to avoid this in NumPy proper, but that's
OK (it's similar to the existing overhead of vectorize).
Numba already has guvectorize (and it's own version of vectorize as well),
which already does exactly this.
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Stephan Hoyer]

I have put a pull request implementing numpy.guvectorize up for review:
https://github.com/numpy/numpy/pull/8054

Cheers,
Stephan

Post by Travis Oliphant
There has been some discussion on the Numba mailing list as well about a
version of guvectorize that doesn't compile for testing and flexibility.
Having this be inside NumPy itself seems ideal.
-Travis

Post by Stephan Hoyer

Post by Nathan Goldbaum
I'm curious whether you have a plan to deal with the python functional
call overhead. Numba gets around this by JIT-compiling python functions -
is there something analogous you can do in NumPy or will this always be
limited by the overhead of repeatedly calling a Python implementation of
the "core" operation?

I don't think there is any way to avoid this in NumPy proper, but that's
OK (it's similar to the existing overhead of vectorize).
Numba already has guvectorize (and it's own version of vectorize as
well), which already does exactly this.
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant, PhD*
*Co-founder and CEO*
@teoliphant
512-222-5440
http://www.continuum.io
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion