[Numpy-discussion] Fwd: Numpy for data manipulation

[Alex Rogozhnikov]

Hi, I have written some numpy tips and tricks I am using, which may be
interesting to you.
This is quite long reading, so I've splitted it into two parts:

http://arogozhnikov.github.io/2015/09/29/NumpyTipsAndTricks1.html
http://arogozhnikov.github.io/2015/09/30/NumpyTipsAndTricks2.html

Comments are welcome, specially if you know any other ways to make this
code faster (or better).

Regards,
Alex.

[Stefan van der Walt]

Post by Alex Rogozhnikov
Hi, I have written some numpy tips and tricks I am using, which may be
interesting to you.
http://arogozhnikov.github.io/2015/09/29/NumpyTipsAndTricks1.html
http://arogozhnikov.github.io/2015/09/30/NumpyTipsAndTricks2.html

I think that's a nice list already!

I would probably start with:

%matplotlib inline
import numpy as np

Then port all the code to Python 3 (or at least Python 2 & 3 compatible).

Perhaps some illustrations could be useful, e.g. how to use the
IronTransform to do histogram equalization.

Stéfan

[Jaime Fernández del Río]

On Thu, Oct 1, 2015 at 11:46 AM, Alex Rogozhnikov <

Post by Alex Rogozhnikov
Hi, I have written some numpy tips and tricks I am using, which may be
interesting to you.
http://arogozhnikov.github.io/2015/09/29/NumpyTipsAndTricks1.html

The recommendation of inverting a permutation by argsort'ing it, while it
works, is suboptimal, as it takes O(n log(n)) time, and you can do it in
linear time:

In [14]: import numpy as np

In [15]: arr = np.random.rand(10)

In [16]: perm = arr.argsort()

In [17]: perm
Out[17]: array([5, 0, 9, 4, 2, 8, 6, 7, 1, 3])

In [18]: inv_perm = np.empty_like(perm)

In [19]: inv_perm[perm] = np.arange(len(perm))

In [20]: np.all(inv_perm == perm.argsort())
Out[20]: True

It does require two lines of code, so for small stuff it is probably good
enough to argsort, but it gave e.g. np.unique a nice boost on larger arrays
when we applied it there.

Jaime

Post by Alex Rogozhnikov
http://arogozhnikov.github.io/2015/09/30/NumpyTipsAndTricks2.html
Comments are welcome, specially if you know any other ways to make this
code faster (or better).
Regards,
Alex.
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Thomas Caswell]

I would suggest

%matplotlib notebook

It will still have to a nice png, but you get an interactive figure when it
is live.

I agree that making the example code Python3 is critical.

Tom

Post by Jaime Fernández del Río
On Thu, Oct 1, 2015 at 11:46 AM, Alex Rogozhnikov <

Post by Alex Rogozhnikov
Hi, I have written some numpy tips and tricks I am using, which may be
interesting to you.
http://arogozhnikov.github.io/2015/09/29/NumpyTipsAndTricks1.html

The recommendation of inverting a permutation by argsort'ing it, while it
works, is suboptimal, as it takes O(n log(n)) time, and you can do it in
In [14]: import numpy as np
In [15]: arr = np.random.rand(10)
In [16]: perm = arr.argsort()
In [17]: perm
Out[17]: array([5, 0, 9, 4, 2, 8, 6, 7, 1, 3])
In [18]: inv_perm = np.empty_like(perm)
In [19]: inv_perm[perm] = np.arange(len(perm))
In [20]: np.all(inv_perm == perm.argsort())
Out[20]: True
It does require two lines of code, so for small stuff it is probably good
enough to argsort, but it gave e.g. np.unique a nice boost on larger arrays
when we applied it there.
Jaime

Post by Alex Rogozhnikov
http://arogozhnikov.github.io/2015/09/30/NumpyTipsAndTricks2.html
Comments are welcome, specially if you know any other ways to make this
code faster (or better).
Regards,
Alex.
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación mundial.
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Juan Nunez-Iglesias]

It will still have to a nice png, but you get an interactive figure when it is live.



You just blew my mind. =D




+1 to Python 3 and aliasing numpy as np.

[Alex Rogozhnikov]

Post by Thomas Caswell
I would suggest
%matplotlib notebook
It will still have to a nice png, but you get an interactive figure when it is live.

Amazing, thanks. I was using mpld3 for this.
(for some strange reason I need to put %matplotlib notebook before each
plot)

Post by Thomas Caswell
The recommendation of inverting a permutation by argsort'ing it, while
it works, is suboptimal, as it takes O(n log(n)) time, and you can do

Actually, there is (later in post) a linear solution using bincount, but
your code is definitely better. Thanks!

[Kiko]

Post by Thomas Caswell
I would suggest

Post by Thomas Caswell
%matplotlib notebook
It will still have to a nice png, but you get an interactive figure when it is live.

Amazing, thanks. I was using mpld3 for this.
(for some strange reason I need to put %matplotlib notebook before each
plot)

You should create a figure before each plot instead of putthon %matplotlib
notebook
plt.figure()
....

Post by Thomas Caswell
The recommendation of inverting a permutation by argsort'ing it, while it

Post by Thomas Caswell
works, is suboptimal, as it takes O(n log(n)) time, and you can do it in

Actually, there is (later in post) a linear solution using bincount, but
your code is definitely better. Thanks!
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Kiko]

Post by Kiko

Post by Thomas Caswell
I would suggest

Post by Thomas Caswell
%matplotlib notebook
It will still have to a nice png, but you get an interactive figure when it is live.

Amazing, thanks. I was using mpld3 for this.
(for some strange reason I need to put %matplotlib notebook before each
plot)

You should create a figure before each plot instead of putthon %matplotlib
notebook
plt.figure()
....

putthon == putting

Post by Kiko

Post by Thomas Caswell
The recommendation of inverting a permutation by argsort'ing it, while it

Post by Thomas Caswell
works, is suboptimal, as it takes O(n log(n)) time, and you can do it in

Actually, there is (later in post) a linear solution using bincount, but
your code is definitely better. Thanks!
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Alex Rogozhnikov]

Thanks for comments, I've fixed the named issues.

Code is python2&3 compatible, I aliased numpy and used better inversion.
Specially thanks for pointing at histogram equalization - I've added
example for images.
Probably some other 'visual' examples would help - I'll try to invent
something to other points, but this is not simple.

(I left %matplolib inline due to more appropriate rendering)

Alex.

2015-10-02 9:38 GMT+02:00 Alex Rogozhnikov
I would suggest
%matplotlib notebook
It will still have to a nice png, but you get an
interactive figure when it is live.
Amazing, thanks. I was using mpld3 for this.
(for some strange reason I need to put %matplotlib notebook
before each plot)
You should create a figure before each plot instead of putthon
%matplotlib notebook
plt.figure()
....
putthon == putting
The recommendation of inverting a permutation by
argsort'ing it, while it works, is suboptimal, as it takes
Actually, there is (later in post) a linear solution using
bincount, but your code is definitely better. Thanks!
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion