[Numpy-discussion] Inconsistent/unexpected indexing semantics

[Lluís Vilanova]

Hi,

TL;DR: There's a pending pull request deprecating some behaviour I find
unexpected. Does anyone object?

Some time ago I noticed that numpy yields unexpected results in some very
specific cases. An array can be used to index multiple elements of a single

a = np.arange(8).reshape((2,2,2))
a[ np.array([[0], [0]]) ]

array([[[[0, 1],
[2, 3]]],
[[[0, 1],
[2, 3]]]])

Nonetheless, if a list is used instead, it is (unexpectedly) transformed into a

a[ [[0], [0]] ]

array([[0, 1]])

a[ [0], [0] ]

array([[0, 1]])

a[( [0], [0] )]

array([[0, 1]])


I've been informed that there's a pending pull request that deprecates this
behaviour [1], which could in the future be reverted to what is expected (at
least what I expect) from the documents (except for an obscure note in [2]).

The discussion leading to this mail can be found here [3].

[1] https://github.com/numpy/numpy/pull/4434
[2] http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#advanced-indexing
[3] https://github.com/numpy/numpy/issues/6564


Thanks,
Lluis

[Sebastian Berg]

Post by Lluís Vilanova
Hi,
TL;DR: There's a pending pull request deprecating some behaviour I find
unexpected. Does anyone object?
Some time ago I noticed that numpy yields unexpected results in some very
specific cases. An array can be used to index multiple elements of a single

a = np.arange(8).reshape((2,2,2))
a[ np.array([[0], [0]]) ]

array([[[[0, 1],
[2, 3]]],
[[[0, 1],
[2, 3]]]])
Nonetheless, if a list is used instead, it is (unexpectedly) transformed into a

a[ [[0], [0]] ]

array([[0, 1]])

a[ [0], [0] ]

array([[0, 1]])

a[( [0], [0] )]

array([[0, 1]])
I've been informed that there's a pending pull request that deprecates this
behaviour [1], which could in the future be reverted to what is expected (at
least what I expect) from the documents (except for an obscure note in [2]).

Obviously, I am not against this ;). I have to admit it worries me a

Post by Lluís Vilanova

slice_object = [slice(None)] * 5
slice_object[2] = 3
arr[slice_object]

and all of this code (numpy also has a lot of it), will probably have to

Post by Lluís Vilanova

arr[tuple(slice_object)]

So the implication of this might actually be more farther reaching then
one might think at first; or at least require quite a lot of code to be
touched (inside numpy that is no problem, but outside).

- Sebastian

Post by Lluís Vilanova
The discussion leading to this mail can be found here [3].
[1] https://github.com/numpy/numpy/pull/4434
[2] http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#advanced-indexing
[3] https://github.com/numpy/numpy/issues/6564
Thanks,
Lluis

[Nathaniel Smith]

Post by Sebastian Berg

Post by Lluís Vilanova
Hi,
TL;DR: There's a pending pull request deprecating some behaviour I find
unexpected. Does anyone object?
Some time ago I noticed that numpy yields unexpected results in some very
specific cases. An array can be used to index multiple elements of a single

a = np.arange(8).reshape((2,2,2))
a[ np.array([[0], [0]]) ]

array([[[[0, 1],
[2, 3]]],
[[[0, 1],
[2, 3]]]])
Nonetheless, if a list is used instead, it is (unexpectedly)

transformed into a

Post by Sebastian Berg

Post by Lluís Vilanova

a[ [[0], [0]] ]

array([[0, 1]])

a[ [0], [0] ]

array([[0, 1]])

a[( [0], [0] )]

array([[0, 1]])
I've been informed that there's a pending pull request that deprecates this
behaviour [1], which could in the future be reverted to what is expected (at
least what I expect) from the documents (except for an obscure note in [2]).

Obviously, I am not against this ;). I have to admit it worries me a

Post by Lluís Vilanova

slice_object = [slice(None)] * 5
slice_object[2] = 3
arr[slice_object]

and all of this code (numpy also has a lot of it), will probably have to

Post by Lluís Vilanova

arr[tuple(slice_object)]

This seems like an improvement to me, so I'm +1 on deprecating. I agree
that it might be a very long time before we can actually change the
behavior though.

I think it would make sense to split it into two separate, parallel
deprecations: one for lists like in Lluís's example that are coerceable to
an integer array, and one for lists like in your example that contain
slices and stuff. The first case needs a FutureWarning, is incredibly
confusing, and is unlikely to be used on purpose; the second only needs a
DeprecationWarning, is less confusing, and is probably in broader use, so
might want a longer deprecation period.

-n