[Numpy-discussion] Possible to pickle new state in NDArray subclasses?

[Stuart Reynolds]

I'm trying to subclass an NDArray as shown here:
https://docs.scipy.org/doc/numpy/user/basics.subclassing.html

My problem is that when I save the new class' state with pickle, the new
attributes are lost. I don't seem to be able to override __getstate__ or
__setstate__ to achieve this?

Is it possible to allow new state to serialized when overriding an NDArray?

In my example below, __setstate__ gets called by pickle but not
__getstate__.
In the final line, a RealisticInfoArray has been deserialized, but it has
no .info attribute.

----

import cPickle as pickle
import numpy as np

class RealisticInfoArray(np.ndarray):
def __new__(cls, arr, info):
obj = np.asarray(arr).view(cls)
obj.info = info
return obj

def __array_finalize__(self, obj):
if obj is None: return
self.info = getattr(obj,"info",None)

def __setstate__(self, *args):
print "SET"
return np.ndarray.__setstate__(self,*args)

def __getstate__(self):
print "GET"
assert False, "EXPLODE"
return np.ndarray.__getstate__(self)

arr = np.zeros((2,3), int)
arr = RealisticInfoArray(arr, "blarg")
print arr.info
arr2 = pickle.loads(pickle.dumps(arr))
print arr2.info # no .info attribute!

[Nathan Goldbaum]

I'm able to do this in my ndarrary subclass using __reduce__ and
__setstate__:

https://bitbucket.org/yt_analysis/yt/src/yt/yt/units/yt_array.py#yt_array.py-1250

Here it's being used to save the unit information into the pickle for a
unit-aware ndarray subclass.

Post by Stuart Reynolds
https://docs.scipy.org/doc/numpy/user/basics.subclassing.html
My problem is that when I save the new class' state with pickle, the new
attributes are lost. I don't seem to be able to override __getstate__ or
__setstate__ to achieve this?
Is it possible to allow new state to serialized when overriding an NDArray?
In my example below, __setstate__ gets called by pickle but not
__getstate__.
In the final line, a RealisticInfoArray has been deserialized, but it has
no .info attribute.
----
import cPickle as pickle
import numpy as np
obj = np.asarray(arr).view(cls)
obj.info = info
return obj
if obj is None: return
self.info = getattr(obj,"info",None)
print "SET"
return np.ndarray.__setstate__(self,*args)
print "GET"
assert False, "EXPLODE"
return np.ndarray.__getstate__(self)
arr = np.zeros((2,3), int)
arr = RealisticInfoArray(arr, "blarg")
print arr.info
arr2 = pickle.loads(pickle.dumps(arr))
print arr2.info # no .info attribute!
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Stuart Reynolds]

Works great! Thank you.

Post by Nathan Goldbaum
I'm able to do this in my ndarrary subclass using __reduce__ and
https://bitbucket.org/yt_analysis/yt/src/yt/yt/units/
yt_array.py#yt_array.py-1250
Here it's being used to save the unit information into the pickle for a
unit-aware ndarray subclass.
On Wed, Dec 14, 2016 at 1:45 PM, Stuart Reynolds <

Post by Stuart Reynolds
https://docs.scipy.org/doc/numpy/user/basics.subclassing.html
My problem is that when I save the new class' state with pickle, the new
attributes are lost. I don't seem to be able to override __getstate__ or
__setstate__ to achieve this?
Is it possible to allow new state to serialized when overriding an NDArray?
In my example below, __setstate__ gets called by pickle but not
__getstate__.
In the final line, a RealisticInfoArray has been deserialized, but it has
no .info attribute.
----
import cPickle as pickle
import numpy as np
obj = np.asarray(arr).view(cls)
obj.info = info
return obj
if obj is None: return
self.info = getattr(obj,"info",None)
print "SET"
return np.ndarray.__setstate__(self,*args)
print "GET"
assert False, "EXPLODE"
return np.ndarray.__getstate__(self)
arr = np.zeros((2,3), int)
arr = RealisticInfoArray(arr, "blarg")
print arr.info
arr2 = pickle.loads(pickle.dumps(arr))
print arr2.info # no .info attribute!
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion