[Numpy-discussion] f2py: ram usage

[Vasco Gervasi]

Hi all,
I am trying to write some code to do calculation onto an array: for each
row I need to do some computation and have a number as return.
To speed up the process I wrote a fortran subroutine that is called from
python [using f2py] for each row of the array, so the input of this
subroutine is a row and the output is a number.
This method works but I saw some speed advantage if I pass the entire array
to fortran and then, inside fortran, call the subroutine that does the
math; so in this case I pass an array and return a vector.
But I noticed that when python pass the array to fortran, the array is
copied and the RAM usage double.
Is there a way to "move" the array to fortran, I don't care if the array is
lost after the call to fortran.
The pyd module is generated using: python f2py.py -c --opt="-ffree-form
-Ofast" -m F2PYMOD F2PYMOD.f90

Thanks
Vasco

[Sebastian Berg]

Post by Vasco Gervasi
Hi all,
I am trying to write some code to do calculation onto an array: for
each row I need to do some computation and have a number as return.
To speed up the process I wrote a fortran subroutine that is called
from python [using f2py] for each row of the array, so the input of
this subroutine is a row and the output is a number.
This method works but I saw some speed advantage if I pass the entire
array to fortran and then, inside fortran, call the subroutine that
does the math; so in this case I pass an array and return a vector.
But I noticed that when python pass the array to fortran, the array
is copied and the RAM usage double.

I expect that the fortran code needs your arrays to be fortran
contiguous, so the wrappers need to copy them.

The easiest solution may be to create your array in python with the
`order="F"` flag. NumPy will have a tendency to prefer C-order and uses
it as default though when doing something with an "F" ordered array.

That said, I have never used f2py, so these are just well founded
guesses.

- Sebastian

Post by Vasco Gervasi
Is there a way to "move" the array to fortran, I don't care if the
array is lost after the call to fortran.
The pyd module is generated using: python f2py.py -c --opt="-ffree
-form -Ofast" -m F2PYMOD F2PYMOD.f90
Thanks
Vasco
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[George Nurser]

Yes, f2py is probably copying the arrays; you can check this by appending
-DF2PY_REPORT_ON_ARRAY_COPY=1 to your call to f2py.

I normally prefer to keep the numpy arrays C-order (most efficient for
numpy) and simply pass the array transpose to the f2py-ized fortran routine.
This means that the fortran array indices are reversed, but this is the
most natural way in any case.

--George Nurser

Post by Sebastian Berg

Post by Vasco Gervasi
Hi all,
I am trying to write some code to do calculation onto an array: for
each row I need to do some computation and have a number as return.
To speed up the process I wrote a fortran subroutine that is called
from python [using f2py] for each row of the array, so the input of
this subroutine is a row and the output is a number.
This method works but I saw some speed advantage if I pass the entire
array to fortran and then, inside fortran, call the subroutine that
does the math; so in this case I pass an array and return a vector.
But I noticed that when python pass the array to fortran, the array
is copied and the RAM usage double.

I expect that the fortran code needs your arrays to be fortran
contiguous, so the wrappers need to copy them.
The easiest solution may be to create your array in python with the
`order="F"` flag. NumPy will have a tendency to prefer C-order and uses
it as default though when doing something with an "F" ordered array.
That said, I have never used f2py, so these are just well founded
guesses.
- Sebastian

Post by Vasco Gervasi
Is there a way to "move" the array to fortran, I don't care if the
array is lost after the call to fortran.
The pyd module is generated using: python f2py.py -c --opt="-ffree
-form -Ofast" -m F2PYMOD F2PYMOD.f90
Thanks
Vasco
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Vasco Gervasi]

Using order='F' solved the problem.

Thanks for reply.