[Numpy-discussion] Software Capabilities of NumPy in Our Tensor Survey Paper

[Li Jiajia]

Hi all,
I’m a PhD student in Georgia Tech. Recently, we’re working on a survey paper about tensor algorithms: basic tensor operations, tensor decomposition and some tensor applications. We are making a table to compare the capabilities of different software and planning to include NumPy. We’d like to make sure these parameters are correct to make a fair compare. Although we have looked into the related documents, please help us to confirm these. Besides, if you think there are more features of your software and a more preferred citation, please let us know. We’ll consider to update them. We want to show NumPy supports tensors, and we also include "scikit-tensor” in our survey, which is based on NumPy.
Please let me know any confusion or any advice!
Thanks a lot! :-)

Notice:
1. “YES/NO” to show whether or not the software supports the operation or has the feature.
2. “?” means we’re not sure of the feature, and please help us out.
3. “Tensor order” means the maximum number of tensor dimensions that users can do with this software.
4. For computational cores,
1) "Element-wise Tensor Operation (A * B)” includes element-wise add/minus/multiply/divide, also Kronecker, outer and Katri-Rao products. If the software contains one of them, we mark “YES”.
2) “TTM” means tensor-times-matrix multiplication. We distinguish TTM from tensor contraction. If the software includes tensor contraction, it can also support TTM.
3) For “MTTKRP”, we know most software can realize it through the above two operations. We mark it “YES”, only if an specified optimization for the whole operation.

<> <>Software Name <>
NumPy

Computational Cores

Element-wise Tensor Operation (A * B)

YES

Tensor Contraction (A Xmn B)

NO

TTM ( A Xn B)

NO

Matriced Tensor Times Khatri-Rao Product (MTTKRP)

NO

Tensor Decomposition

CP

NO

Tucker

NO

Hierarchical Tucker (HT)

NO

Tensor Train (TT)

NO

Tensor Features

Tensor Order

Arbitrary

Dense Tensors

YES

Sparse Tensors

NO ?

Parallelized

NO ?

Software Information

Application Domain

General

Programming Environment

Python

Latest Version

1.10.4
Release Date

2016

Citation:
1. AN DER WALT, S., COLBERT, S., AND VAROQUAUX, G. The NumPy array: A structure for efficient numerical computation. Computing in Science Engineering 13, 2 (March 2011), 22–30.
2. OLIPHANT, T. E. Python for scientific computing. Computing in Science Engineering 9, 3 (May 2007), 10–20.
3. NumPy (Version1.10.4).Available from http://www.numpy.org, Jan <http://www.numpy.org, Jan> 2016.

Best regards!
Jiajia Li

------------------------------------------
E-mail: ***@gatech.edu
Tel: [phone]
Computational Science & Engineering
Georgia Institute of Technology

[Bryan Van de Ven]

Your first citation is incorrect. It is "VAN DER WALT" (missing V in yours)

Bryan

Post by Li Jiajia
Hi all,
I’m a PhD student in Georgia Tech. Recently, we’re working on a survey paper about tensor algorithms: basic tensor operations, tensor decomposition and some tensor applications. We are making a table to compare the capabilities of different software and planning to include NumPy. We’d like to make sure these parameters are correct to make a fair compare. Although we have looked into the related documents, please help us to confirm these. Besides, if you think there are more features of your software and a more preferred citation, please let us know. We’ll consider to update them. We want to show NumPy supports tensors, and we also include "scikit-tensor” in our survey, which is based on NumPy.
Please let me know any confusion or any advice!
Thanks a lot! :-)
1. “YES/NO” to show whether or not the software supports the operation or has the feature.
2. “?” means we’re not sure of the feature, and please help us out.
3. “Tensor order” means the maximum number of tensor dimensions that users can do with this software.
4. For computational cores,
1) "Element-wise Tensor Operation (A * B)” includes element-wise add/minus/multiply/divide, also Kronecker, outer and Katri-Rao products. If the software contains one of them, we mark “YES”.
2) “TTM” means tensor-times-matrix multiplication. We distinguish TTM from tensor contraction. If the software includes tensor contraction, it can also support TTM.
3) For “MTTKRP”, we know most software can realize it through the above two operations. We mark it “YES”, only if an specified optimization for the whole operation.
Software Name
NumPy
Computational Cores
Element-wise Tensor Operation (A * B)
YES
Tensor Contraction (A Xmn B)
NO
TTM ( A Xn B)
NO
Matriced Tensor Times Khatri-Rao Product (MTTKRP)
NO
Tensor Decomposition
CP
NO
Tucker
NO
Hierarchical Tucker (HT)
NO
Tensor Train (TT)
NO
Tensor Features
Tensor Order
Arbitrary
Dense Tensors
YES
Sparse Tensors
NO ?
Parallelized
NO ?
Software Information
Application Domain
General
Programming Environment
Python
Latest Version
1.10.4
Release Date
2016
1. AN DER WALT, S., COLBERT, S., AND VAROQUAUX, G. The NumPy array: A structure for efficient numerical computation. Computing in Science Engineering 13, 2 (March 2011), 22–30.
2. OLIPHANT, T. E. Python for scientific computing. Computing in Science Engineering 9, 3 (May 2007), 10–20.
3. NumPy (Version1.10.4).Available from http://www.numpy.org, Jan 2016.
Best regards!
Jiajia Li
------------------------------------------
Tel: [phone]
Computational Science & Engineering
Georgia Institute of Technology
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Nathaniel Smith]

Post by Li Jiajia
Hi all,
I’m a PhD student in Georgia Tech. Recently, we’re working on a survey

paper about tensor algorithms: basic tensor operations, tensor
decomposition and some tensor applications. We are making a table to
compare the capabilities of different software and planning to include
NumPy. We’d like to make sure these parameters are correct to make a fair
compare. Although we have looked into the related documents, please help us
to confirm these. Besides, if you think there are more features of your
software and a more preferred citation, please let us know. We’ll consider
to update them. We want to show NumPy supports tensors, and we also include
"scikit-tensor” in our survey, which is based on NumPy.

Post by Li Jiajia
Please let me know any confusion or any advice!
Thanks a lot! :-)
1. “YES/NO” to show whether or not the software supports the operation or

has the feature.

Post by Li Jiajia
2. “?” means we’re not sure of the feature, and please help us out.
3. “Tensor order” means the maximum number of tensor dimensions that

users can do with this software.

Post by Li Jiajia
4. For computational cores,
1) "Element-wise Tensor Operation (A * B)” includes element-wise

add/minus/multiply/divide, also Kronecker, outer and Katri-Rao products. If
the software contains one of them, we mark “YES”.

Post by Li Jiajia
2) “TTM” means tensor-times-matrix multiplication. We distinguish TTM

from tensor contraction. If the software includes tensor contraction, it
can also support TTM.

Post by Li Jiajia
3) For “MTTKRP”, we know most software can realize it through the above

two operations. We mark it “YES”, only if an specified optimization for the
whole operation.

NumPy has support for working with multidimensional tensors, if you like,
but it doesn't really use the tensor language and notation (preferring
instead to think in terms of "arrays" as a somewhat more computationally
focused and less mathematically focused conceptual framework).

Which is to say that I actually have no idea what all those jargon terms
you're asking about mean :-) I am suspicious that NumPy supports more of
those operations than you have marked, just under different names/notation,
but really can't tell either way for sure without knowing what exactly they
are.

(It is definitely correct though that NumPy includes no support for sparse
tensors, and NumPy itself is not multi-threaded beyond what we get for free
through the BLAS, though there are external libraries that can perform
multi-threaded computations on top of data stored in numpy arrays.)

-n

[Robert Kern]

Post by Nathaniel Smith

Post by Li Jiajia
Hi all,
I’m a PhD student in Georgia Tech. Recently, we’re working on a survey

paper about tensor algorithms: basic tensor operations, tensor
decomposition and some tensor applications. We are making a table to
compare the capabilities of different software and planning to include
NumPy. We’d like to make sure these parameters are correct to make a fair
compare. Although we have looked into the related documents, please help us
to confirm these. Besides, if you think there are more features of your
software and a more preferred citation, please let us know. We’ll consider
to update them. We want to show NumPy supports tensors, and we also include
"scikit-tensor” in our survey, which is based on NumPy.

Post by Nathaniel Smith

Post by Li Jiajia
Please let me know any confusion or any advice!
Thanks a lot! :-)
1. “YES/NO” to show whether or not the software supports the operation

or has the feature.

Post by Nathaniel Smith

Post by Li Jiajia
2. “?” means we’re not sure of the feature, and please help us out.
3. “Tensor order” means the maximum number of tensor dimensions that

users can do with this software.

Post by Nathaniel Smith

Post by Li Jiajia
4. For computational cores,
1) "Element-wise Tensor Operation (A * B)” includes element-wise

add/minus/multiply/divide, also Kronecker, outer and Katri-Rao products. If
the software contains one of them, we mark “YES”.

Post by Nathaniel Smith

Post by Li Jiajia
2) “TTM” means tensor-times-matrix multiplication. We distinguish TTM

from tensor contraction. If the software includes tensor contraction, it
can also support TTM.

Post by Nathaniel Smith

Post by Li Jiajia
3) For “MTTKRP”, we know most software can realize it through the above

two operations. We mark it “YES”, only if an specified optimization for the
whole operation.

Post by Nathaniel Smith
NumPy has support for working with multidimensional tensors, if you like,

but it doesn't really use the tensor language and notation (preferring
instead to think in terms of "arrays" as a somewhat more computationally
focused and less mathematically focused conceptual framework).

Post by Nathaniel Smith
Which is to say that I actually have no idea what all those jargon terms

you're asking about mean :-) I am suspicious that NumPy supports more of
those operations than you have marked, just under different names/notation,
but really can't tell either way for sure without knowing what exactly they
are.

In particular check if your operations can be expressed with einsum()

http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.einsum.html

--
Robert Kern

[Yuxiang Wang]

I echo with Robert that the contraction can be done with np.einsum().
Also, check out the np.tensordot() as well - it can also be used to
perform contraction.

Shawn

Post by Robert Kern

Post by Nathaniel Smith

Post by Li Jiajia
Hi all,
I’m a PhD student in Georgia Tech. Recently, we’re working on a survey
paper about tensor algorithms: basic tensor operations, tensor decomposition
and some tensor applications. We are making a table to compare the
capabilities of different software and planning to include NumPy. We’d like
to make sure these parameters are correct to make a fair compare. Although
we have looked into the related documents, please help us to confirm these.
Besides, if you think there are more features of your software and a more
preferred citation, please let us know. We’ll consider to update them. We
want to show NumPy supports tensors, and we also include "scikit-tensor” in
our survey, which is based on NumPy.
Please let me know any confusion or any advice!
Thanks a lot! :-)
1. “YES/NO” to show whether or not the software supports the operation
or has the feature.
2. “?” means we’re not sure of the feature, and please help us out.
3. “Tensor order” means the maximum number of tensor dimensions that
users can do with this software.
4. For computational cores,
1) "Element-wise Tensor Operation (A * B)” includes element-wise
add/minus/multiply/divide, also Kronecker, outer and Katri-Rao products. If
the software contains one of them, we mark “YES”.
2) “TTM” means tensor-times-matrix multiplication. We distinguish TTM
from tensor contraction. If the software includes tensor contraction, it can
also support TTM.
3) For “MTTKRP”, we know most software can realize it through the above
two operations. We mark it “YES”, only if an specified optimization for the
whole operation.

NumPy has support for working with multidimensional tensors, if you like,
but it doesn't really use the tensor language and notation (preferring
instead to think in terms of "arrays" as a somewhat more computationally
focused and less mathematically focused conceptual framework).
Which is to say that I actually have no idea what all those jargon terms
you're asking about mean :-) I am suspicious that NumPy supports more of
those operations than you have marked, just under different names/notation,
but really can't tell either way for sure without knowing what exactly they
are.

In particular check if your operations can be expressed with einsum()
http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.einsum.html
--
Robert Kern
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Stephan Hoyer]

Robert beat me to it on einsum, but also check tensordot for general tensor contraction.

Post by Li Jiajia

Post by Li Jiajia
Hi all,
I’m a PhD student in Georgia Tech. Recently, we’re working on a survey

paper about tensor algorithms: basic tensor operations, tensor
decomposition and some tensor applications. We are making a table to
compare the capabilities of different software and planning to include
NumPy. We’d like to make sure these parameters are correct to make a fair
compare. Although we have looked into the related documents, please help us
to confirm these. Besides, if you think there are more features of your
software and a more preferred citation, please let us know. We’ll consider
to update them. We want to show NumPy supports tensors, and we also include
"scikit-tensor” in our survey, which is based on NumPy.

Post by Li Jiajia
Please let me know any confusion or any advice!
Thanks a lot! :-)
1. “YES/NO” to show whether or not the software supports the operation or

has the feature.

Post by Li Jiajia
2. “?” means we’re not sure of the feature, and please help us out.
3. “Tensor order” means the maximum number of tensor dimensions that

users can do with this software.

Post by Li Jiajia
4. For computational cores,
1) "Element-wise Tensor Operation (A * B)” includes element-wise

add/minus/multiply/divide, also Kronecker, outer and Katri-Rao products. If
the software contains one of them, we mark “YES”.

Post by Li Jiajia
2) “TTM” means tensor-times-matrix multiplication. We distinguish TTM

from tensor contraction. If the software includes tensor contraction, it
can also support TTM.

Post by Li Jiajia
3) For “MTTKRP”, we know most software can realize it through the above

two operations. We mark it “YES”, only if an specified optimization for the
whole operation.
NumPy has support for working with multidimensional tensors, if you like,
but it doesn't really use the tensor language and notation (preferring
instead to think in terms of "arrays" as a somewhat more computationally
focused and less mathematically focused conceptual framework).
Which is to say that I actually have no idea what all those jargon terms
you're asking about mean :-) I am suspicious that NumPy supports more of
those operations than you have marked, just under different names/notation,
but really can't tell either way for sure without knowing what exactly they
are.
(It is definitely correct though that NumPy includes no support for sparse
tensors, and NumPy itself is not multi-threaded beyond what we get for free
through the BLAS, though there are external libraries that can perform
multi-threaded computations on top of data stored in numpy arrays.)
-n