[Numpy-discussion] The --compilers flag is ignored for build_ext

[Andres Guzman-Ballen]

Hello Numpy community!


First time poster here. I was recommended by Charles Harris to present a question I have to you guys regarding using --compiler=intelem being ignored when I use build_ext.


I am trying to compile Numpy v1.11.2 that I am getting from https://pypi.python.org/packages/16/f5/b432f028134dd30cfbf6f21b8264a9938e5e0f75204e72453af08d67eb0b/numpy-1.11.2.tar.gz, using Python 2.7.12 on my Ubuntu 10.04 LTS Machine. I specifically want to compile this with the Intel compiler. I have sourced the compilervars.sh so the Intel compiler is in the path. This is the command that is failing:


'python setup.py build_ext --inplace --debug --force --compiler=intelem --fcompiler=intelem'.


The reason it fails is because I have added compiler flags to umath's configuration extension code in numpy/core/setup.py as extra_compiler_arg and they are specifically meant for Intel. Since the compiler setup is ignored (unlike in setup.py build, where config_cc is correctly called and takes care of this), it defaults to GCC. This is a snippet of what you get for 'build_ext':
running build_clib
running build_src
build_src
building py_modules sources
building library "npymath" sources
creating build
creating build/src.linux-x86_64-2.7
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using config


This is what you get for using 'build':
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
creating build
creating build/src.linux-x86_64-2.7
creating build/src.linux-x86_64-2.7/numpy
creating build/src.linux-x86_64-2.7/numpy/distutils
building library "npymath" sources
Found executable /localdisk/psxe_16/compilers_and_libraries_2016.3.210/linux/bin/intel64/icc
Could not locate executable ecc
customize IntelEM64TFCompiler
Found executable /localdisk/psxe_16/compilers_and_libraries_2016.3.210/linux/bin/intel64/ifort
customize IntelEM64TFCompiler using config


I do not experience this problem with 'python setup.py build --debug --force --compiler=intelem --fcompiler=intelem'. because the configuration function is correctly called here: https://github.intel.com/SAT/numpy/blob/develop/numpy/distutils/command/build.py#L11 (that is where I got the output from above). However, I do not see anything like this in build_ext: https://github.intel.com/SAT/numpy/blob/develop/numpy/distutils/command/build_ext.py#L34. This of course could be because build_ext is doing something else below in the run command on line 109.


This is most likely not the right answer, but what harm does this do?

diff --git a/numpy/distutils/command/build_ext.py b/numpy/distutils/command/build_ext.py
index 0fa52a2..ffa88ae 100644
--- a/numpy/distutils/command/build_ext.py
+++ b/numpy/distutils/command/build_ext.py
@@ -79,6 +79,8 @@ class build_ext (old_build_ext):
return


# Make sure that extension sources are complete.
+ self.run_command('config_cc')
+ self.run_command('config_fc')
self.run_command('build_src')


if self.distribution.has_c_libraries():


I am able to get build_ext to behave properly by adding this snippet, but I'm pretty sure this is the wrong approach because there seems to be code below line 109 in numpy/distutils/command/build_ext.py that sets compiler-related things up.


This is the ticket I created under Numpy's issues page: https://github.com/numpy/numpy/issues/8283. Any feedback is very well appreciated and I'd be more than happy to help contribute and get this fixed for the community. I don't want to just throw a problem at you guys and expect someone else to fix it for me. Thanks!


Andres