[PyCUDA] (no subject)

[David A. Markowitz]

Hi, thanks for the quick reply (and good advice!). I wiped my cuda 6.5
installation and reinstalled from scratch. nvcc now works when called from
the command line on simple CUDA samples. It compiles for my GPU's
architecture (3.5) by default, so PATH and LD_LIBRARY_PATH are definitely
configured correctly.

I also wiped PyCUDA and reinstalled from scratch, per the instructions for
Ubuntu 14.04 64 bit on the PyCUDA Installation page. No errors during this
process, and I can successfully import pycuda.autoinit. However, now when I
try to run any of the PyCUDA examples, I get the following error:

nvcc fatal : Path to libdevice library not specified

Since I do not encounter this error when compiling CUDA samples or my own
CUDA code with nvcc, my guess is that my environment variables aren't being
seen by PyCUDA. I googled this error and found a few threads on the
subject, but no effective solutions.

I was wondering if I could trouble this list for a pointer or two?
Hopefully there's a quick fix.

Many thanks,

-David

Hi, I just installed PyCUDA, but test_driver.py crashes with the
CompileError: nvcc compilation of /tmp/tmpNht4bp/kernel.cu failed
[command: nvcc --cubin -arch sm_35

-I/usr/local/lib/python2.7/dist-packages/pycuda-2014.1-py2.7-linux-x86_64.egg/pycuda/cuda

kernel.cu]
/usr/bin/../nvvm/libdevice/libdevice.compute_35.10.bc No such
file or directory ]

This looks like nvcc is unable to find its own parts. Does nvcc work
when called from the command line on a simple CUDA sample? (My guess is
no.)
Andreas

[Andreas Kloeckner]

Post by David A. Markowitz
Hi, thanks for the quick reply (and good advice!). I wiped my cuda 6.5
installation and reinstalled from scratch. nvcc now works when called from
the command line on simple CUDA samples. It compiles for my GPU's
architecture (3.5) by default, so PATH and LD_LIBRARY_PATH are definitely
configured correctly.
I also wiped PyCUDA and reinstalled from scratch, per the instructions for
Ubuntu 14.04 64 bit on the PyCUDA Installation page. No errors during this
process, and I can successfully import pycuda.autoinit. However, now when I
nvcc fatal : Path to libdevice library not specified
Since I do not encounter this error when compiling CUDA samples or my own
CUDA code with nvcc, my guess is that my environment variables aren't being
seen by PyCUDA. I googled this error and found a few threads on the
subject, but no effective solutions.
I was wondering if I could trouble this list for a pointer or two?
Hopefully there's a quick fix.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=725649

suggests that you may be able to tweak /etc/nvcc.profile.

HTH,
Andreas

[David A. Markowitz]

Thanks again, Andreas. I'm really looking forward to getting started with
PyCUDA.

Unfortunately, I've already tried your suggested approach (updating
nvcc.profile with NVVMIR_LIBRARY_DIR = /usr/local/cuda-6.5/nvvm/libdevice,
which contains libdevice.compute_35.10.bc in my installation), but it did
not solve the problem. When I asked NVIDIA about this directly (hoping not
to bother you further), they told me I should never need to modify
nvcc.profile under any circumstances, which wasn't very helpful.

Is there some other line in nvcc.profile that I can modify so that nvcc
will be able to find the appropriate libdevice library when called by
PyCUDA's compiler script? e.g. perhaps I could append something to the
INCLUDES or LIBRARIES variables?

Is there an easy way for me to see which environment variables are
available when PyCUDA's compiler.py code calls nvcc? This might help
diagnose the problem.

Thanks,

-David

Post by Andreas Kloeckner
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=725649
suggests that you may be able to tweak /etc/nvcc.profile.
HTH,
Andreas

[Andreas Kloeckner]

Post by David A. Markowitz
Thanks again, Andreas. I'm really looking forward to getting started with
PyCUDA.
Unfortunately, I've already tried your suggested approach (updating
nvcc.profile with NVVMIR_LIBRARY_DIR = /usr/local/cuda-6.5/nvvm/libdevice,
which contains libdevice.compute_35.10.bc in my installation), but it did
not solve the problem. When I asked NVIDIA about this directly (hoping not
to bother you further), they told me I should never need to modify
nvcc.profile under any circumstances, which wasn't very helpful.
Is there some other line in nvcc.profile that I can modify so that nvcc
will be able to find the appropriate libdevice library when called by
PyCUDA's compiler script? e.g. perhaps I could append something to the
INCLUDES or LIBRARIES variables?
Is there an easy way for me to see which environment variables are
available when PyCUDA's compiler.py code calls nvcc? This might help
diagnose the problem.

The call happens here:

https://github.com/inducer/pycuda/blob/master/pycuda/compiler.py#L114

using this:

https://github.com/inducer/pytools/blob/master/pytools/prefork.py#L34

https://docs.python.org/2.7/library/subprocess.html

says that processes should inherit the parent's environment unless
otherwise specified (and it isn't), so it's not clear to me what would
override your variable... You can also use the 'keep=True' flag and try
and run nvcc yourself in the temp directory that PyCUDA creates. That's
perhaps the best way of figuring out what's up.

Andreas

[David A. Markowitz]

Many thanks Andreas, I've solved the problem now. While digging through the
compiler.py code, I noticed a check for the PYCUDA_DEFAULT_NVCC_FLAGS
environment variable, which is then passed to nvcc. Ultimately I was able
to solve my problem by putting a file in /etc/profile.d/ with the contents:

export PYCUDA_DEFAULT_NVCC_FLAGS="--dont-use-profile
-ldir=/usr/local/cuda/nvvm/libdevice"

This offers a simple but effective way for PyCUDA to point nvcc in the
right direction.

I'm still not sure why I wasn't able to fix the "libdevice library not
found" error by modifying nvcc.profile directly, but the above is probably
a better solution regardless, because it doesn't require modifications to a
file that affects nvcc's operation in all use cases (i.e. even when it's
not called by PyCUDA). Since I've only encountered this problem using
PyCUDA, it makes sense that the solution should only kick in when nvcc is
called by PyCUDA.

I'm really looking forward to using this software package. Thank you for
all of your hard work putting it together!

-David

Post by David A. Markowitz

Post by David A. Markowitz
Thanks again, Andreas. I'm really looking forward to getting started with
PyCUDA.
Unfortunately, I've already tried your suggested approach (updating
nvcc.profile with NVVMIR_LIBRARY_DIR =

/usr/local/cuda-6.5/nvvm/libdevice,

Post by David A. Markowitz
which contains libdevice.compute_35.10.bc in my installation), but it did
not solve the problem. When I asked NVIDIA about this directly (hoping

not

Post by David A. Markowitz
to bother you further), they told me I should never need to modify
nvcc.profile under any circumstances, which wasn't very helpful.
Is there some other line in nvcc.profile that I can modify so that nvcc
will be able to find the appropriate libdevice library when called by
PyCUDA's compiler script? e.g. perhaps I could append something to the
INCLUDES or LIBRARIES variables?
Is there an easy way for me to see which environment variables are
available when PyCUDA's compiler.py code calls nvcc? This might help
diagnose the problem.

https://github.com/inducer/pycuda/blob/master/pycuda/compiler.py#L114
https://github.com/inducer/pytools/blob/master/pytools/prefork.py#L34
https://docs.python.org/2.7/library/subprocess.html
says that processes should inherit the parent's environment unless
otherwise specified (and it isn't), so it's not clear to me what would
override your variable... You can also use the 'keep=True' flag and try
and run nvcc yourself in the temp directory that PyCUDA creates. That's
perhaps the best way of figuring out what's up.
Andreas

[Andreas Kloeckner]

Post by David A. Markowitz
Many thanks Andreas, I've solved the problem now. While digging through the
compiler.py code, I noticed a check for the PYCUDA_DEFAULT_NVCC_FLAGS
environment variable, which is then passed to nvcc. Ultimately I was able
export PYCUDA_DEFAULT_NVCC_FLAGS="--dont-use-profile
-ldir=/usr/local/cuda/nvvm/libdevice"
This offers a simple but effective way for PyCUDA to point nvcc in the
right direction.
I'm still not sure why I wasn't able to fix the "libdevice library not
found" error by modifying nvcc.profile directly, but the above is probably
a better solution regardless, because it doesn't require modifications to a
file that affects nvcc's operation in all use cases (i.e. even when it's
not called by PyCUDA). Since I've only encountered this problem using
PyCUDA, it makes sense that the solution should only kick in when nvcc is
called by PyCUDA.
I'm really looking forward to using this software package. Thank you for
all of your hard work putting it together!

Glad to hear you got things to work!

Andreas