Discussion:
Please help - can't find any answer on pycuda._driver cuModuleLoadDataEx error
(too old to reply)
Hezy, Sharon
2017-12-25 15:38:35 UTC
Permalink
Hello,

I’m pretty familiar with CUDA (writing code since CUDA3.0), but PyCUDA is quite new for me.

I’ve been asked to configure our code that runs on CUDA 6.5, to run on GeForce GTX 1080 (compute capability 6.1).

OS is Windows 64bit, CUDA version – 6.5, GPU devices that should be supported: GeForce GTX 980Ti (compute capability 5.2) and GeForce GTX 1080 (comp.cap. 6.1), Python version – 2.7.

NVidia driver was updated (much after the CUDA), to support both the new 1080 and the old 980Ti cards.



Just to answer the question before it’s asked – there are some technical reasons that prevent us from moving now to CUDA 8.0 or 9.0 (it will be done some time later, but the new cards have to be supported today, with the old CUDA and all the existing code
).

The usual C/C++ code, compiled with nvcc with sm 52 (sm 50 is ok too), with CUDA6.5 – runs fine on both 980Ti and 1080.



The same “trick” (nvcc with sm 52 or 50), and compilation to .cubin files from python – gives the following error when trying to run:



pycuda._driver.LogicError: cuModuleLoadDataEx failed: invalid source –



The only guess I have is that the pycuda package (which was installed from binary distribution) - was linked with too old version of NVIDIA binaries (such as driver rt), and I need to recompile the pycuda sources with the current packages installed on my system.



Am I right? Or there is another explanation for this?

I’ve been looking for an answer in many blogs, but nobody describes my problem




Any help is appreciated!

Sharon.


---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
Andreas Kloeckner
2017-12-27 05:06:53 UTC
Permalink
Post by Hezy, Sharon
Hello,
I’m pretty familiar with CUDA (writing code since CUDA3.0), but PyCUDA is quite new for me.
I’ve been asked to configure our code that runs on CUDA 6.5, to run on GeForce GTX 1080 (compute capability 6.1).
OS is Windows 64bit, CUDA version – 6.5, GPU devices that should be supported: GeForce GTX 980Ti (compute capability 5.2) and GeForce GTX 1080 (comp.cap. 6.1), Python version – 2.7.
NVidia driver was updated (much after the CUDA), to support both the new 1080 and the old 980Ti cards.
Just to answer the question before it’s asked – there are some technical reasons that prevent us from moving now to CUDA 8.0 or 9.0 (it will be done some time later, but the new cards have to be supported today, with the old CUDA and all the existing code…).
The usual C/C++ code, compiled with nvcc with sm 52 (sm 50 is ok too), with CUDA6.5 – runs fine on both 980Ti and 1080.
pycuda._driver.LogicError: cuModuleLoadDataEx failed: invalid source –
The only guess I have is that the pycuda package (which was installed from binary distribution) - was linked with too old version of NVIDIA binaries (such as driver rt), and I need to recompile the pycuda sources with the current packages installed on my system.
Am I right? Or there is another explanation for this?
I’ve been looking for an answer in many blogs, but nobody describes my problem…
You could try using this branch, which adds support for using the NVRTC
API by way of a separate JIT class, DynamicSourceModule:

https://gitlab.tiker.net/inducer/pycuda/merge_requests/3

That might help you get around the compiler/driver mismatch you're
seeing.

Andreas

Loading...