[PyCUDA] mpi4py and GPUArray in Python 3

[Freddie Witherden]

I ran into issues when running the following gist written by Lev.
https://gist.github.com/lebedov/8514d3456a94a6c73e6d#file-mpi4py_pycuda_demo-py
Basically the code works fine under Python 2.7 but I got the following
error in Python 3.6 (both with openmpi 3.0.0 and mpi4py 3.0.0).
  File "mpi4py_pycuda_demo.py", line 64, in <module>
    comm.Recv([bufint(x_gpu), dtype_to_mpi(x_gpu.dtype)], source=0)
  File "mpi4py/MPI/Comm.pyx", line 283, in mpi4py.MPI.Comm.Recv
  File "mpi4py/MPI/msgbuffer.pxi", line 402, in mpi4py.MPI.message_p2p_recv
  File "mpi4py/MPI/msgbuffer.pxi", line 388, in
mpi4py.MPI._p_msg_p2p.for_recv
  File "mpi4py/MPI/msgbuffer.pxi", line 155, in mpi4py.MPI.message_simple
  File "mpi4py/MPI/msgbuffer.pxi", line 94, in mpi4py.MPI.message_basic
  File "mpi4py/MPI/asbuffer.pxi", line 308, in mpi4py.MPI.getbuffer
  File "mpi4py/MPI/asbuffer.pxi", line 135, in mpi4py.MPI.PyMPI_GetBuffer
BufferError: memoryview: underlying buffer is not writable
I am using 2017.1.1, so presumably the as_buffer method should already
return a writable buffer from a fix two years ago. Any suggestion is
appreciated!

The fix was incomplete. If I recall there are two places where it need
be applied, and it somehow only ended up being fixed in one of them.

As a workaround:

from ctypes import c_int, c_ssize_t, c_void_p, pythonapi, py_object

_make_pybuf = pythonapi.PyMemoryView_FromMemory
_make_pybuf.argtypes = [c_void_p, c_ssize_t, c_int]
_make_pybuf.restype = py_object

buf = _make_pybuf(my_alloc, nbytes, 0x200)

Regards, Freddie.

Thanks,
Yiyin
_______________________________________________
PyCUDA mailing list
https://lists.tiker.net/listinfo/pycuda

[Yiyin Zhou]

It seems that I was looking at a wrong line and referred to the as_buffer
method for the pointer_holder_base class that was fixed.
The same fix is needed for the device_allocation class as well, I believe:
https://github.com/inducer/pycuda/blob/master/src/cpp/cuda.hpp#L1559

Yiyin

Hi,
I ran into issues when running the following gist written by Lev.
https://gist.github.com/lebedov/8514d3456a94a6c73e6d#
file-mpi4py_pycuda_demo-py
Basically the code works fine under Python 2.7 but I got the following
error in Python 3.6 (both with openmpi 3.0.0 and mpi4py 3.0.0).
File "mpi4py_pycuda_demo.py", line 64, in <module>
comm.Recv([bufint(x_gpu), dtype_to_mpi(x_gpu.dtype)], source=0)
File "mpi4py/MPI/Comm.pyx", line 283, in mpi4py.MPI.Comm.Recv
File "mpi4py/MPI/msgbuffer.pxi", line 402, in mpi4py.MPI.message_p2p_recv
File "mpi4py/MPI/msgbuffer.pxi", line 388, in
mpi4py.MPI._p_msg_p2p.for_recv
File "mpi4py/MPI/msgbuffer.pxi", line 155, in mpi4py.MPI.message_simple
File "mpi4py/MPI/msgbuffer.pxi", line 94, in mpi4py.MPI.message_basic
File "mpi4py/MPI/asbuffer.pxi", line 308, in mpi4py.MPI.getbuffer
File "mpi4py/MPI/asbuffer.pxi", line 135, in mpi4py.MPI.PyMPI_GetBuffer
BufferError: memoryview: underlying buffer is not writable
I am using 2017.1.1, so presumably the as_buffer method should already
return a writable buffer from a fix two years ago. Any suggestion is
appreciated!
Thanks,
Yiyin