Hi Lev,
Thanks for the reply!
My GPUs do support GPUDirect. And I've tested using the "simpleP2P" from
cuda samples.
I've been trying a little bit on that but without success. I'm new to
pycuda and multiprocessing, excuse me if I ask dumb questions.
If I understand it right, I need to do pycuda.driver.init() and
make_context() inside each process.
But to use pycuda.driver.memcpy_peer, I need to pass the context defined in
one process to another. But when I try to pass the context with Pipe or
Queue from multiprocessing, it returns pickling error.
Is this the right way of doing it? assuming pickling error can be solved.
I've attached my toy example code, if that could explain my problem better.
Thanks!
import atexit
import numpy as np
import pycuda.driver as drv
import pycuda.gpuarray as gpuarray
from multiprocessing import Process, Queue
def gen_gpuarray(obj_queue):
# generate a gpuarray to pass
drv.init()
dev0 = drv.Device(1)
ctx0 = dev0.make_context()
atexit.register(ctx0.pop)
ctx0.push()
obj_queue.put(ctx0)
ctx1 = obj_queue.get()
ctx0.enable_peer_access(ctx1)
x = np.random.rand(1000)
x_gpu = gpuarray.to_gpu(x)
obj_queue.put(x_gpu)
ctx0.pop()
def get_gpuarray(obj_queue):
# try to receive the gpu array
drv.init()
dev1 = drv.Device(2)
ctx1 = dev1.make_context()
atexit.register(ctx1.pop)
ctx1.push()
ctx0 = obj_queue.get()
obj_queue.put(ctx1)
ctx1.enable_peer_access(ctx0)
to_get_gpu = obj_queue.get()
y_gpu = gpuarray.zeros_like(to_get_gpu)
drv.memcpy_peer(y_gpu.ptr, to_get_gpu.ptr,
to_get_gpu.dtype.itemsize * to_get_gpu.size,
ctx1, ctx0)
ctx1.pop()
if __name__ == '__main__':
q = Queue()
process_gen = Process(target=gen_gpuarray, args=(q, ))
process_get = Process(target=get_gpuarray, args=(q, ))
process_gen.start()
process_get.start()
process_gen.join()
process_get.join()
Post by Gavin Weiguang DingPost by Gavin Weiguang DingHi,
I'm trying to do p2p communication between 2 GPUs without going through
the
Post by Gavin Weiguang DingCPU memory.
And I need to communicate between 2 processes (due to one process in
Theano
Post by Gavin Weiguang Dingcan only use 1 GPU), is that possible with pycuda? or
specifically pycuda.driver.memcpy_peer?
Yes, but your GPU needs to support GPUDirect peer-to-peer communication.
Assuming that you have the appropriate hardware, you can also use mpi4py to
transfer data between GPUs if it has been compiled against an MPI implementation
that supports GPU-to-GPU communication (e.g., OpenMPI 1.8, MVAPICH2 2.0).
--
Lev Givon
Bionet Group | Neurokernel Project
http://www.columbia.edu/~lev/
http://lebedov.github.io/
http://neurokernel.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.tiker.net/pipermail/pycuda/attachments/20141006/6e9601ba/attachment.html>