multiprocessing - Local variable not updated in a loop in the same way as shared memory objects in Python -
in following python code, multiprocessing module starts 3 processes print out values of 1 local variable , 2 multiprocessing shared memory objects.
import multiprocessing mp import os,time # local variable count = 0 # shared memory objects (int , array) scalar = mp.value('i', 0) vector = mp.array('d', 3) def showdata(label, val, arr): print(label, "==> pid:", os.getpid(), ", count:", count, ", int:", val.value, ", arr:", list(arr)) ps = [] in range(3): count += 1 scalar.value += 1 vector[i] += 1 p=mp.process(target=showdata, args=(('process %s' % i), scalar, vector)) p.start() ps.append(p) # time.sleep(.1) # block main thread until processes have finished... p in ps: p.join()
the output code following...
process 0 ==> pid: 35499 , count: 1 , int: 3 , arr: [1.0, 1.0, 1.0] process 1 ==> pid: 35500 , count: 2 , int: 3 , arr: [1.0, 1.0, 1.0] process 2 ==> pid: 35501 , count: 3 , int: 3 , arr: [1.0, 1.0, 1.0]
if change code add delay uncommenting time.sleep(0.1) object, output changes following:
process 0 ==> pid: 35499 , count: 1 , int: 1 , arr: [1.0, 0.0, 0.0] process 1 ==> pid: 35500 , count: 2 , int: 2 , arr: [1.0, 1.0, 0.0] process 2 ==> pid: 35501 , count: 3 , int: 3 , arr: [1.0, 1.0, 1.0]
it makes sense without delay (ie, first output above) shared memory objects same values 3 processes, since once started "for" loop completes rapidly , updates shared objects' values before separate processes can run target "showdata" functions.
however, not seeing why local "count" variable allowed update incrementally. expect treated shared memory objects, without delays, count incremented 3 times rapidly before "showdata" functions run in separate processes. logic "count" should have value of 3 3 processes.
can explain why not occurring?
this running in python 3.4.3 in os x 10.10.3.
i believe because mp.process
starts new process (please note not new thread of same process , new process own pid , can see in output) each of functions, each process having own memory (stack/heap). local variables stored in each process's own memory, hence when particular process called access own stack , contains count
present when process started.
but shared memory , created multiprocessing thread, shared between each of child processes spawned multiprocessing.process
, parent process.
Comments
Post a Comment