apache spark - Does the scala ! subprocess command waits for the subprocess to finish? -
so, have following kind of code running in each map tasks on spark. @volatile var res = (someprogram + filename) ! var cmdres = ("rm " + filename) !; the filenames each map tasks unique. basic idea once first command finishes, second command deletes file. however, notice program complain file not exist. seems subprocess call not synchronous, is, not wait subprocess complete. correct. , if indeed case, how can correct that? as can see in docs , ! method blocks until exit. docs it: starts process represented builder, blocks until exits, , returns exit code. it's possible should checking exit code interpret result , deal exceptional cases. when creating process commands concatenation, better off using seq extensions (as opposed string ones) create processbuilder . docs include helper, might you: // uses ! exit code def fileexists(name: string) = seq("test", "-f", name).! == 0
