multithreading - Parallel stream creates only one thread and gives result as fast as normal stream -
below code try process lines read file in parallel stream , in normal stream. surprisingly, parallel stream gives no improvements on normal stream . missing here ?
files.walk(paths.get(tweetfilepath + localdate.now())).foreach( filepath -> { if (files.isregularfile(filepath) && !filepath.tostring().endswith(".ds_store")) { long starttime = system.currenttimemillis(); try { files.lines(filepath).parallel().foreach(line -> { try { system.out.println(line); } catch (exception e) { system.out.println("not able crunch"+ e); } }); } catch (exception e) { system.out.println("bad line in file "); }finally { system.out.println("total time required:" + (system.currenttimemillis() - starttime)); } } });
looks currently, files.lines
reads file linearly, parallel call cannot split source stream sub-streams parallel processing.
see here details. relevant section quoted below:
what if source based on io?
currently, jdk io-based stream sources (for example bufferedreader.lines()) geared sequential use, processing elements one-by-one arrive. opportunities exist supporting highly efficient bulk processing of buffered io, these require custom development of stream sources, spliterators, and/or collectors. common forms may supported in future jdk releases.
Comments
Post a Comment