UPDATE: 05 September 2013 (later) Thanks to some prodding from Jozef Wagner, I reworked the solution to use Clojure reducers for the combine step.
Speedup now nearly scales linearly with processor count.
previously on 24 … Guy Lewis Steele’s Wordsplit and the Reducers - in which we applied GLS’ algorithm on clojure.core.reducers/fold and learned some limitations. We then used some trickery to batch-reduce text sections on fork/join and successfully split text in parallel.
Guy Lewis Steele's Wordsplit, Redux full article
UPDATE: 05 September 2013 Guy Lewis Steele’s Wordsplit, Redux, in which we address the issue raised below.
UPDATE: 04 September 2013 A reader pointed out an issue with the benchmark, namely some lazy seqs not being realized. I updated the code and post. Now the parallel speedup, while existing, is not as impressive as previously measured. The initial observations regarding the use of reducers for this problem are still relevant. More work to come.
Guy Lewis Steele's Wordsplit and the Reducers full article
Most men age twenty-six can boast few accomplishments. Sir Charles Antony Richard Hoare is not most men. At that age, C.A.R. Hoare invented Quicksort (pdf).
Quicksort is typically more efficient than other comparison sorting algorithms in both auxiliary memory requirements and execution cycles. Yet, it shares the same common case time complexity limit of its peers - O(n*log n).
Parallelism is the tool to effectively breach that barrier. As a binary tree sort, Quicksort is especially suited for parallelism since multiple threads of execution can work on sorting task parts without synchronized data access.
Quick(er)sort, Parallelism, and Fork/Join in JDK7 full article