[concurrency-interest] jsr166y.forkjoin API comments

Doug Lea dl at cs.oswego.edu
Sat Jan 26 14:23:17 EST 2008


jason marshall wrote:
>  And this is where 
> ParallelArray offers some things that don't quite exist in j.u.c, which 
> is why I'm interested. 

Glad we agree! Most people do seem to like the basic idea here,
which is that:
   1. A lot of everyday code involves traversal
   2. A lot of that traversal takes (or could be easily recast to)
      a parallelizable form (which again overlaps with SIMDable and
      cluster-parallel forms).
   3. A lot of developers would be happy if we put together something
      making it easy and attractive to program in this way to exploit
      increasingly ubiquitous multicores and MPs.

Of course this is not intended to address all possible forms of
concurrency and parallelism; just a common one that is
finally becoming realistically supportable, and is not addressed
by other components and frameworks.

>  But if jsr166 is predominantly about number 
> crunching in the small ...

Not directly, but this emerges as a sort of corollary to (1) and (2):
Most traversals in practice appear to mainly involve, at base,
operations on simple scalar values. So if you ignore this part,
then it hard to make good on the overall goals.

Backing up even further though: Taking all this on encounters
a continuing series of misfits with Java. We could give up
because of them, but we are much too masochistic for that :-).
The highlights so far:

Misfit #1: Java syntax for passing in snippets of code for per-element
operations is awkward and verbose. So we've tried:
   Plan A: Help instigate the "Java closures controversy"
   Plan B: Develop and recommend a usage style that is fairly
     tolerable regardless of syntax support, and likely to be
     much moreso if/when IDEs help out with it.

Misfit #2: Java has no syntax or names or conventions for the
types of these snippets of code needed for apply, reduce,
filter, etc actions. so:
    Plan A: Help intensify language controversies to include
      nominal vs function type disagreements.
    Plan B: Just define all the types the framework will ever need.
      (i..e, all the ones in Ops). It's weird, but people get over
      the weirdness pretty quickly.
      Unexpected consequence: The fact that javadoc thinks you want to
         even see all these boring types is even more offputting.
         We've gotta do something about that.

Misfit #3: Java has at best partial support for allowing checked
exceptions in these snippets of code.
   Plan A: Encourage language folks to fix this.
   Plan B: Just don't allow checked exceptions (which is very
     defensible on other grounds anyway, and likely to continue
     even if better language support emerges).

Misfit #4: Java does not allow you parameterize any of the above
over scalar types, just boxed versions, which as we've been
discussing, are the enemies of efficient parallel processing.
    Plan A: Well, there is no Plan A.
    Plan B: Internally implement all the useful cross products
      of objects vs scalars, limiting API impact to only
      a few essential ones (like the underlying array type).
      Status: Not there yet. There are still too many visible byproducts,
         odd-looking type names, etc. But still enough plausible ideas
         about addressing these to believe this can be made acceptable.

And as a final 30,000ft remark: Even if we cannot achieve usability
goals, we will still have created a useful library that developers
in the future might mainly access from other languages running on JVMs.
In fact, all of the new, evolving JVM-based languages I know that aim to
support fine-grained parallelism already do use some form of
our underlying forkjoin mechanisms, so might be expected to
use the aggregate-operation APIs as well. (Fortress uses jsr166y directly;
last I looked Scala uses FJTask predecessor (which they ought to upgrade!);
and as-yet unreleased X10 runtimes use a continuation-based variant
of jsr166y version.)

(OK, while I'm at it, since people do keep asking me what I
think of these: To me, X10 looks very promising
at least over the medium term for this style of programming while
otherwise maintaining most of the look and feel of Java. Scala
has better integration with programming support unrelated to
parallelism. Fortress is bound to be among the nicest-looking
and most carefully crafted languages ever, but may take a while
to reach the point of everyday usability.)

-Doug


More information about the Concurrency-interest mailing list