[concurrency-interest] Project FishFarm; making a ForkJoinPool distributeable
mbien at fh-landshut.de
Sat Jun 7 09:01:19 EDT 2008
Gregg Wonderly wrote:
> Michael Bien wrote:
>> Ben Manes wrote:
>>> How far along is this project?
>>> I wrote a distributed master/slave framework and (for fun) have a
>>> map-reduce abstraction prototyped. I was hoping to add fork-join,
>>> similar to your message, when we moved to Java-6 (and could use the
>>> jsr lib). So far my framework has been in production for 2 years,
>>> gone through performance reviews, etc. I've always wanted to open
>>> source it, but never got the powers that be to approve.
>>> If your interested, we can discuss such things off-line.
>> For sure we can discuss this off-line if you like. Feel free to
>> contact me directly.
>> But keep in mind all the motivation to build FishFarm was actually
>> doing distribution without introducing a new Framework ;) The idea is
>> to make grid computing optional in case you are using the
>> fj-framework anyway.
>> FishFarm is around 6 month old and is a freetime project, it was
>> never in production since I would never introduce a project which is
>> based on a fork of a jsr ;-)
> I hope that everyone working in this direction with Java, remembers to
> pay attention to all the work that has gone into Jini to make this
> possible with security and all the other tools such as leases and
> transactions, already implemented and working to help track
> distributed resources effectively, including the security features of
> Java, extended in the JERI stack to help eliminate the impact of DOS
> The current version of Jini is now the River Podling in the Apache
> Gregg Wonderly
There are many distribution systems around and each of them has its use
case. Java Spaces (and its commercial forks), Grid Gain, Terracotta,
Apache River... and even more specialized and therefore not mainstream
projects like Helios for Sunflow ray tracing tasks. Almost each of them
introduced its own framework or lets say it in other words - IS a
framework. (Terracotta is the exception which is a layer on top of the JVM)
FishFarm is no full featured distribution system, it concentrates
currently only on the distribution of ForkJoinPools and has no framework
at all (jsr166y is its framework). Again, it simple works if you
exchange "new ForkJoinPool()" with "new DistributedForkJoinPool()" and
put the jars in the classpath. It has currently no security at all since
we haven't got the requirement yet to install it outside closed networks
which are pretty common for clusters. We even decided to try it without
transactions (there is no JEE in it just simple messaging based on Shoal
which is based on JXTA) because the master simple does not care if
worker succeed with the computation or got its requested work or not.
The master simple puts all of its "stolen" work into a backup and copies
it back into the pool when the pool is empty - it still behaves like a
But my initial question was pretty general even FishFarm independent and
could be used in any framework you like:
Is there interest in making ForkJoinTasks Serializeable and providing a
mechanism for querying not yet executed tasks from a ForkJoinPool?
More information about the Concurrency-interest