[concurrency-interest] Project FishFarm; making a ForkJoinPool distributeable

[Michael Bien]

Gregg Wonderly wrote:
> Michael Bien wrote:
>> Ben Manes wrote:
>>> How far along is this project?
>>>
>>> I wrote a distributed master/slave framework and (for fun) have a 
>>> map-reduce abstraction prototyped.  I was hoping to add fork-join, 
>>> similar to your message, when we moved to Java-6 (and could use the 
>>> jsr lib).  So far my framework has been in production for 2 years, 
>>> gone through performance reviews, etc.  I've always wanted to open 
>>> source it, but never got the powers that be to approve.
>>>
>>> If your interested, we can discuss such things off-line.
>> For sure we can discuss this off-line if you like. Feel free to 
>> contact me directly.
>> But keep in mind all the motivation to build FishFarm was actually 
>> doing distribution without introducing a new Framework ;) The idea is 
>> to make grid computing optional in case you are using the 
>> fj-framework anyway.
>>
>> FishFarm is around 6 month old and is a freetime project, it was 
>> never in production since I would never introduce a project which is 
>> based on a fork of a jsr ;-)
>
> I hope that everyone working in this direction with Java, remembers to 
> pay attention to all the work that has gone into Jini to make this 
> possible with security and all the other tools such as leases and 
> transactions, already implemented and working to help track 
> distributed resources effectively, including the security features of 
> Java, extended in the JERI stack to help eliminate the impact of DOS 
> attacks.
>
> The current version of Jini is now the River Podling in the Apache 
> Incubator.
>
> Gregg Wonderly
>
Hello Gregg,

There are many distribution systems around and each of them has its use 
case. Java Spaces (and its commercial forks), Grid Gain, Terracotta, 
Apache River... and even more specialized and therefore not mainstream 
projects like Helios for Sunflow ray tracing tasks. Almost each of them 
introduced its own framework or lets say it in other words - IS a 
framework. (Terracotta is the exception which is a layer on top of the JVM)
 
FishFarm is no full featured distribution system, it concentrates 
currently only on the distribution of ForkJoinPools and has no framework 
at all (jsr166y is its framework). Again, it simple works if you 
exchange "new ForkJoinPool()" with "new DistributedForkJoinPool()" and 
put the jars in the classpath. It has currently no security at all since 
we haven't got the requirement yet to install it outside closed networks 
which are pretty common for clusters. We even decided to try it without 
transactions (there is no JEE in it just simple messaging based on Shoal 
which is based on JXTA) because the master simple does not care if 
worker succeed with the computation or got its requested work or not. 
The master simple puts all of its "stolen" work into a backup and copies 
it back into the pool when the pool is empty - it still behaves like a 
ForkJoinPool.

But my initial question was pretty general even FishFarm independent and 
could be used in any framework you like:
Is there interest in making ForkJoinTasks Serializeable and providing a 
mechanism for querying not yet executed tasks from a ForkJoinPool?

best regards,

Michael