[concurrency-interest] Proposal for Hybrid Threading Model and simpler Async IO

Joel Richard ri.joel at gmail.com
Mon May 5 17:59:43 EDT 2014

Hi David,

I have created a code example which illustrates the usage and behavior of
how I would implement coroutines* with simple async IO support in Java:

* Before I called them tasks or lightweight threads, but coroutine (or
fiber) seems to be the correct name: https://en.wikipedia.org/wiki/Coroutine

On Mon, May 5, 2014 at 10:08 PM, David Holmes <davidcholmes at aapt.net.au>wrote:

> Joel Richard writes:
> >
> > Yes, my intent is to implement a M:N hybrid threading model;
> > however, I think it could co-exist with the existing 1:1 model. I
> > would suggest that the lightweight threads don't switch
> > automatically until they block or call Thread.yield (similar to
> > Node.js). Therefore, the M:N model would be great for servers
> > while the 1:1 model might still be better suited for
> > computational tasks or precise timers. In order that library
> > developers don't have to know whether their code is running in a
> > native or in a lightweight thread, it would make sense to create
> > some utilities, e.g. one which executes computational tasks in a
> > worker thread if the current thread is just a lightweight thread.
> > Moreover, some methods would change their behavior (without
> > breaking the API) if they are executed in lightweight threads.
> > For example, Thread.yield would rather just start/continue
> > executing another task instead of switching to another thread.
> Conceptually you may be able to have a system where a M:N scheduler
> co-exists with a 1:1 scheduler; but I do not think you can simply add that
> to hotspot without completely rewriting the existing code.

It would definitely require a lot of work and adjustments - I don't deny
that. Apart from the already mentioned changes, we would have to add a
feature to HotSpot which allows to pause/continue code of a coroutine at an
arbitrary point without blocking the kernel thread. However, Quasar proofs
that this is even possible at a language level (with some limitations).

> Further, trying to generalize a M:N model to apply to all threads is
> extremely difficult - how exactly do you pick up another thread and when
> can you return to the original? Specialized approaches, like ForkJoin,
> define the computational chunks so that it is easy to do this.

I think it should use a cooperative scheduling similar to Node.js (however
with multiple kernel threads). This would be easy, effective and optimal
for IO servers.

> There are also issues with thread identity that would have to be handled
> somehow.

Where do you see the problem here?

> > I agree, this would change the performance and even more the
> > scalability model, but I think that would be something positive
> Can you back that up with any numbers? "Words are wind ..." ;-)
> I think that is really hard before implementing a prototype. However, I
would say that it is general consensus that async socket IO scales better
than blocking socket IO. I don't think that coroutines would scale better
than callbacks, but probably comparable. The main advantage would be that
Java developers would suddenly have access to a huge number of libraries
which use asynchronous IO and existing software stacks could scale better.
Moreover, it would be easier to write async IO code.

> > and bring a lot of opportunities - especially for application
> > servers. If I am not mistaken, the JVM doesn't guarantee a
> > certain performance characteristic or thread behavior.
> It may not guarantee it but we go to a lot of trouble to try and not
> introduce discontinuities in the performance curves. People invest a lot of
> time and effort in tuning large apps, and in designing them for the way
> things actually work. You can't completely change the threading and
> performance model for these apps.
> > and since lightweight threads would be an optional feature, I
> > think this should be considered as a chance to make Java and the
> > JVM future proof without re-implementing hundreds of libraries.
> Are you saying that the VM is either started in traditional mode, or else
> this new mode? That would address the compatability issue - provided you
> can implement the new mode without changing the existing performance model.
> That would be an option, but I think it might be smarter to support both
modes at the same time as shown in my Gist example above. Application
servers could then use a coroutine executor service for the request
handling and all servlets would run in the new non-blocking mode. In
addition to that, it might make sense to add a command line option which
enables the coroutine mode by default.

Regards, Joel

> -----
> > It would make async IO and scalable servers easier than with
> > Node.js and prevent that companies switch from Java to Go (or
> > similar languages) just because they have a more appealing
> > concurrency model right now.
> >
> >
> > Regards, Joel
> >
> >
> >
> > On Mon, May 5, 2014 at 9:20 AM, Kirk Pepperdine <kirk at kodewerk.com>
> wrote:
> >
> >
> > On May 4, 2014, at 11:37 PM, Gregg Wonderly <gregg at wonderly.org> wrote:
> >
> > > The original version of Java used a runtime environment which
> > used the term "Green Threads".  This was of course taken out when
> > sun started trying to make Java more than an Applet engine.
> >
> >
> > I believe the idea for Green Threads stems from lightweight
> > threads in Solaris. There, a lightweight thread ran on top of a
> > single kernel thread. The kernel thread time sliced between each
> > of it’s LWP. I’m not sure it ever worked all that well as CPU’s
> > at the time of inception weren’t quite what they are today and
> > scheduling in Solaris always felt sluggish.
> >
> > >
> > > There are two problems.  First is all the visibility and
> > contention issues which the concurrency package has demonstrated
> > to not be a simple issue due to rather arcane processor
> > architecture issues.
> > >
> > > What we actually need is infinitely fast cache and cache
> > coherency algorithms, and much more sane processor designs which
> > try to allow processors to work together rather than trying
> > desperately to keep them from interfering with each other.
> >
> >
> > Completely agree, copies are always going to be a problem and
> > immutability can only be part of the answer.
> >
> > Regards,
> > Kirk
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20140505/c2e13360/attachment-0001.html>

More information about the Concurrency-interest mailing list