[concurrency-interest] Proposal for Hybrid Threading Model and simpler Async IO

Joel Richard ri.joel at gmail.com
Wed May 7 11:54:09 EDT 2014

Quick update: The original thread was resurrected and Jeremy Manson from
Google wrote that they are implementing something quite similar internally (
Before I reinvent the wheel, I am trying to find out if is interested to
share his research and concept. That would help a lot and could even decide
whether or not this will ever be implemented officially in the JVM.

On Tue, May 6, 2014 at 12:37 AM, David Holmes <davidcholmes at aapt.net.au>wrote:

>  Hi Joel,
> That's an example of how you would program with coroutines. I'm concerned
> about the actual implementation under covers. When your coroutine blocks on
> the I/O call where does the current thread go to execute some other
> co-routine? When the I/O is ready how does the blocked thread proceed? The
> programming models for these kinds of things are well established. But your
> proposal is to try and add this to the JDK so you need to consider how it
> can be implemented.
> Cheers,
> David
> -----Original Message-----
> *From:* concurrency-interest-bounces at cs.oswego.edu [mailto:
> concurrency-interest-bounces at cs.oswego.edu]*On Behalf Of *Joel Richard
> *Sent:* Tuesday, 6 May 2014 8:00 AM
> *To:* dholmes at ieee.org
> *Cc:* concurrency-interest
> *Subject:* Re: [concurrency-interest] Proposal for Hybrid Threading Model
> and simpler Async IO
>  Hi David,
> I have created a code example which illustrates the usage and behavior of
> how I would implement coroutines* with simple async IO support in Java:
> https://gist.github.com/joelrich/a97d7d364f69c86848ca
> * Before I called them tasks or lightweight threads, but coroutine (or
> fiber) seems to be the correct name:
> https://en.wikipedia.org/wiki/Coroutine
> On Mon, May 5, 2014 at 10:08 PM, David Holmes <davidcholmes at aapt.net.au>wrote:
>> Joel Richard writes:
>> >
>> > Yes, my intent is to implement a M:N hybrid threading model;
>> > however, I think it could co-exist with the existing 1:1 model. I
>> > would suggest that the lightweight threads don't switch
>> > automatically until they block or call Thread.yield (similar to
>> > Node.js). Therefore, the M:N model would be great for servers
>> > while the 1:1 model might still be better suited for
>> > computational tasks or precise timers. In order that library
>> > developers don't have to know whether their code is running in a
>> > native or in a lightweight thread, it would make sense to create
>> > some utilities, e.g. one which executes computational tasks in a
>> > worker thread if the current thread is just a lightweight thread.
>> > Moreover, some methods would change their behavior (without
>> > breaking the API) if they are executed in lightweight threads.
>> > For example, Thread.yield would rather just start/continue
>> > executing another task instead of switching to another thread.
>> Conceptually you may be able to have a system where a M:N scheduler
>> co-exists with a 1:1 scheduler; but I do not think you can simply add that
>> to hotspot without completely rewriting the existing code.
> It would definitely require a lot of work and adjustments - I don't deny
> that. Apart from the already mentioned changes, we would have to add a
> feature to HotSpot which allows to pause/continue code of a coroutine at an
> arbitrary point without blocking the kernel thread. However, Quasar proofs
> that this is even possible at a language level (with some limitations).
>> Further, trying to generalize a M:N model to apply to all threads is
>> extremely difficult - how exactly do you pick up another thread and when
>> can you return to the original? Specialized approaches, like ForkJoin,
>> define the computational chunks so that it is easy to do this.
> I think it should use a cooperative scheduling similar to Node.js (however
> with multiple kernel threads). This would be easy, effective and optimal
> for IO servers.
>> There are also issues with thread identity that would have to be handled
>> somehow.
> Where do you see the problem here?
>  >
>> > I agree, this would change the performance and even more the
>> > scalability model, but I think that would be something positive
>> Can you back that up with any numbers? "Words are wind ..." ;-)
>> I think that is really hard before implementing a prototype. However, I
> would say that it is general consensus that async socket IO scales better
> than blocking socket IO. I don't think that coroutines would scale better
> than callbacks, but probably comparable. The main advantage would be that
> Java developers would suddenly have access to a huge number of libraries
> which use asynchronous IO and existing software stacks could scale better.
> Moreover, it would be easier to write async IO code.
>> > and bring a lot of opportunities - especially for application
>> > servers. If I am not mistaken, the JVM doesn't guarantee a
>> > certain performance characteristic or thread behavior.
>> It may not guarantee it but we go to a lot of trouble to try and not
>> introduce discontinuities in the performance curves. People invest a lot of
>> time and effort in tuning large apps, and in designing them for the way
>> things actually work. You can't completely change the threading and
>> performance model for these apps.
>> > and since lightweight threads would be an optional feature, I
>> > think this should be considered as a chance to make Java and the
>> > JVM future proof without re-implementing hundreds of libraries.
>> Are you saying that the VM is either started in traditional mode, or else
>> this new mode? That would address the compatability issue - provided you
>> can implement the new mode without changing the existing performance model.
>> That would be an option, but I think it might be smarter to support both
> modes at the same time as shown in my Gist example above. Application
> servers could then use a coroutine executor service for the request
> handling and all servlets would run in the new non-blocking mode. In
> addition to that, it might make sense to add a command line option which
> enables the coroutine mode by default.
> Regards, Joel
> David
>> -----
>> > It would make async IO and scalable servers easier than with
>> > Node.js and prevent that companies switch from Java to Go (or
>> > similar languages) just because they have a more appealing
>> > concurrency model right now.
>> >
>> >
>> > Regards, Joel
>> >
>> >
>> >
>> > On Mon, May 5, 2014 at 9:20 AM, Kirk Pepperdine <kirk at kodewerk.com>
>> wrote:
>> >
>> >
>> > On May 4, 2014, at 11:37 PM, Gregg Wonderly <gregg at wonderly.org> wrote:
>> >
>> > > The original version of Java used a runtime environment which
>> > used the term "Green Threads".  This was of course taken out when
>> > sun started trying to make Java more than an Applet engine.
>> >
>> >
>> > I believe the idea for Green Threads stems from lightweight
>> > threads in Solaris. There, a lightweight thread ran on top of a
>> > single kernel thread. The kernel thread time sliced between each
>> > of it’s LWP. I’m not sure it ever worked all that well as CPU’s
>> > at the time of inception weren’t quite what they are today and
>> > scheduling in Solaris always felt sluggish.
>> >
>> > >
>> > > There are two problems.  First is all the visibility and
>> > contention issues which the concurrency package has demonstrated
>> > to not be a simple issue due to rather arcane processor
>> > architecture issues.
>> > >
>> > > What we actually need is infinitely fast cache and cache
>> > coherency algorithms, and much more sane processor designs which
>> > try to allow processors to work together rather than trying
>> > desperately to keep them from interfering with each other.
>> >
>> >
>> > Completely agree, copies are always going to be a problem and
>> > immutability can only be part of the answer.
>> >
>> > Regards,
>> > Kirk
>> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20140507/aa3da74c/attachment-0001.html>

More information about the Concurrency-interest mailing list