[concurrency-interest] Proposal for Hybrid Threading Model and simpler Async IO

David Holmes davidcholmes at aapt.net.au
Mon May 5 18:37:23 EDT 2014

Hi Joel,

That's an example of how you would program with coroutines. I'm concerned about the actual implementation under covers. When your coroutine blocks on the I/O call where does the current thread go to execute some other co-routine? When the I/O is ready how does the blocked thread proceed? The programming models for these kinds of things are well established. But your proposal is to try and add this to the JDK so you need to consider how it can be implemented.

  -----Original Message-----
  From: concurrency-interest-bounces at cs.oswego.edu [mailto:concurrency-interest-bounces at cs.oswego.edu]On Behalf Of Joel Richard
  Sent: Tuesday, 6 May 2014 8:00 AM
  To: dholmes at ieee.org
  Cc: concurrency-interest
  Subject: Re: [concurrency-interest] Proposal for Hybrid Threading Model and simpler Async IO

  Hi David,

  I have created a code example which illustrates the usage and behavior of how I would implement coroutines* with simple async IO support in Java: https://gist.github.com/joelrich/a97d7d364f69c86848ca

  * Before I called them tasks or lightweight threads, but coroutine (or fiber) seems to be the correct name: https://en.wikipedia.org/wiki/Coroutine

  On Mon, May 5, 2014 at 10:08 PM, David Holmes <davidcholmes at aapt.net.au> wrote:

    Joel Richard writes:
    > Yes, my intent is to implement a M:N hybrid threading model;
    > however, I think it could co-exist with the existing 1:1 model. I
    > would suggest that the lightweight threads don't switch
    > automatically until they block or call Thread.yield (similar to
    > Node.js). Therefore, the M:N model would be great for servers
    > while the 1:1 model might still be better suited for
    > computational tasks or precise timers. In order that library
    > developers don't have to know whether their code is running in a
    > native or in a lightweight thread, it would make sense to create
    > some utilities, e.g. one which executes computational tasks in a
    > worker thread if the current thread is just a lightweight thread.
    > Moreover, some methods would change their behavior (without
    > breaking the API) if they are executed in lightweight threads.
    > For example, Thread.yield would rather just start/continue
    > executing another task instead of switching to another thread.

    Conceptually you may be able to have a system where a M:N scheduler co-exists with a 1:1 scheduler; but I do not think you can simply add that to hotspot without completely rewriting the existing code.

  It would definitely require a lot of work and adjustments - I don't deny that. Apart from the already mentioned changes, we would have to add a feature to HotSpot which allows to pause/continue code of a coroutine at an arbitrary point without blocking the kernel thread. However, Quasar proofs that this is even possible at a language level (with some limitations).

    Further, trying to generalize a M:N model to apply to all threads is extremely difficult - how exactly do you pick up another thread and when can you return to the original? Specialized approaches, like ForkJoin, define the computational chunks so that it is easy to do this.

  I think it should use a cooperative scheduling similar to Node.js (however with multiple kernel threads). This would be easy, effective and optimal for IO servers. 

    There are also issues with thread identity that would have to be handled somehow.

  Where do you see the problem here?

    > I agree, this would change the performance and even more the
    > scalability model, but I think that would be something positive

    Can you back that up with any numbers? "Words are wind ..." ;-)

  I think that is really hard before implementing a prototype. However, I would say that it is general consensus that async socket IO scales better than blocking socket IO. I don't think that coroutines would scale better than callbacks, but probably comparable. The main advantage would be that Java developers would suddenly have access to a huge number of libraries which use asynchronous IO and existing software stacks could scale better. Moreover, it would be easier to write async IO code.

    > and bring a lot of opportunities - especially for application
    > servers. If I am not mistaken, the JVM doesn't guarantee a
    > certain performance characteristic or thread behavior.

    It may not guarantee it but we go to a lot of trouble to try and not introduce discontinuities in the performance curves. People invest a lot of time and effort in tuning large apps, and in designing them for the way things actually work. You can't completely change the threading and performance model for these apps.

    > and since lightweight threads would be an optional feature, I
    > think this should be considered as a chance to make Java and the
    > JVM future proof without re-implementing hundreds of libraries.

    Are you saying that the VM is either started in traditional mode, or else this new mode? That would address the compatability issue - provided you can implement the new mode without changing the existing performance model.

  That would be an option, but I think it might be smarter to support both modes at the same time as shown in my Gist example above. Application servers could then use a coroutine executor service for the request handling and all servlets would run in the new non-blocking mode. In addition to that, it might make sense to add a command line option which enables the coroutine mode by default.

  Regards, Joel


    > It would make async IO and scalable servers easier than with
    > Node.js and prevent that companies switch from Java to Go (or
    > similar languages) just because they have a more appealing
    > concurrency model right now.
    > Regards, Joel
    > On Mon, May 5, 2014 at 9:20 AM, Kirk Pepperdine <kirk at kodewerk.com> wrote:
    > On May 4, 2014, at 11:37 PM, Gregg Wonderly <gregg at wonderly.org> wrote:
    > > The original version of Java used a runtime environment which
    > used the term "Green Threads".  This was of course taken out when
    > sun started trying to make Java more than an Applet engine.
    > I believe the idea for Green Threads stems from lightweight
    > threads in Solaris. There, a lightweight thread ran on top of a
    > single kernel thread. The kernel thread time sliced between each
    > of it’s LWP. I’m not sure it ever worked all that well as CPU’s
    > at the time of inception weren’t quite what they are today and
    > scheduling in Solaris always felt sluggish.
    > >
    > > There are two problems.  First is all the visibility and
    > contention issues which the concurrency package has demonstrated
    > to not be a simple issue due to rather arcane processor
    > architecture issues.
    > >
    > > What we actually need is infinitely fast cache and cache
    > coherency algorithms, and much more sane processor designs which
    > try to allow processors to work together rather than trying
    > desperately to keep them from interfering with each other.
    > Completely agree, copies are always going to be a problem and
    > immutability can only be part of the answer.
    > Regards,
    > Kirk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20140506/0def5ab3/attachment.html>

More information about the Concurrency-interest mailing list