There are basically three efforts that have influenced the Liberty threading package. The overall concept is that the execution framework we deliver should be able to support a pipelined execution strategy, a fork/join programming model, and employ work-stealing within the scheduler.
Intel TBB is a C++ template library that provides a set of concurrent containers, a task scheduler, and infrastructure to support the creation of scalable, parallel applications.
The scheduler implemented by TBB employs a work stealing model on top of a managed thread pool. The thread pool is generally sized such that there is one thread per hardware thread and is expected to run largely compute-bound work.
TBB has first class support for pipline execution models as well as a fork/join decomposition model. TBB also provides direct access to the scheduler to implement custom policies.
For more information see the Intel Threading Building Blocks web site.
The Staged Event Driven Architecture (SEDA) describes a server architecture where the flow of work through a system as a pipeline of work stages connected by queues. Each stage hosts an event handler that processes a stream of similar events.
Within a stage, a controller works to throttle the amount of work coming in to avoid overloading the system. Another controller attempts to adjust the execution characteristics of the stage to provide optimal throughput for the stage.
While not fully implemented, the threading package is structured such that
all work stages (backed by a shared or dedicated ExecutorService
)
can be controlled and monitored. The characteristics of the stages will be
adjusted at runtime in an attempt to provide the best overall throughput for
the system.
The Liberty Event Engine is capable of associating event topics with a named work stage. The idea is that mapping mainline event topics to stages will allow us to implement a SEDA-style architecture if we choose to. If, instead of a SEDA-style implementation we prefer to process all events on the thread that fires them, we can map events to a single stage. If we're looking for mostly single threaded behavior but with load balancing, we can use work stealing to process independent events when the system has free cycles.
The java.util.concurrent
packages introduced with Java 5
have generated follow on efforts known as JSR 166x, JSR 166y, and JSR 166z.
JSR 166y, planned for Java 7, introduces implementations that support
fork-join style parallel programming. This style of programming works to
decompose the processing of a data set into smaller chunks that can be
scheduled to run in parallel. By implementing a work-stealing style
ExecutorService
in Liberty, it should be possible to either
reuse our implementation or replace it with what comes out with the JDK.
For more information, please see the JSR 166y web site.