Capriccio: scalable threads for internet services

2005

Design of high performance Web servers has become a recent research thrust to meet the increasing demand of networkbased services. In this paper, we propose a new Web server architecture, called multi-threaded PIPELINED Web server, suitable for Symmetric Multi-Processor (SMP) or Systemon-Chip (SoC) architectures. The proposed PIPELINED model consists of multiple thread pools, where each thread pool consists of five basic threads and two helper threads. The main advantages of the proposed model are global information sharing by the threads, minimal synchronization overhead due to less number of threads, and non-blocking I/O operations, possible with the helper threads.

Journal of Parallel and Distributed …, 1996

Lightweight threads are becoming increasingly useful in supporting parallelism and asynchronous events in applications and language implementations. Traditionally, lightweight threads are supported only within the single address space of a process, or in shared memory environments with multiple processes. We introduce and describe the design of Chant, a runtime system supporting lightweight threads in a distributed memory environment. In addition to communication between any two threads in the system, Chant provides support for remote service requests, remote thread operations, and collective communication between thread groups called ropes. Chant provides the rst implementation of lightweight threads for a distributed memory platform whose design incorporates existing standards for lightweight threads and interprocess communication. This paper details the issues that arise in extending a standard threads package to support distributed memory execution, and the solutions that are provided by the Chant system.

Proceedings of the 2nd conference on Hot …, 2006

Many applications take advantage of parallelism to increase performance. Servers are a particularly common case as they must multiplex resources across many simultaneous users. Unfortunately, writing concurrent applications is difficult and prone to subtle and non-deterministic bugs that are difficult to reproduce. We advocate an approach to developing concurrent programs that is safe by default. Conservative static analysis determines when two code segments may safely run in parallel, and a runtime scheduler respects these constraints. We have built an analyzer for event-driven servers that discovers data sharing to find safe parallelism among event handlers. As a prototype, the analysis currently considers only global data, assuming that request-specific data structures passed to event handlers are completely independent. We have also begun work on a runtime system that schedules event handler execution within the constraints determined by the analyzer. For performance reasons, the scheduler makes additional conservative assumptions about contention. We have analyzed thttpd, an event-driven web server. We show how our system can be used to increase performance without complex synchronization schemes.

Proceedings of the …, 2001

Web server performance has steadily improved since the inception of the World Wide Web. We observe performance gains of two orders of magnitude between the original process-based Web servers and today's threaded Web servers. Commercial and academic Web servers achieved much of these gains using new or improved event-notification mechanisms and techniques to eliminate reading and copying data, both of which required new operating system primitives. More recently, experimental and production Web servers began integrating HTTP processing in the TCP/IP stack and providing zero copy access to a kernel-managed cache. These kernelmode Web servers improved upon newer user-mode Web servers by a factor of two to six.

Lecture Notes in Computer Science, 2005

With the development of high-speed backbone network, more and more traffic load is pushed to the Internet end system. The satisfactory execution of common business applications depends on the efficient performance of web server. In this paper, we propose a pipeline multi-thread kernel web server open KETA which divides the processing of a request into several independent phases. This architecture reduces parallelism granularity and achieves inner-request parallelism to enhance its processing capability. Furthermore, a thread allocation model is used to manage threads effectively in this special architecture. This model can adjust the thread allocation based on the server load and the work character of each phase so that the thread resource of web server can be utilized properly. Experimental result shows the capability of this web server and the zeffectiveness of the thread allocation model.

2008

With its throughput computing strategy, Sun Microsystems seeks to reverse long-standing trends towards increasingly elaborate processor designs by focusing instead on simple, replicated designs that effectively exploit threaded workloads and thus enable radically higher levels of performance scaling. 1 Throughput computing is based on chip multithreading processor design technology. In CMT technology, maximizing the amount of work accomplished per unit of time or other relevant resource, rather than minimizing the time needed to complete a given task or set of tasks, defines performance. Similarly,

2006

Resumen|The e-commerce web sites receive a great and varied number of visitors every day. These vis- itors share the application server's limited resources and when there are too many clients connecting to the web site, it is possible that they hinder between them, even to overload the application server. These visitors can be divided in dieren t categories, depend- ing on their importance from site viewpoint. Consid- ering the importance that in these web sites some client connections (e.g. buyers' connections) nish successfully before other connections, in this paper we propose a mechanism to provide dieren t quality of service to the dieren t client categories by assign- ing dieren t priorities to the threads attending the connections. After observing that Java Thread Pri- orities are only applied within the JVM, and more- over, these priorities do not reach the O.S. threads, we propose to schedule threads using the Linux Real Time priorities. Our results demonstrate th...

Performance Evaluation, 2009

Popular web sites are expected to handle huge number of requests concurrently within a reasonable time frame. The performance of these web sites is largely dependent on effective thread management of their web servers. Although the implementation of static and dynamic thread policies is common practice, remarkably little is known about the implications on performance. Moreover, the commonly used policies do not take into account the complex interaction between the threads that compete for access to a shared resource.

2011

Message-passing is an attractive thread coordination mechanism because it cleanly delineates points in an execution when threads communicate, and unifies synchronization and communication. To enable greater performance, asynchronous or non-blocking extensions are usually provided, that allow senders and receivers to proceed even if a matching partner is unavailable. However, realizing the potential of these techniques in practice has remained a difficult endeavor due to the associated runtime overheads.

International Conference on Parallel Processing, 2004. ICPP 2004., 2004

The two major strategies used to construct highperformance web servers are thread pools and eventdriven architectures. The Java platform is commonly used in web environments but up to the moment it did not provide any standard API to implement event-driven architectures efficiently. The new 1.4 release of the J2SE introduces the NIO (New I/O) API to help in the development of event-driven I/O intensive applications. In this paper we evaluate the scalability that this API provides to the Java platform in the field of web servers, bringing together the majorly used commercial server (Apache) and one experimental server developed using the NIO API. We study the scalability of the NIO-based server as well as of its rival in a number of different scenarios, including uniprocessor, multiprocessor, bandwidthbounded and CPU-bounded environments. The study concludes that the NIO API can be successfully used to create event-driven Java servers that can scale as well as the best of the commercial native-compiled web server, at a fraction of its complexity and using only one or two worker threads.