Running FastCGI over SCTP
Many applications on the web are run in self-contained processes that are approached from a web server over a protocol such as FastCGI or WSGI. Many such protocols run over a single TCP-connection which mixes the various input/output streams of the application. We believe that it is more efficient to use SCTP instead.
Back-end processes are compiled against a FastCGI library, which enables them to pick up requests from a webserver, handle them and recycle the process that did the work for another run. The result is superior speed relative to classical cgi-bin scripts. Furthermore, the longevity of processes enables stateful processes, which can be much faster than for instance php processing, where each script starts from scratch and needs to load both scripts and data for what may end up being a simple result.
When used remotely, a FastCGI daemon process is approached over TCP. The protocol is a fairly simple multiplex of a number of information streams. The problem of such a multiplex may be that packet loss over one of the multiplexed streams will uphold the delivery of the other streams as well. For example, error reporting could slow down content delivery or, perhaps worse, security alerts could get overruled by massive data traffic (e.g. DDoS).
The SCTP protocol can take a different approach, where streams are independently communicated. Moreover, it is possible to allocate enough streams to uphold a connection and start a new request in another set of streams.
We are asking students to modify the existing FastCGI implementation in C, and make it use SCTP instead of TCP. This will take a modification of both client and server code. It may have implications for the point where streams are allocated. The FastCGI API is agnostic to the transport protocol, so SCTP can be a drop-in replacement for TCP. This means that an application can simply be recompiled against the modified FastCGI library to use SCTP instead of TCP.
For connections across realms, protective measures may be required, such as DTLS or GSS-API. This aspect is not necessarily part of this assignment; we will resort to leaving stream 0 silent and reserved for this purpose. Any dynamic configuration will be communicated over stream 1. Streams 2 and beyond are intended for the FastCGI content streams.
First and foremost, we want to see a working demonstration of an Nginx webserver with a FastCGI backend communicating over SCTP. This means that programming skills are required from the students.
The trick of a good SCTP implementation is to find a suitable division of work over streams, and a suitable setup of the various channels. As an example, a
sysloginterface could be made available to supply feedback from the plugin to the webserver; it could be setup per request or once (with lines prefixed by the
SERVER_NAMEperhaps). Make your choices well, and document them. Where proper, demonstrate that the choices work (better than the alternatives).