Concurrent Servers

Problem:  How to allow a single server to handle multiple clients simultaneously

Four methods:

1. Multiple processes using fork
2. Multiple threads within a single process
3. Single process and thread using select
4. Single process and thread using signals

(Note:  UDP servers are less likely to use concurrency, since they can read requests from multiple clients on a single socket.)

version 1  Multiple processes

create a socket s;
bind it to an address;
make it a listening socket;
while(1) {
    ss = accept(s,...);  //  accept a connection
    if(fork()==0) {
       while (read(ss, request, requestlen)>0){
          process request;
          while(ss, results,...);
       }
       exit(0);
    }
}

Problems:

1. Parent and children do not share any variables, so how can they communicate?
• maybe they don't have to
• through file system
2. Need to clean up when children terminate
• Set up a SIGCHLD signal handler
• The signal handler calls wait to remove zombie children
3. This is the most expensive (i.e., most time-consuming) approach, due to the high overhead of process creation
   

Alternative designs using mutliple processes:

1. Prefork each child process, with each child calling accept. 
   
• When a connection request arrives, one of the accept calls will return, thus assigning one child to the client.
• If all children are busy, the client must wait.
• When a child finishes its session with a client, it calls accept again to get another client.
• May not work on some versions of Unix.
  
2. Prefork each child process, use file locking to protect accept.
• Child must obtain lock before calling accept.
  
• Only one child calls accept.  The others wait on the file lock.
  
3. Prefork each child process, use thread mutex to protect accept.
• Only one child calls accept.  The others wait on the mutex.
• The mutex must be in static memory shared by parent and all child processes.
  
4. (Prefork with parent passing socket descriptor to child.)

version 2  Multiple threads

create a socket s;
bind it to an address;
make it a listening socket;
while(1) {
    ss = accept(s,...);  //  accept a connection
    create thread to handle connection;  //  use pthread_create
}

handler(void * arg)
{
    while(read(arg,...)>0){
       process request;
       write result to socket;
    }
    return;
}

Problem

• Parent and children can share data now, but need to synchronize access.
• so, it may be necessary to use mutexes and condition variables.

Alternative designs using multiple threads:

1. Prespawn the threads, using distributed control.
   
• Each thread listens for connections on the listening socket.
• A mutex is used so that only one thread can listen at one time.
• When a connection is accepted, the thread handles it and allows another thread to accept the next connection.  This saves the overhead of thread creation time for each request.
  
  Child thread:
  
  while(1) {
      mutex_lock(&mlock);
      connfd = accept(listenfd, ...);
      mutex_unlock(&mlock);
      while(read request){
         process request;
         write results;
      }
  }
2. Prespawn the threads, using a master-slave approach
• The main thread (master) spawns the slave threads and is the only one to call accept.
• Master keeps a table indicating which slave threads are busy.
• When a connection is accepted, the master assigns it to a ready slave, and marks the slave busy.
  
• When a slave completes a session with a client, it marks itself ready.
  

version 3  Single process concurrent server using select

"Apparent concurrency"

The select system call

int retcode = select(int maxfds,            //  maximum file descriptor number
                              fdset * readfds,
                              fdset * writefds,
                              fdset * exfds,
                              struct timeval * timeout);

"fdset" is a data type defined in the Unix system library.  It represents a set of file descriptors; that is, a set of open files.  It is represented by a bit string with one bit per file descriptor -- the ith bit corresponds to descriptor i.  The library supports several macros for programming with descriptor sets:

FD_SET(i, &fdset);   //  Adds i to to fdset
FD_CLR(i, &fdset);  //  Removes i from fdset
FD_ISSET(i, &fdset);  //  Returns 1 if i belongs to fdset, 0 otherwise
FD_ZERO(&fdset);  //  Clears all bits in fdset

In a call to select, the programmer sets readfds, writefds, and exfds to the sets of descriptors the program is interested in.  On return the fdsets are the sets of ready descriptors; i.e., ready to perform I/O.  The return value from the call is equal to the number of ready descriptors.  It returns 0 if none of the descriptors becomes ready within the timeout period, and -1 if an error condition occurs.  One possible cause of a -1 is that a signal arrives before the timeout expires.

How to use it:

int maxfd;
fd_set fdset, testset;

create a socket;
bind it to an address;
make it a listening socket;

maxfd = listenfd;
FD_ZERO(&fdset);
FD_SET(listenfd, &fdset);

while(1) {
    testset = fdset;
    nready = select(maxfd+1, &testset, 0, 0, 0);
    if(FD_ISSET(listenfd, &testset)) {
       connfd = accept connection;
       insert connfd into fdset;
       if(connfd > maxfd)
          maxfd = connfd;
    }
    for(i=0; i<=maxfd; i++){
       if(FD_ISSET(i, &testset)){
          read request from descriptor i;
          if(okay){
             process request;
             write reply to descriptor i;
          }
          else { // on end of file
             FD_CLR(i, &fdset);
             close(i);
          }
       }
    }
}

Problems

Not so good if processing time for requests is highly variable

version 4  Single process concurrent server using signals

• Set the descriptors for the listening socket at any open connected sockets so that they generate signals when data is available.
• When a connection request or client request arrives, the signal handler processes it.

Possible issue:

What happens if a signal arrives while a signal handler is executing?