Server-side Daemon

Overview

The server-side daemon oversees NMS operation on the server. It maintains a database giving information about all pages currently stored on the server. It controls the transfer of data between the NMS cache and backing store.

Interfaces

The CSD uses the following interfaces with other system components:

• Control Interface
• Server-side Ioctl Interface
• Server-side Event Interface
• Server-side Administrative Interface
• CSD/SSD TCP Interface

Data Structures

The SSD maintains the following principal data structures:

• A database is maintained that gives complete information about each page of data stored by this server. The following information is stored:
  • Host ID of client who owns the data.
  • Virtual unit number on which data resides.
  • Logical page number of the data on this virtual unit.
  • Session ID in whose context the data was sent.
  • Physical memory address of data, if in NMS cache, or disk address, if on backing store.
  • Size of the page of data.
  • Date/time the data was accepted for storage.

  The database permits retrieval of pages by supplying various subsets of the identifying information: host ID of client, session ID, virtual unit number, logical page number.

  A traditional database organization is used to maintain this information. We expect to be able to use, e.g., the Berkeley "DBM" libraries, and store the database in the virtual address space of the CSD.

• The SSD maintains a swap map that describes the allocation status of all the files or disk partitions that are being used for NMS backing store.

Server Operation

Initialization

The CSD performs the following initialization actions when it starts:

1. It opens a connection to the syslog facility, used for logging status and debugging messages.
2. It registers itself with the SSKM by performing an open() call on the control interface.
3. It initializes its 64-bit global session ID to the current local time of day.
4. It joins a multicast group, with a well-known address, used for communicating general system statistics between NMS hosts.
5. It creates a stream socket for accepting administrative control connections, and binds it to a well-known port.

Main Loop

After initialization, the CSD enters its main loop, in which the following processing is performed:

• Issue periodic heartbeat messages along TCP connections with clients, so that broken connections can be detected. A heartbeat interval of about 10 seconds will be adequate.
• Accept new TCP connections from clients, and perform up client processing.
• Detect error conditions resulting from broken TCP connections with clients, and perform "down client" processing.
• Handle replacement on the NMS cache.
• Read events from event stream interface and process them, to keep data structures up to date.
• Periodically multicast statistics information over the Ethernet.
• Accept connections over the control socket, and process administrative commands received over that connection.

Shutdown

The crash recovery mechanisms permit the SSD to shutdown at any time without any warning. However, under normal circumstances, the SSD will issue, via the TCP connection to each client with which it is in contact, a "server shutdown" message. The server will then continue normal processing until it receives a "server shutdown acknowledge" message from each client. When it receives such a message from a client, it knows that the client has successfully moved to other servers any data the server had stored for it. At this time, it closes, the TCP connection to that client.

Detailed Descriptions

Status Tracking

Up Client Processing

When a new TCP connection is established with the CSD on an NMS client, "up client" processing is performed. This processing includes the following:

• The client and server negotiate a new session ID, as described here.

• An ioctl() is used to inform the kernel that the indicated client is now up, and to supply the kernel with the session ID for that client, negotiated in the previous step. The kernel uses the session ID as follows when receiving messages from a remote client:
  • When a remote client sends a message, it puts its current session ID in the message.
  • Messages arriving at a server from a client marked as "up" are accepted, but only if the session ID in the message is greater than or equal to the session ID negotiated with that server the last time TCP contact was established.
  • Messages arriving at a server from a client marked as "down", or for which the session ID in the messages is less than the last session ID negotiated with that client, are dropped.

Down Client Processing

When a TCP connection with a client is broken, "down client" processing is performed. This processing includes the following:

• Closing the server side of the connection to ensure that the client will also detect a broken connection if it is still up.
• Issuing an ioctl() to inform the SSKM that the client down, and that all data pages for this client currently in the NMS cache should be flushed.
• Identifying all data pages for this client currently residing on backing store, and freeing the associated storage.

Cache Replacement

The SSD is responsible for managing pagein and pageout between the NMS cache and backing store. Pageout is accomplished by issuing an ioctl() operation that indicates the page to be transferred, together with the location on backing store where it is to be stored. Replacement is done according to an LRU policy, and the goal is to keep, say, 5%-10% of the NMS cache memory free. We expect pageout from the NMS cache to be a normal feature of execution for certain kinds of applications that generate a large amount of data during processing, but only ever revisit a small amount of that data.

Pagein is done by the SSD in response to a pagein request event issued by the SSKM. The CSD performs pagein by an ioctl() call back to the SSKM, similarly to pageout. This call will block, in general, hence it must be performed in a separate thread. To avoid flooding the SSD with pagein requests under certain circumstances, the SSKM should probably track the number of pending pagein requests, and drop client pagein requests as necessary to throttle the number of messages that must be sent to the SSD. Note that we do not attempt to optimize the pagein operation, since significant amounts of pagein from disk on an NMS server are not a expected normal feature of operation, and will seriously degrade performance anyway.

Administrative Commands

The SSD maintains a socket on which it will accept TCP connections from system adminstrators. A simple interactive command language is supported over these connections. This permits system administrators, and perhaps system administration scripts, to contact the SSD in order to change system parameters or get status information.

When a connection is made on the administrative connection, the SSD responds by issuing a single greeting line that identifies it, followed by a prompt. The user then issues a one-line command, after which the SSD issues a zero or more line response, followed by a prompt. Commands supported are:

• set <variable> <value>
• show <variable>
• restart
• suspend
• resume
• quit