Active Processing Daemons (APD's)

---

Overview

The active processing daemons have two main functions. The APD's can be used for debugging purposes and/or for improving performance of data intensive applications by pushing pages from servers in anticipation of expected memory use.

There are two entry points into APD processing. The APD- Administrator socket interface is one of them, and the other is a fork from an application process.

An administrator can send a request to start apd debugging over a socket interface. They will be able to turn on/off switches and set relevent thresholds interactivly to indicate the desired reports to be generated. Reporting can be done by process, by client and/or by server, showing average or maximum values.

In the virtual unit mode, the daemons use pagin/pageout statisics in conjunction with a priori knowlege of the type of application to determine when and which pages should be pushed. Debugging can also be done on a virtual unit basis.

There are three types of active processing daemon. On every server.there is a master apd that is responsible for the administrator interface and for processing the APD_TCP incoming messages. Virtual unit apd's are forked from a client application process requesting debugging and/or page pushing, and slave apd's are used for data collecting .

Interfaces

The APD uses the following interfaces with other system components:

• APD - Administrator Interface
• APD/APD TCP Interface
• Client-side Event Interface
• Server-side Event Interface
• Server-side Page-push Interface
• Virtual Unit Interface

Data Structures

The APD maintains the following internal data strutures:

• Page pushing criteria - Contains application-specific information for page push algorithms
• TCP interface table - Contains client and server multicast addresses and event stream filter information
• Statistics requirements - Set by the manager daemons, used by the stats daemons
• Statisics table - Maintained by the stats daemon, used by the apd manager daemons.

Master APD Operation

Initialization

The master apd performs the following initalization actions when it starts:

1. Create a stream socket for accepting administrative control connections for debugging requests

Main Loop

After initialization, the master apd enters its main loop in which the following processes are performed:

1. Accept and process APD_TCP messages.
2. Accept debug requests coming in on the APD - Administrator socket connection.
3. Set up filters for use by slave apd's and virtual unit apd's reading the event stream from the CSKM or SSKM
4. Issue requests to start up slave apd's using APD-TCP multicast. A multicast address is s kept for clients and for servers. If there is a virtual unit associated with the statistics, the virtual unit apd will collect the data on the client machine, and that unit's multicast address for servers holding pages for that application process will be used as the multicast address for servers. For system-wide debugging, either a single address can be used for a client or server, or a multicast address with all clents or servers is used, as indicated by the administrator over the socket interface.
5. Issue requests to close slave apd's when they are no longer needed.
6. Send push messages to the correct client virtual unit apd or server master apd over the APD-TCP interface.
7. Send push messages to the SSKM on the Server Side, or to the apd controlling the NMS Virtual Unit on the client side

Slave APD Operation

The slave apd's will be respnsible for:

1. reading the event stream coming from the CSKM or SSKM, using the filter provided by the master apd issuing the data gathering request.
2. This data will be sent in batches over the APD-TCP

Virtual Unit APD Operation

An apd_push_controller daemon is forked from an application program requesting page pushing and/or debugging information.

Initialization

The virtual unit apd performs the following initalization actions when it starts:

1. Open the NMS virtual unit, becoming the controlling process of that unit. The corresponding block device is then available to both the application process and the apd_push_controller daemon.
2. Run the appropriate apd_push_algorithm and gather any debug information requested, or choose a server to do this. If a server machine is chosen then the request for this is sent over the APD-TCP interface.
3. read the event stream coming from the CSKM and send the data in batches to the master apd running the apd_push_algorithm or tabulating the statisics for this virtual unit, if these jobs are not being done locally.
4. Send pre-paging requests to the CSKM.

Detailed Descriptions

Debug Processing

Debugging statistics can be displayed on a system wide basis or on a application process or virtual unit basis. The administrator can initiate and update requests for either on the socket interface. An application process can also request debug information. In the second case, a server may be selected to control the statistics gathering, and will periodically send updated information to the client. Either the master apd resident on the server side or the virtual unit apd on the client side maintain a statisics requirements table to be used by the statistics gathering process.

Page push processing

Page push processing is initiated by a client application during initialization. A virtual unit apd is forked to open and become the controlling process for the NMS virtual unit. The application issuing this request can set switches to turn on debugging and one to r uthe page push algorithm and debugging if any, locally on the client side.

If the algorithm isn't to run locally, a server is selected to start up a master apd for this purpose. In either case, a statisics process is started to calculate the statistics required to run the appropriate page pushing algorithm.  

When it has been determined that a page should be pushed, a message is sent to the virtual unit apd or one of the servers containing that page. The virtual unit apd uses ioctl() calls on that unit to request pre-paging as determined by the apd_push_algorithm daemon on a remote server.

Statistics Gathering

This is done in either a virtual unit apd on the client side, or on a master apd on the server side. The stastics requirement table is used to determine the filters to use on the event streams in the CSKM's and SSKM's.This filter is not necessarily the same for clients and servers. The statistics requirement table is also used to determine which clients and servers need to monitor the event streams.

Requests for data collecting apd's to be started up on clients and servers are sent over the ethernet. When there is an associated virtual unit, the virtal unit multicast group is used for the servers,.. If this is not the client machine then the event stream filter is sent over the ethernet to the virtual unit apd.

For system wide debugging, an individual request is sent to a single server /client when it is being monitored individually.  In this case the multicast group for all clients/servers is used to monitor all paging information pertaining to this server/client. A multicast message is used for all servers and another for all clients when all are to be monitored.

Last modified: Thus. May 20, 1999