mod_watch/4.3-P

      		 

Per Vhost Monitoring Interface

This module will watch and collect the bytes, requests, and documents in & out per virtual host, file owner, remote-ip address, directory or location, and the web server as a whole. This module can be for used with MRTG, which will make nice graphical representations of the data, but is general enough that it can be applied to other purposes, as the raw data is accessed by a URL.

Configuration

The directives below can be added to the general Apache configuration file, httpd.conf.

SetHandler watch-flush
Context: global, <Directory>, <Location>, <VirtualHost>

This handler allows a remote script to request that the shared hash table be flushed and saved to disk for the purpose of backing up the data, without having to restart Apache. This handler is not required for normal operation. It should only be made available when required and protected by Apache access directives to allow only the localhost to make this request. 

    <Location /watch-flush>
SetHandler watch-flush
Order deny,allow
Deny from all
Allow from 127.0.0.1
</Location>
SetHandler watch-info
Context: global, <Directory>, <Location>, <VirtualHost>

This handler is used to display a line of plain text containing the following space separated fields:

    ifName ifUptime ifInOctets ifOutOctets ifRequests ifDocuments ifActive ifOutRate

The ifName is the target name of the request made in a URL. The remaining fields are all numerical. ifUptime is the web server's uptime in seconds. The following is required in the global context of the httpd.conf file in order for MRTG to function: 

    <Location /watch-info>
SetHandler watch-info
</Location>

<LocationMatch "^/~.+/watch-info$">
SetHandler watch-info
</LocationMatch>

This will permit the following URL forms to get the virtual-host, file-owner, server, spool-file, and IP address data respectively:

The string between the tilde (~) and "/watch-info" can be the name of any key that appears in the watch-table list and is the only way to obtain an entry when watching by-port.

SetHandler watch-list
Context: global, <Directory>, <Location>, <VirtualHost>

This handler is similar to watch-info, but presents a plain text list of all the entries that would appear in watch-table. 

    <Location /watch-list>
SetHandler watch-list
</Location>
SetHandler watch-table
Context: global, <Directory>, <Location>, <VirtualHost>

This handler is intended for realtime analysis of the shared memory hash table performance and weenie files: 

    <Location /watch-table>
SetHandler watch-table
</Location>
WatchDocuments mime-type ...
Context: global

A list of MIME types considered to be documents, for the purpose of counting. For example "text/html" would be considered a document, which loads page elements such as images, style sheets, audio files, etc. which are not documents, but elements of one. An asterisk in the mime-type string will act as a wildcard for zero or more of anything. This directive will override the default list:

    application/*word     application/*excel     application/pdf
    application/postscript application/rtf text/enriched
    text/html text/plain text/richtext
    text/xml
WatchDynamicVirtualHost boolean
Context: global

Enable this flag for dynamic virtual host support (mod_vhost_alias, mod_vd, etc.) in addition to the static <VirtualHost> blocks defined in httpd.conf. This flag is disabled by default, unless mod_vhost_alias or mod_vd are present. If you use some other dynamic virtual host module, which mod_watch doesn't know how to detect, you should enable this flag.

WatchHashTableSize size
Context: global

A positive number specifying the size of the shared memory hash table used. The default is 991 (a prime number, but not necessary to be so).

WatchLogAll boolean
Context: global

Unsing with MRTG can result in a significant number of uninteresting entries concerning watch-info to appear in the access log. By default we chose NOT to count nor log any of the watch handlers. Enabling this option will generate more log data.

A better way to achive the similar behaviour would be to alter the httpd.conf file with something like the following: 

    SetEnvIf Request_URI "/watch-info$" IGNORE
CustomLog /dev/null common env=IGNORE
CustomLog logs/access_log common env=!IGNORE
WatchNetwork exclude ip/mask ...
WatchNetwork include ip/mask ...
Context: global

Add one or more network IP/mask pairs to either the exclusion or inclusion list. Requests coming from a network or host that is a member of the exclusion list and is not a member of the inclusion list are not counted. Otherwise the requests will be counted as outlined in WatchStorage.

The ip/mask can be specified in one of several ways and is the same as for Apache's Allow or Deny directives (domain or host name strings are not supported):

    Form     Example     Equivalent
    a1010.0.0.0/8
    a.b169.254169.254.0.0/16
    a.b.c192.168.1192.168.1.0/24
    a.b.c.d127.0.0.1127.0.0.1/32
    a.b.c.d/n192.168.1.0/24192.168.1
    a.b.c.d/A.B.C.D192.168.1.0/255.255.255.0192.168.1.0/24

Note that support for IPv6 is present and has been implemented using the Apache APR. It should be considered experimental.

WatchStateFile filename
Context: <Directory>, <Location>

Used to watch by directory or location. Each filename must be unique amongst all the virtual hosts, file owners, IP addresses, and other WatchStateFile directives, otherwise the behaviour is unspecified. The filename is used to store runtime data about a location within the WatchStateDirectory and is used to extract this data using a URL of the form:

    http://www.apachelounge.com/~filename/watch-info

The filename will also appear as an entry in the watch-table list.



WatchStateDirectory directory
Context: global

An absolute or server-root relative path to a directory where support and the "weenie" text files will be kept. This directory must be readable and writeable by the Apache child processes. If the directory does not exist, mod_watch will create it with the necessary permissions. The default is /var/spool/mod_watch.

The filename of each "weenie" text file stored here is either the virtual host name, file owner (login ID), the remote-IP address, or the string "SERVER". The contents of each text file is a string of ASCII text containing several space separated numerical fields given below.

    Name     Type     Description
    ifInOctets32-bit Counterbytes received from requests (except chunked input)
    ifOutOctets32-bit Counterbytes sent in response to requests
    ifRequests32-bit Counternumber of requests (raw hits) received
    ifDocuments32-bit Counternumber of documents sent
    ifActive16-bit Signednumber of active (concurrent) connections
    ifOutRateFloatapprox. 5 minute average of bytes sent per second
    periodOctets32-bit Unsignedbytes sent during the current 5 minute period
    periodMarker32-bit Unsignedstart time() of the current 5 minute period

When updating a set of counters, they are first looked up in the shared memory hash table otherwise in the WatchStateDirectory. The number of octets in and out for a request are added to the current counters, and the number of requests is incremented by one. The number of documents is incremented by one, if the response MIME type matches one of the MIME types specified for the WatchDocuments directive. The number of active connections is incremented at tne start of a request and decremented at the end of the same request. The counters are then stored in the shared memory hash table, which may result in collisions being flushed to disk if the probe length is exceeded.

Note that the counters, ifInOctets, ifOutOctets, ifRequest, ifDocuments, mimick SNMP style counters in that they are allowed to roll over.

The ifOutRate, periodOctets, and periodMarker are experimental fields and may change in future versions. The ifOutRate value is computed on the first request after a minimum 5 minute wait period. The periodOctets counts the number of octets sent for each period. The periodMarker is the the value returned from time() and marks the start of the current period. Note that for very active sites, this value should be fairly accurate since the rate calculation will be triggered more or less every 5 minutes, but for low volume sites the calculation may be calculated hours after the last request when someone new comes along.

WatchWhat item ...
Context: global

mod_watch, by default, will collect information for the virtual host, file owner, and the server as a whole for each request made. For performance reasons, it might not be necessary to watch all these things. Specify one or more of the following to select which items to watch:

    all        nothing
    virtual-host virtual-host-by-port
    file-owner file-owner-by-port
    remote-ip remote-ip-by-port
    server server-by-port
    headers-in * headers-out *
    content-length*  

The by-port variants collect and separate the information according to the port number, instead of grouping all requests regardless of port. For example, virtual-host-by-port would consider requests for http://www.apachelounge.com/ different from https://www.apachelounge.com/ and so would separate the information into "www.apachelounge.com:80" and "www.apachelounge.com:443". This also holds true if Apache is listening on other ports. Specifying both variants is allowed and would provide both a break down by port and the sum on all ports.

Specifying all, remote-ip, or remote-ip-by-port should be used with care as monitoring remote-IP addresses can generate large volumes of data and impact web server performance. Note that the values of ifInOctets and ifOutOctets reflect data received from and sent to the remote-IP address.

[ Specifying headers-in or headers-out will add the length of the headers of the request and response to the ifInOctets and ifOutOctets counters repectively. These are not on by default, because Apache does not count them in the access log and may skew comparisions made between web log analysis tools and the MRTG data. ] *

[ Specifying content-length will count the value of the Content-Length header on POST requests. Note however that this value can be falsified by malicious people in order to deliberately skew the graphs and so its value is suspect. There is no way to guarantee that a CGI or module actually read the input data. ] *