tipimaid 0.1

Tipimaid is a collection of tools to ease the handling of Apache logs. It can be used as a piped logger for Apache and allows you to distribute log entries to different logfiles according to their virtual host information and split / rotate logfiles according to the date and time of the request. Logging can be done either locally or over the network so that you can concentrate logs of a server farm on one logging computer.

The Tipimaid collection comprises four scripts which are:

tipimaid.py

Used either as a piped logging tool or receives its loglines from tipimaid_server.py. It rotates logfiles and distributes loglines to different logfiles and is able to execute stuff when a file is rotated. Furthermore it keeps loglines in memory for buffertime seconds before they are written to the disk, so if you want to send your logs over the net, tipimaid.py ensures that all loglines which arrive in a time window of buffetime seconds are chronologically sorted according to their timestamp (so latency or short connection losses are no problem).

tipimaid_sender.py

Used as a piped logging tool to send loglines across the network to tipimaid_server.py. If the connection to tipimaid_server.py dies, it buffers the entries it was not able to send and tries to re-establish the connection. If this works in less than buffertime seconds, the buffered loglines are simply sent to tipimaid_server.py where tipimaid.py will sort and insert them into its buffer according to the timestamp of the request. However, if the connection loss takes longer than buffertime seconds, the loglines of the buffer are written to the local disk for later recovery and merging by tipimaid_mergelogs.py.

tipimaid_server.py

receives loglines sent by tipimaid_sender.py or netcat and outputs them to stdout where they are usually piped to tipimaid.py

tipimaid_mergelogs.py

The disaster recovery tool if something went terribly wrong during logging over the network. It does a mergesort of several logfiles and outputs chronologically sorted loglines to stdout.

Please have a look at <http://flloss.livinglogic.de/wiki/tipimaid> for more documentation and examples.