skip to navigation
skip to content

gocept.amqparchive 1.2.0

Archiving, indexing and search for AMQP messages.

Downloads ↓

Latest Version: 1.2.3

This package is an add-on application for gocept.amqprun that provides three features:

  • Archive queue messages by writing them to the filesystesm
  • Index those messages using ElasticSearch
  • Search for messages with a HTML/JavaScript GUI

Installation

gocept.amqparchive requires an ElasticSearch server. To set up the archive and index portion, add something like this to your gocept.amqprun configuration:

<eventlog>...</eventlog>
<amqp-server>...</amqp-server>
<worker>
  amount 2
  component-configuration site.zcml
</worker>
<settings>
  gocept.amqparchive.elastic_hostname localhost:9200
</settings>

and in site.zcml:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:amqp="http://namespaces.gocept.com/amqp">

  <include package="gocept.amqprun" />
  <include package="gocept.amqparchive" />

  <amqp:writefiles
    routing_key="key.one key.two key.three"
    queue_name="archive"
    directory="/path/to/archive"
    pattern="{routing_key}/{date}/{msgid}-{unique}.xml"
    />
</configure>

The HTML/JavaScript GUI expects /elasticsearch to proxy to the ElasticSearch server, and /messages to point to the archive directory (/path/to/archive in our example). Here's an nginx config snippet:

http {
  upstream elasticsearch {
    server localhost:9200;
  }

  server {
    listen localhost:8080;

    location /search/ {
      alias /path/to/gocept.amqparchive.egg/gocept/amqparchive/browser/;
      index index.html;
    }

    location /elasticsearch/ {
      proxy_pass http://elasticsearch/;
    }

    location /messages/ {
      alias /path/to/archive/;
      autoindex on;
    }
}

CHANGES

1.2.0 (2011-08-23)

  • Transform the XML body into a nested dict so ElasticSearch can index the fields individually.

1.1.0 (2011-08-23)

  • Add command-line script reindex_directory.

1.0.0 (2011-08-22)

  • first release.
 
File Type Py Version Uploaded on Size # downloads
gocept.amqparchive-1.2.0.tar.gz (md5) Source 2011-08-23 132KB 313
  • Author: Wolfgang Schnerring <ws@gocept.com>, Christopher Grebs <shoxi@webshox.org>
  • License: ZPL
  • Package Index Owner: wosc, zagy
  • DOAP record: gocept.amqparchive-1.2.0.xml