collective.solr Solr integration for external indexing and searching. Introduction ------------ `collective.solr`_ is an approach to integrate the `Solr`_ search engine with `Plone`_. It provides an indexing processor for use with `collective.indexing`_ as well as a search API similar to the standard portal catalog. `GenericSetup`_ profiles can be applied to set up content indexing in `Solr`_ and use it as a backend for `Plone`_'s site and live search facilities. .. _`collective.solr`: http://plone.org/products/collective.solr/ .. _`collective.indexing`: http://plone.org/products/collective.indexing/ .. _`Solr`: http://lucene.apache.org/solr/ .. _`Plone`: http://www.plone.org/ .. _`GenericSetup`: http://www.zope.org/Products/GenericSetup/ Current Status -------------- The implementation is considered to be nearly finished. The package can be installed in a `Plone`_ 3.x site to enable indexing operations as well as searching (site and live search) using `Solr`_. Doing so will not only significantly improve search performance |---| especially for a large number of indexed objects, but also reduce the memory footprint of your `Plone`_ instance by allowing to remove the ``SearchableText`` index from the portal catalog |---| at least for most sites. A sample buildout_ is provided for your convenience. .. |--| unicode:: U+2013 .. en dash .. |---| unicode:: U+2014 .. em dash .. _buildout: http://svn.plone.org/svn/collective/collective.solr/trunk/buildout.cfg The code was written with emphasis on minimalism, clarity and maintainability. It comes with extensive tests covering the code base. The package is currently in use in production and considered stable. .. at more than 95%. XXX: make coverage run pick up all modules! For outstanding issues and features remaining to be implemented please see the `to-do list`__ included in the package as well as it's `issue tracker`__. .. __: http://svn.plone.org/svn/collective/collective.solr/trunk/TODO.txt .. __: http://plone.org/products/collective.solr/issues FAQs / Troubleshooting ---------------------- **"AssertionError: cannot use multiple direct indexers; please enable queueing"** Symptom When installing additional packages or applying a GenericSetup profile you're getting the following error:: AssertionError: cannot use multiple direct indexers; please enable queueing Problem Early versions of the package used a persistent local utility, which is still present in your ZODB. This utility has meanwhile been replaced so that there are currently two instances present. However, without queued indexing being enabled, only one such indexer is allowed at a time. Solution Please simply re-install the package via Plone's control panel or the quick-installer. Alternatively you can also use the ZMI "Components" tab on your site root object, typically located at http://localhost:8080/plone/manage_components, to remove the broken utilities from the XML. Search for "broken". Credits ------- This code was inspired by `enfold.solr`_ by `Enfold Systems`_ as well as `work done at the snowsprint'08`__. The `solr.py` module is based on the original python integration package from `Solr`_ itself. Development was kindly sponsored by `Elkjop`_. .. _`enfold.indexing`: https://svn.enfoldsystems.com/browse/public/enfold.solr/branches/snowsprint08-buildout/enfold.indexing/ .. _`enfold.solr`: https://svn.enfoldsystems.com/browse/public/enfold.solr/branches/snowsprint08-buildout/enfold.solr/ .. _`Enfold Systems`: http://www.enfoldsystems.com/ .. __: http://tarekziade.wordpress.com/2008/01/20/snow-sprint-report-1-indexing/ .. _`Elkjop`: http://www.elkjop.no/ Changelog ========= 1.0b11 - Released July 2, 2009 ------------------------------ * Fix maintenance view for adding/syncing single indexes using catalog data. [witsch] * Allow to configure query parameters for which filter queries should be used (see http://wiki.apache.org/solr/FilterQueryGuidance for more info) [fschulze, witsch] * Encode unicode strings when building facet links. [fschulze, witsch] * Fix facet display to try to keep the given order of facets. [witsch] * Allow facet values to be translated. [witsch] 1.0b10 - Released June 11, 2009 ------------------------------- * Range queries must not be quoted with the new query parser. [witsch] * Disable socket timeouts during maintenance tasks. [witsch] * Close the response object after searching in order to avoid `ResponseNotReady` errors triggering duplicate queries. [witsch] * Use proper way of accessing jQuery & fix IE6 syntax error. [fschulze] * Format relevance value for search results. [witsch] 1.0b9 - Released May 12, 2009 ----------------------------- * Add safety net for using a translation map on unicode strings. This fixes http://plone.org/products/collective.solr/issues/4 [witsch] * Add workaround for issue with `SearchableText` criteria in topics. This fixes http://plone.org/products/collective.solr/issues/3 [witsch] * Add maintenance view for adding/syncing single indexes using already existing data from the portal catalog. [witsch] * Fix hard-coded unique key in maintenance view. [witsch] 1.0b8 - Released May 4, 2009 ---------------------------- * Fix indexing regarding Plone 3.3, `plone.indexer`_ & `PLIP 239`_. This fixes http://plone.org/products/collective.solr/issues/6 [witsch] .. _`plone.indexer`: http://pypi.python.org/pypi/plone.indexer/ .. _`PLIP 239`: http://plone.org/products/plone/roadmap/239 1.0b7 - Released April 28, 2009 ------------------------------- * Fix unintended (de)activation of the Solr integration during profile (re)application. [witsch] * Fix display of facet information with no active facets. [witsch] * Register import and export steps using ZCML. [witsch] 1.0b6 - Released April 20, 2009 ------------------------------- * Add support for facetted searches. [witsch] * Update code to comply to PEP8 style guide lines. [witsch] * Expose additional information provided by Solr - for example about headers and search facets. [witsch] * Handle edge cases like invalid range queries by quoting [tesdal] * Parse and quote the query to filter invalid query syntax. [tesdal] * In solrSearchResults, if the passed in request is a dict, look up request to enable adaptation into PloneFlare. [tesdal] * Added support for objects with a 'query' attribute as search values. [tmog] 1.0b5 - Released December 16, 2008 ---------------------------------- * Fix and extend logging in "sync" maintenance view. [witsch] 1.0b4 - Released November 23, 2008 ---------------------------------- * Filter control characters to prevent indexing errors. This fixes http://plone.org/products/collective.solr/issues/1 [witsch] * Avoid using brains when getting all objects from the catalog for sync runs. [witsch] * Prefix output from maintenance views with a time-stamp. [witsch] 1.0b3 - Released November 12, 2008 ---------------------------------- * Fix url fallback during schema retrieval. [witsch] * Fix issue regarding quoting of white space when searching. [witsch] * Make indexing operations more robust in case the schema is missing a unique key or couldn't be parsed. [witsch] 1.0b2 - Released November 7, 2008 --------------------------------- * Make schema retrieval slightly more robust to not let network failures prevent access to the site. [witsch] 1.0b1 - Released November 5, 2008 --------------------------------- * Initial release [witsch] http://cheeseshop.python.org/pypi/collective.solr/ Plone Foundation 7a7fecc394759b12e477801b9a3b3df2747d421c 1.0b11