Skip to main content

XML-Director

Project description

xmldirector.plonecore

xmldirector.plonecore is the technical foundation of the XML-Director project (www.xml-director.info). The goal of the XML-Director project is building an enterprise-level XML content management system on top of the CMS Plone (www.plone.org) with support for XML databases like eXis-db or Base-X. However the underlaying implementation can also be used to mount arbitrary backend through WebDAV into Plone.

xmldirector.plonecore integrates Plone 4.3 or 5.1 with eXist-db or Base-X or providing the following features:

  • mounts an arbitary eXist-db or Base-X collection into Plone

  • ACE editor integration

  • ZIP export from eXist-db or Base-X

  • ZIP import into eXist-db or Base-X

  • pluggable view mechanism for configuring custom views for XML database content by filename and view name

  • create, rename or delete collections through the web

  • extensible architecture through Plone Dexterity behaviors

  • support for XQuery scripts called through the RESTXQ layer of eXist-db (allows you to call XQuery scripts and return the output format (JSON, HTML, XML) depending on your application requirements)

  • dedicated per-connector logging facility

  • small and extensible

  • experimental support for mounting arbitrary WebDAV service into Plone

    • XMLText - a field for storing XML content in BaseX or eXist-db

    • XPathField - for retrieving parts of XML content stored within a XMLText field through an XPath expression (e.g. for extracting and displaying metadata from XML content)

    • XMLBinary and XMLImage fields for storing binary data and images in BaseX or eXist-db. The functionality is identical with the standard Dexterity file and image fields (except for the different storage layer)

The primary usecase for xmldirector.plonecore is the integration of XML document collections into Plone using eXist-db or Base-X as storage layer. xmldirector.plonecore is not storage layer for Plone content in the first place although it could be used in some way for storing primary Plone content (or parts of the content) inside eXist-db or Base-X. There is no build-in support for mapping metadata as stored in XML documents to Plone metadata or vice versa. However this could be implemented easily in application specific code build on top of xmldirector.plonecore. The design goal of xmldirector.plonecore is to provide the basic functionality for integrating Plone with eXist-db or Base-X without implementing any further specific application requirements. Additional functionality can be added through Dexterity behaviors, supplementary browser views, event lifecycle subscribers and related technology.

Requirements

  • Plone 4.3 (tested)

  • Plone 5.0 (tested)

  • Plone 5.1 (tested)

  • Supported backends:

    • eXist-db 2.2 and 3.0

    • Base-X >= 8.2

    • OwnCloud

    • Alfresco

    • Marklogic Server

    • Dropbox (via dropdav.com (Dropbox to WebDAV bridge, paid SaaS) or via xmldirector.dropbox)

    • AWS S3

    • Cloud federation services

      • Otixo.com

      • Storagemadeeasy.com

  • experimental support for the following backends/protocols (don’t use XML Director with these protocols/backends in production):

    • FTP (working in general, open issues with the connection pool)

    • SFTP (working in general, open issues with the connection pool)

Configuration

Goto the Plone control panel and click on the XML-Director Core configlet and configure the your service

ExistDB

BaseX

Owncloud

Alfresco

Dropbox (via dropdav.com SaaS)

Dropbox (via xmldirector.dropbox)

  • dropbox://dropbox.com

  • access to Dropbox must be authenticated using OAuth (see xmldirector.dropbox documentation)

SME (storagemadeeasy.com)

Otixo.com

Local filesystem

AWS S3

  • s3://bucketname

  • enter your AWS access key as username and the AWS secret key as password

  • you need to install the boto module (either using pip or using zc.buildout)

FTP

SFTP

  • sftp://hostname/path/to/directory

  • username and password that are necessary to access the configured directory path through SFTP. Username and password can be omitted in case the XML Director server (your Plone instance) has access using SSH keys (without passphrase) to the remote SFTP service. The handling of username + password vs. SSH authentication using SSH keys is currently under investigation and might change in a further release.

Protocol/Service

Native support

3rd-party SaaS support (e.g. Otixo.com, storagemadeeasy.com)

Local filesystem

Yes

No

WebDAV

Yes

Yes

ExistDB 2.2/3.0

Yes

Yes

BaseX 8.3

Yes

Yes

Amazon AWS S3

Yes

Yes

Alfresco

Yes

Yes

Owncloud

Yes

Yes

Dropbox

(experimental)

Yes

SFTP

(experimental)

Yes

FTP

(experimental)

Yes

Using xmldirector.plonecore

The package provides a new content-types Connector that will include eXist-db or Base-X into Plone - either from the top-level collection of your eXist-db/Base-X database or from a subcollection. You can browse and traverse into subcollections, view single documents or edit text-ish content through the web (using the build-in ACE editor integration).

All connection settings (URL, username and password can be overriden on the connector level) in order to ignore the Plone site-wide eXist-db settings).

Dexterity fields

xmldirector.plonecore comes with the following Dexterity fields that can be either used programmatically in your own field schema or through-the-web.

XMLText

The XMLText can be used to store valid XML content. The field is rendered without Plone using the ACE editor. You can perform a client-side XML validation within the edit mode of a document by clicking on the Validate XML button. A document with invalid XML content can not be submitted or saved. Invalid XML will be rejected with an error message through the edit form.

XMLXPath

The XMLXPath field can be used to reference an XMLText field in order to display a part of the XML content using an XPath expression.

Example

An XMLPath field with field name myxml might contain the following XML content:

<?xml version="1.0"?>
<doc>
    <metadata>
        <title>This is a text</title>
    </metdata>
    <body>....</body>
</doc>

In order to extract and display the <title> text within a dedicated Dexterity field you can use the following extended expression:

field=<fieldname>,xpath=<xpath expression>

In this case you would use:

field=myxml,xpath=/doc/metadata/title/text()

Note that the current syntax is very rigid and does not allow any whitespace characters expect within the <xpath expression>.

XMLBinary, XMLImage

Same as file and image field in Plone but with BaseX or eXist-db as storage layer.

License

This package is published under the GNU Public License V2 (GPL 2)

Source code

See https://bitbucket.org/onkopedia/xmldirector.plonecore

Bugtracker

See https://bitbucket.org/onkopedia/xmldirector.plonecore

Travis-CI

All releases and code changes to XML Director are automatically tested against various combinations of Plone and backend versions. The current test setup consists of 14 different combinations against the most common databases and services. See https://github.com/xml-director/xmldirector.plonecore/blob/master/.travis.yml for testing details. A complete test run consists of over 1400 single tests.

See https://travis-ci.org/xml-director/xmldirector.plonecore

https://travis-ci.org/xml-director/xmldirector.plonecore.svg?branch=master

Credits

The development of xmldirector.plonecore was funded as part of a customer project by Deutsche Gesellschaft für Hämatologie und medizinische Onkologie (DGHO).

Author

Andreas Jung/ZOPYX
Hundskapfklinge 33
D-72074 Tuebingen, Germany
www.zopyx.com

Changelog

2.1.1 (2017-07-23)

  • fixed JS issue in rename action

2.1.0 (2017-04-04)

  • compatibility with Plone 5.1

2.0.4 (2017-01-10)

  • pinned fs==0.5.4

2.0.3 (2016-12-05)

  • minor bug fixes

2.0.2 (2016-08-26)

  • added explicit setContext() to BaseWrapper class for setting a custom connector class

2.0.0 (2016-08-18)

  • full compatibility with Plone 5.0 (limited to Plone 5.1 alpha)

  • replaced DataTables.net with agGrid

  • major UI changes

  • updated to latest ACE version

  • switch to Dropzone.js for multi-file uploads

1.6.1 (2016-04-08)

  • improved unicode filename handling across drivers

  • PEP8 fixes

1.6.0 (2016-03-18)

  • experimental native Dropbox support (requires installed ‘dropbox’ SDK for Python using ‘pip install dropbox’ or by adding ‘dropbox’ as dependency inside your buildout) - not ready for production

  • workaround for sporadic open() failures with Exist-DB: open() will be tried up to three times (with a slight delay between calls in order to give the backend a chance to recover in between)

  • support for latest plone.api releases (which minor monkey patch)

  • move restapi permission checks to ZCML configuration

  • updated to ACE 1.2.3

  • updated to DataTables 1.10.11

  • support for latest plone.api release

1.5.0 (2016-02-17)

  • huge internal renaming: renamed all webdav_* variables to connector_*

  • adjusted tests to Marc Logic Server

1.4.2 (2016-02-02)

  • fix in restapi in the context of local fs tests

1.4.1 (2016-01-18)

  • pass query string down to redirection call within __call__()

1.4.0 (2016-01-11)

  • new REST API

1.3.0 (2015-12-20)

  • added ‘create_if_not_existing’ parameter to webdav_handle() method

  • added ensuredir() to wrapper base class

1.3.0b1 (2015-10-15)

  • support for OSFS (local filesystem), S3

  • individual per-connector URL configuration

  • support for multi-file uploads

  • unicode fixes in the context of testing federated cloud solutions

  • massive amount of smaller internal and UI fixes

  • lots of testing with different storage backends

  • updated tests

1.2.0 (2015-09-29)

  • refactored zip export functionality

  • new supported WebDAV storage backends:

    • Alfresco

    • Owncloud

    • Dropbox (via dropdav.com SaaS)

1.1.1 (2015-08-25)

  • updated selenium version pin

  • better encapsulation of the DataTables Javascript initialization code in local.js

  • fixed integration bug in Plone 5.0

1.1.0 (2015-08-21)

  • some CSS styles fine-tuning

  • added optional dirs parameter to ZIP export API for exporting only a subset of the export directory structure

  • added control panel functionality for installing Exist-DB specific RESTXQ script (e.g. all-locks.xql which is needed by the lockmanager introspection control panel for getting hold of all locks).

  • ZIP import now works inside the given subdirectory and no longer only on the top level directory of the connector

  • delete actions for collections and collection items

  • delete actions now ask for confirmation

  • massive speedup of ZIP import by reducing and caching WebDAV operations

  • using Datatables.net for collections and collection items

  • ZIP export/import is now more robust with directories or filenames containing non-ascii characters

  • improved Plone 5.0 integration

1.0.4 (2015-07-22)

  • updated ACeditor to version 1.2.0

1.0.3 (2015-07-22)

  • using DataTables.net for connector logging view instead of TableUtils JS

1.0.2 (2015-06-12)

  • updated Saxon 6.0.6HE

  • added get_last_user(), get_last_date() to logger API for getting hold of the username performing the last logger entry

1.0.0 (2015-05-30)

  • using defusedxml module for protecting XML Director against malicious data and XML related security attacks

  • added support for ‘force_default_view’ URL parameter to enforce redirection to the default anonymous view

  • support for logging HTML messages

0.4.2 (2015-04-11)

  • updated lxml, cssselect dependencies to newest versions

  • analyzed XSD parsing slowness and logging/warning long-running XSD parsing

  • first serious take on Plone 5.0 compatibility on the UI level (backend tests have been always passing but we had serious UI issues until 5.0 beta 1 and there are still issues). Plone 5 beta support is work-in-progress and not fully completed.

0.4.1 (2015-04-07)

  • added entry_by_uuid() to PersistentLogAdapter API

  • fixed unicode issues with uploaded binaries/images with non-ascii filenames

  • added ‘version_suffix’ parameter to parser_folder() of validator registry

  • Javascript cleanup

0.4.0 (2015-02-18)

  • added @@transformer-registery view

  • added @@transformer-registery-view view

  • updated xmldirector.demo to use Transformer registry

  • added (optional) debug option for debugging Transformer steps (input and output data of a step is written to disk)

  • added more tests

  • support for XSLT2+3 transformations by integrating Saxon 9.6 HE

0.3.6 (2015-02-06)

  • re-added Dexterity tests

  • added validator registry for XML schemas, DTDs, Schematron files and RelaxNG schemas

  • added @@validator-registry view

  • added unified validation API based on registered validation files

  • documented validator registry

0.3.5 (2015-01-30)

  • rewritten persistent logger internals: now uses an OOBTree for holding all logging entries instead of a persistent list in order to support filtering of log entries by min-max values

  • logger table now uses a paginated view with searchable columns

  • webdav password setting is no longer required (empty password allowed)

  • fixed Webdav authentication issue with empty passwords

  • moved demo related code into a dedicated package xmldirector.demo

0.3.4 (2015-01-13)

  • default view handler accept a custom request/filename argument in order to override the name of downloaded file

  • fixed bug in view registry with BrowserView as view handler

  • added PersistentLoggerAdapter for adopting arbitrary persistent objects for persistent logging through a Zope annotation

0.3.3 (2015-01-05)

  • running the tests should not leave any testing directory traces within the XML databases

  • almost 100% test coverage for the core functionality

  • more tests

  • added documentation on content-types

0.3.2 (2014-12-30)

  • SHA256 calculation for xml content now generated in a more stable way (but possibly much slower way)

  • API for service-side XML validation

  • added Docker support

  • added XSLT registry

  • added Shakespeare XML data for XMLDocument demo content-type

  • added ‘test_all.sh’ script for running tests against BaseX and eXist-db Docker containers

0.3.1 (2014-12-12)

  • added Test connection button to controlpanel

  • moved test content type into a dedicated profile democontent

  • Moved metadata handling from JSON to XML on the storage layer in order to let the underlaying database index the .metadata.xml files as well

0.3.0 (2014-12-11)

  • renamed zopyx.existdb to xmldirector.plonecore

  • experimental Dexterity support with four new fields:

    • XMLText - for XML content

    • XMLXPath - for referencing XMLText parts through an XPath expression

    • XMLImage and XMLBinary - same as image and file fields in Dexterity but with eXist-db as storage layer

  • removed emulation configuration option

  • added plone.app.dexterity as dependency

  • upgraded to ACE editor V 1.1.8

  • added progressbar for zip_upload()

  • added support for importing a single file through the ZIP import form into the current subdirectory

0.2.11 (2014-11-08)

  • updated documentation

0.2.10 (2014-11-08)

  • bugfix release

0.2.9 (2014-11-01)

  • support for overriding credentials locally

0.2.8 (2014-11-01)

  • minor fix for mounting Plone sites over WebDAV into another Plone site

0.2.7 (2014-11-01)

  • experimental support for BaseX XML database through the WebDAV API. Limitations: REMOVE operations over WebDAV do not seem to work against BaseX 7.9

0.2.6 (2014-11-01)

  • more tests

0.2.5 (2014-10-30)

  • experimental traversal support for accessing WebDAV resources by path using (un)restrictedTraverse()

  • minor URL fixes

  • more tests

0.2.4 (2014-10-22)

  • configuration option for default view for authenticated site visitors

0.2.3 (2014-10-13)

  • fix in saving ACE editor content

0.2.2 (2014-10-12)

  • typo in page template

0.2.1 (2014-10-12)

  • added support for renaming a collection through the web

0.2.0 (2014-10-02)

  • various minor bug fixes

  • added basic tests

0.1.17 (2014-09-25)

  • fixed action links

0.1.16 (2014-09-25)

  • Connector is no longer a folderish object

0.1.15 (2014-09-22)

  • removed indexing support completely (leaving a specific indexing functionality to policy packages using zopyx.existdb)

0.1.14 (2014-09-15)

  • fixed subpath handling in create/remove collections

0.1.13 (2014-09-07)

  • support for removing collections TTW

0.1.12 (2014-09-05)

  • support for creating new collections TTW

0.1.11 (2014-08-21)

  • action “Clear log” added

0.1.10 (2014-08-05)

  • log() got a new ‘details’ parameter for adding extensive logging information

0.1.9 (2014-08-01)

  • human readable timestamps

0.1.8 (2014-07-31)

  • minor visual changes

0.1.7 (2014-07-29)

  • rewritten code exist-db browser code (dealing the correct way with paths, filenames etc.)

0.1.6 (2014-07-29)

  • fixed improper view prefix in directory browser

0.1.5 (2014-07-13)

  • minor fixes and cleanup

0.1.4 (2014-07-12)

  • made webservice query API aware of all output formats (xml, html, json)

  • timezone handling: using environment variable TZ for converting eXist-db UTC timestamps to the TZ timezone (or UTC as default) for display purposes with Plone

0.1.3 (2014-07-07)

  • added webservice API interface

  • various bug fixes

0.1.2 (2014-06-30)

  • various bug fixes

0.1.0 (2014-06-20)

  • initial release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

xmldirector.plonecore-2.1.1.zip (16.6 MB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page