ofs 0.1
OFS - provides plugin-orientated low-level blobstore.
Downloads ↓ | Package Documentation
Latest Version: 0.4.1
OFS is a bucket/object storage library.
It provides a common API for doing storing bitstreams (+ a bit of related metadata) in 'bucket/object' stores such as:
- Filesystem (+ pairtree)
- S3-compatible: S3, Google Storage, Eucalytus, Archive.org
- Riak (buggy)
- 'REST' Store (see remote/reststore.py - implementation at http://bitbucket.org/pudo/repod/)
- add a backend here
Why use the library:
Abstraction: write common code but use different storage backends
More than a filesystem, less than a database - support for metadata as well bitstreams
Extra features:
- Sharding - automatic sharding of files to support distributed file-system like structure (planned)
Example Usage
(local version - depends on 'pairtree', and 'simplejson'):
>>> from ofs.local import OFS
>>> o = OFS()
(Equivalent to 'o = OFS(storage_dir = "data", uri_base="urn:uuid:", hashing_type="md5")')
# Claim a bucket - this will add the bucket to the list of existing ones
>>> uuid_id = o.claim_a_bucket()
>>> uuid_id
'4aaa43cdf5ba44e2ad25acdbd1cf2f70'
# Choose a bucket name - if it exists, a new UUID one will be formed instead and returned
>>> bucket_id = o.claim_a_bucket("foo")
>>> bucket_id
'foo'
>>> bucket_id = o.claim_a_bucket("foo")
>>> bucket_id
'1bf93208521545879e79c13614cd12f0'
# Store a file:
>>> o.put_stream(bucket_id, "foo.txt", open("foo....))
{'_label': 'foo.txt', '_content_length': 10, '_checksum': 'md5:10feda25f8da2e2ebfbe646eea351224', '_last_modified': '2010-08-02T11:37:21', '_creation_date': '2010-08-02T11:37:21'}
# or:
>>> o.put_stream(bucket_id, "foo.txt", "asidaisdiasjdiajsidjasidji")
{'_label': 'foo.txt', '_content_length': 10, '_checksum': 'md5:10feda25f8da2e2ebfbe646eea351224', '_last_modified': '2010-08-02T11:37:21', '_creation_date': '2010-08-02T11:37:21'}
# adding a file with some parameters:
>>> o.put_stream(bucket_id, "foooo", "asidaisdiasjdiajsidjasidji", params={"original_uri":"http://...."})
{'_label': 'foooo', 'original_uri': 'http://....', '_last_modified': '2010-08-02T11:39:11', '_checksum': 'md5:3d690d7e0f4479c5a7038b8a4572d0fe', '_creation_date': '2010-08-02T11:39:11', '_content_length': 26}
# Get the underlying URL pointing to a resource
>>> o.get_url(bucket_id, "foo")
[typical local pairtree response:]
"file:///opt/ofs_store/pairtree_root/1b/f9/32/......./obj/foo"
[typical remote response]
"http://..."
"ftp://..."
# adding to existing metadata:
>>> o.update_metadata(bucket_id, "foooo", {'foo':'bar'})
{'_label': 'foooo', 'original_uri': 'http://....', '_last_modified': '2010-08-02T11:39:11', '_checksum': 'md5:3d690d7e0f4479c5a7038b8a4572d0fe', '_creation_date': '2010-08-02T11:39:11', '_content_length': 26, 'foo': 'bar'}
# Remove keys
>>> o.remove_metadata_keys(bucket_id, "foooo", ['foo'])
{'_label': 'foooo', 'original_uri': 'http://....', '_last_modified': '2010-08-02T11:39:11', '_checksum': 'md5:3d690d7e0f4479c5a7038b8a4572d0fe', '_creation_date': '2010-08-02T11:39:11', '_content_length': 26}
# Delete blob
>>> o.exists(bucket_id, "foooo")
True
>>> o.del_stream(bucket_id, "foooo")
>>> o.exists(bucket_id, "foooo")
False
# Iterate through ids for buckets held:
>>> for item in o.list_buckets():
... print item
...
447536aa0f1b411089d12399738ede8e
4a726b0a33974480a2a26d34fa0d494d
4aaa43cdf5ba44e2ad25acdbd1cf2f70
.... etc
| File | Type | Py Version | Uploaded on | Size | # downloads |
|---|---|---|---|---|---|
| ofs-0.1.tar.gz (md5) | Source | 2010-10-14 | 9KB | 349 | |
- Author: Ben O'Steen, Friedrich Lindenberg, Rufus Pollock
- Documentation: ofs package documentation
- Home Page: http://bitbucket.org/okfn/ofs
- License: http://www.apache.org/licenses/LICENSE-2.0
- Package Index Owner: rgrp, wwaites
- Package Index Maintainer: wwaites
- DOAP record: ofs-0.1.xml
