Storage¶

Backends¶

PostgreSQL¶

Redis¶

class cliquet.storage.redis.Redis(*args, **kwargs)¶

Storage backend implementation using Redis.

Warning

Useful for very low server load, but won’t scale since records sorting and filtering are performed in memory.

Enable in configuration:

cliquet.storage_backend = cliquet.storage.redis

(Optional) Instance location URI can be customized:

cliquet.storage_url = redis://localhost:6379/0

A threaded connection pool is enabled by default:

cliquet.storage_pool_size = 50

Memory¶

class cliquet.storage.memory.Memory(*args, **kwargs)¶

Storage backend implementation in memory.

Useful for development or testing purposes, but records are lost after each server restart.

Enable in configuration:

cliquet.storage_backend = cliquet.storage.memory

API¶

Implementing a custom storage backend consists in implementating the following interface:

class cliquet.storage.Filter(field, value, operator)¶

Filtering properties.

field¶: Alias for field number 0

operator¶: Alias for field number 2

value¶: Alias for field number 1

class cliquet.storage.Sort(field, direction)¶

Sorting properties.

direction¶: Alias for field number 1

field¶: Alias for field number 0

class cliquet.storage.StorageBase¶

Storage abstraction used by resource views.

It is meant to be instantiated at application startup. Any operation may raise a HTTPServiceUnavailable error if an error occurs with the underlying service.

Configuration can be changed to choose which storage backend will persist the objects.

Raises:	`HTTPServiceUnavailable`

initialize_schema()¶

Create every necessary objects (like tables or indices) in the backend.

This is excuted when the cliquet migrate command is ran.

flush(auth=None)¶: Remove every object from this storage.

collection_timestamp(collection_id, parent_id, auth=None)¶

Get the highest timestamp of every objects in this collection_id for this parent_id.

Note

This should take deleted objects into account.

Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent.
Returns:	the latest timestamp of the collection.
Return type:	int

create(collection_id, parent_id, object, id_generator=None, unique_fields=None, id_field='id', modified_field='last_modified', auth=None)¶

Create the specified object in this collection_id for this parent_id. Assign the id to the object, using the attribute cliquet.resource.BaseResource.id_field.

Note

This will update the collection timestamp.

Raises:	`cliquet.storage.exceptions.UnicityError`
Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent. object (dict) – the object to create.
Returns:	the newly created object.
Return type:	dict

get(collection_id, parent_id, object_id, id_field='id', modified_field='last_modified', auth=None)¶

Retrieve the object with specified object_id, or raise error if not found.

Raises:	`cliquet.storage.exceptions.RecordNotFoundError`
Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent. object_id (str) – unique identifier of the object
Returns:	the object object.
Return type:	dict

update(collection_id, parent_id, object_id, object, unique_fields=None, id_field='id', modified_field='last_modified', auth=None)¶

Overwrite the object with the specified object_id.

If the specified id is not found, the object is created with the specified id.

Note

This will update the collection timestamp.

Raises:	`cliquet.storage.exceptions.UnicityError`
Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent. object_id (str) – unique identifier of the object object (dict) – the object to update or create.
Returns:	the updated object.
Return type:	dict

delete(collection_id, parent_id, object_id, with_deleted=True, id_field='id', modified_field='last_modified', deleted_field='deleted', auth=None)¶

Delete the object with specified object_id, and raise error if not found.

Deleted objects must be removed from the database, but their ids and timestamps of deletion must be tracked for synchronization purposes. (See cliquet.storage.StorageBase.get_all())

Note

This will update the collection timestamp.

Raises:	`cliquet.storage.exceptions.RecordNotFoundError`
Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent. object_id (str) – unique identifier of the object with_deleted (bool) – track deleted record with a tombstone
Returns:	the deleted object, with minimal set of attributes.
Return type:	dict

delete_all(collection_id, parent_id, filters=None, with_deleted=True, id_field='id', modified_field='last_modified', deleted_field='deleted', auth=None)¶

Delete all objects in this collection_id for this parent_id.

Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent. filters (list of `cliquet.storage.Filter`) – Optionnally filter the objects to delete. with_deleted (bool) – track deleted records with a tombstone
Returns:	the list of deleted objects, with minimal set of attributes.
Return type:	list of dict

purge_deleted(collection_id, parent_id, before=None, id_field='id', modified_field='last_modified', auth=None)¶

Delete all deleted object tombstones in this collection_id for this parent_id.

Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent. before (int) – Optionnal timestamp to limit deletion (exclusive)
Returns:	The number of deleted objects.
Return type:	int

get_all(collection_id, parent_id, filters=None, sorting=None, pagination_rules=None, limit=None, include_deleted=False, id_field='id', modified_field='last_modified', deleted_field='deleted', auth=None)¶

Retrieve all objects in this collection_id for this parent_id.

Parameters:	collection_id (str) – the collection id. parent_id (str) – the collection parent. filters (list of `cliquet.storage.Filter`) – Optionally filter the objects by their attribute. Each filter in this list is a tuple of a field, a value and a comparison (see cliquet.utils.COMPARISON). All filters are combined using AND. sorting (list of `cliquet.storage.Sort`) – Optionnally sort the objects by attribute. Each sort instruction in this list refers to a field and a direction (negative means descending). All sort instructions are cumulative. pagination_rules (list of list of `cliquet.storage.Filter`) – Optionnally paginate the list of objects. This list of rules aims to reduce the set of objects to the current page. A rule is a list of filters (see filters parameter), and all rules are combined using OR. limit (int) – Optionnally limit the number of objects to be retrieved. include_deleted (bool) – Optionnally include the deleted objects that match the filters.
Returns:	the limited list of objects, and the total number of matching objects in the collection (deleted ones excluded).
Return type:	tuple (list, integer)

Exceptions¶

Exceptions raised by storage backend.

exception cliquet.storage.exceptions.BackendError(original=None, message=None, *args, **kwargs)¶

A generic exception raised by storage on error.

Parameters:	original (Exception) – the wrapped exception raised by underlying library.

exception cliquet.storage.exceptions.RecordNotFoundError¶: An exception raised when a specific record could not be found.

exception cliquet.storage.exceptions.UnicityError(field, record, *args, **kwargs)¶

An exception raised on unicity constraint violation.

Raised by storage backend when the creation or the modification of a record violates the unicity constraints defined by the resource.

Store custom data¶

Storage can be used to store arbitrary data.

data = {'subscribed': datetime.now()}
user_id = request.authenticated_userid

storage = request.registry.storage
storage.create(collection_id='__custom', parent_id='', record=data)

See the collection class to manipulate collections of records.