Storage¶
Backends¶
PostgreSQL¶
Redis¶
-
class
cliquet.storage.redis.
Redis
(*args, **kwargs)¶ Storage backend implementation using Redis.
Warning
Useful for very low server load, but won’t scale since records sorting and filtering are performed in memory.
Enable in configuration:
cliquet.storage_backend = cliquet.storage.redis
(Optional) Instance location URI can be customized:
cliquet.storage_url = redis://localhost:6379/0
A threaded connection pool is enabled by default:
cliquet.storage_pool_size = 50
API¶
Implementing a custom storage backend consists in implementating the following interface:
-
class
cliquet.storage.
Filter
(field, value, operator)¶ Filtering properties.
-
field
¶ Alias for field number 0
-
operator
¶ Alias for field number 2
-
value
¶ Alias for field number 1
-
-
class
cliquet.storage.
Sort
(field, direction)¶ Sorting properties.
-
direction
¶ Alias for field number 1
-
field
¶ Alias for field number 0
-
-
class
cliquet.storage.
StorageBase
¶ Storage abstraction used by resource views.
It is meant to be instantiated at application startup. Any operation may raise a HTTPServiceUnavailable error if an error occurs with the underlying service.
Configuration can be changed to choose which storage backend will persist the objects.
Raises: HTTPServiceUnavailable
-
initialize_schema
()¶ Create every necessary objects (like tables or indices) in the backend.
This is excuted when the
cliquet migrate
command is ran.
-
flush
(auth=None)¶ Remove every object from this storage.
-
collection_timestamp
(collection_id, parent_id, auth=None)¶ Get the highest timestamp of every objects in this collection_id for this parent_id.
Note
This should take deleted objects into account.
Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
Returns: the latest timestamp of the collection.
Return type: int
-
create
(collection_id, parent_id, object, id_generator=None, unique_fields=None, id_field='id', modified_field='last_modified', auth=None)¶ Create the specified object in this collection_id for this parent_id. Assign the id to the object, using the attribute
cliquet.resource.BaseResource.id_field
.Note
This will update the collection timestamp.
Raises: Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
- object (dict) – the object to create.
Returns: the newly created object.
Return type: dict
-
get
(collection_id, parent_id, object_id, id_field='id', modified_field='last_modified', auth=None)¶ Retrieve the object with specified object_id, or raise error if not found.
Raises: Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
- object_id (str) – unique identifier of the object
Returns: the object object.
Return type: dict
-
update
(collection_id, parent_id, object_id, object, unique_fields=None, id_field='id', modified_field='last_modified', auth=None)¶ Overwrite the object with the specified object_id.
If the specified id is not found, the object is created with the specified id.
Note
This will update the collection timestamp.
Raises: Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
- object_id (str) – unique identifier of the object
- object (dict) – the object to update or create.
Returns: the updated object.
Return type: dict
-
delete
(collection_id, parent_id, object_id, with_deleted=True, id_field='id', modified_field='last_modified', deleted_field='deleted', auth=None)¶ Delete the object with specified object_id, and raise error if not found.
Deleted objects must be removed from the database, but their ids and timestamps of deletion must be tracked for synchronization purposes. (See
cliquet.storage.StorageBase.get_all()
)Note
This will update the collection timestamp.
Raises: Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
- object_id (str) – unique identifier of the object
- with_deleted (bool) – track deleted record with a tombstone
Returns: the deleted object, with minimal set of attributes.
Return type: dict
-
delete_all
(collection_id, parent_id, filters=None, with_deleted=True, id_field='id', modified_field='last_modified', deleted_field='deleted', auth=None)¶ Delete all objects in this collection_id for this parent_id.
Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
- filters (list of
cliquet.storage.Filter
) – Optionnally filter the objects to delete. - with_deleted (bool) – track deleted records with a tombstone
Returns: the list of deleted objects, with minimal set of attributes.
Return type: list of dict
-
purge_deleted
(collection_id, parent_id, before=None, id_field='id', modified_field='last_modified', auth=None)¶ Delete all deleted object tombstones in this collection_id for this parent_id.
Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
- before (int) – Optionnal timestamp to limit deletion (exclusive)
Returns: The number of deleted objects.
Return type: int
-
get_all
(collection_id, parent_id, filters=None, sorting=None, pagination_rules=None, limit=None, include_deleted=False, id_field='id', modified_field='last_modified', deleted_field='deleted', auth=None)¶ Retrieve all objects in this collection_id for this parent_id.
Parameters: - collection_id (str) – the collection id.
- parent_id (str) – the collection parent.
- filters (list of
cliquet.storage.Filter
) – Optionally filter the objects by their attribute. Each filter in this list is a tuple of a field, a value and a comparison (see cliquet.utils.COMPARISON). All filters are combined using AND. - sorting (list of
cliquet.storage.Sort
) – Optionnally sort the objects by attribute. Each sort instruction in this list refers to a field and a direction (negative means descending). All sort instructions are cumulative. - pagination_rules (list of list of
cliquet.storage.Filter
) – Optionnally paginate the list of objects. This list of rules aims to reduce the set of objects to the current page. A rule is a list of filters (see filters parameter), and all rules are combined using OR. - limit (int) – Optionnally limit the number of objects to be retrieved.
- include_deleted (bool) – Optionnally include the deleted objects that match the filters.
Returns: the limited list of objects, and the total number of matching objects in the collection (deleted ones excluded).
Return type: tuple (list, integer)
-
Exceptions¶
Exceptions raised by storage backend.
-
exception
cliquet.storage.exceptions.
BackendError
(original=None, message=None, *args, **kwargs)¶ A generic exception raised by storage on error.
Parameters: original (Exception) – the wrapped exception raised by underlying library.
-
exception
cliquet.storage.exceptions.
RecordNotFoundError
¶ An exception raised when a specific record could not be found.
-
exception
cliquet.storage.exceptions.
UnicityError
(field, record, *args, **kwargs)¶ An exception raised on unicity constraint violation.
Raised by storage backend when the creation or the modification of a record violates the unicity constraints defined by the resource.
Store custom data¶
Storage can be used to store arbitrary data.
data = {'subscribed': datetime.now()}
user_id = request.authenticated_userid
storage = request.registry.storage
storage.create(collection_id='__custom', parent_id='', record=data)
See the collection class to manipulate collections of records.