Ecosystem

This section gathers information about extending Cliquet, and third-party packages.

Packages

Note

If you build a package that you would like to see listed here, just get in touch with us!

Extending Cliquet

Pluggable components

Pluggable components can be substituted from configuration files, as long as the replacement follows the original component API.

# myproject.ini
cliquet.logging_renderer = cliquet_fluent.FluentRenderer

This is the simplest way to extend Cliquet, but will be limited to its existing components (cache, storage, log renderer, ...).

In order to add extra features, including external packages is the way to go!

Include external packages

Appart from usual python «import and use», Pyramid can include external packages, which can bring views, event listeners etc.

import cliquet
from pyramid.config import Configurator


def main(global_config, **settings):
    config = Configurator(settings=settings)

    cliquet.initialize(config, '0.0.1')
    config.scan("myproject.views")

    config.include('cliquet_elasticsearch')

    return config.make_wsgi_app()

Alternatively, packages can also be included via configuration:

# myproject.ini
pyramid.includes = cliquet_elasticsearch
                   pyramid_debugtoolbar

There are `many available packages`_, and it is straightforward to build one.

Include me

In order to be included, a package must define an includeme(config) function.

For example, in cliquet_elasticsearch/init.py:

def includeme(config):
    settings = config.get_settings()

    config.add_view(...)

Configuration

In order to ease the management of settings, Cliquet provides a helper that reads values from environment variables and uses default application values.

import cliquet
from pyramid.settings import asbool


DEFAULT_SETTINGS = {
    'cliquet_elasticsearch.refresh_enabled': False
}


def includeme(config):
    cliquet.load_default_settings(config, DEFAULT_SETTINGS)
    settings = config.get_settings()

    refresh_enabled = settings['cliquet_elasticsearch.refresh_enabled']
    if asbool(refresh_enabled):
        ...

    config.add_view(...)

In this example, if the environment variable CLIQUET_ELASTICSEARCH_REFRESH_ENABLED is set to true, the value present in configuration file is ignored.

Custom backend

As a simple example, let’s add add another kind of cache backend to Cliquet.

cliquet_riak/cache.py:

from cliquet.cache import CacheBase
from riak import RiakClient


class Riak(CacheBase):
    def __init__(self, **kwargs):
        self._client = RiakClient(**kwargs)
        self._bucket = self._client.bucket('cache')

    def set(self, key, value, ttl=None):
        key = self._bucket.new(key, data=value)
        key.store()
        if ttl is not None:
            # ...

    def get(self, key):
        fetched = self._bucked.get(key)
        return fetched.data

    #
    # ...see cache documentation for a complete API description.
    #


def load_from_config(config):
    settings = config.get_settings()
    uri = settings['cliquet.cache_url']
    uri = urlparse.urlparse(uri)

    return Riak(pb_port=uri.port or 8087)

Once its package installed and available in Python path, this new backend type can be specified in application configuration:

# myproject.ini
cliquet.cache_backend = cliquet_riak.cache

Adding features

Another use-case would be to add extra-features, like indexing for example.

  • Initialize an indexer on startup;
  • Add a /search/{collection}/ end-point;
  • Index records manipulated by resources.

Inclusion and startup in cliquet_indexing/__init__.py:

DEFAULT_BACKEND = 'cliquet_indexing.elasticsearch'

def includeme(config):
    settings = config.get_settings()
    backend = settings.get('cliquet.indexing_backend', DEFAULT_BACKEND)
    indexer = config.maybe_dotted(backend)

    # Store indexer instance in registry.
    config.registry.indexer = indexer.load_from_config(config)

    # Activate end-points.
    config.scan('cliquet_indexing.views')

End-point definitions in cliquet_indexing/views.py:

from cornice import Service

search = Service(name="search",
                 path='/search/{collection_id}/',
                 description="Search")

@search.post()
def get_search(request):
    collection_id = request.matchdict['collection_id']
    query = request.body

    # Access indexer from views using registry.
    indexer = request.registry.indexer
    results = indexer.search(collection_id, query)

    return results

Example indexer class in cliquet_indexing/elasticsearch.py:

class Indexer(...):
    def __init__(self, hosts):
        self.client = elasticsearch.Elasticsearch(hosts)

    def search(self, collection_id, query, **kwargs):
        try:
            return self.client.search(index=collection_id,
                                      doc_type=collection_id,
                                      body=query,
                                      **kwargs)
        except ElasticsearchException as e:
            logger.error(e)
            raise

    def index_record(self, collection_id, record, id_field):
        record_id = record[id_field]
        try:
            index = self.client.index(index=collection_id,
                                      doc_type=collection_id,
                                      id=record_id,
                                      body=record,
                                      refresh=True)
            return index
        except ElasticsearchException as e:
            logger.error(e)
            raise

Indexed resource in cliquet_indexing/resource.py:

class IndexedCollection(cliquet.resource.Collection):
    def create_record(self, record):
        r = super(IndexedCollection, self).create_record(self, record)

        self.indexer.index_record(self, record)

        return r

class IndexedResource(cliquet.resource.BaseResource):
    def __init__(self, request):
        super(IndexedResource, self).__init__(request)
        self.collection.indexer = request.registry.indexer

Note

In this example, IndexedResource must be used explicitly as a base resource class in applications. A nicer pattern would be to trigger Pyramid events in Cliquet and let packages like this one plug listeners. If you’re interested, we started to discuss it!

JavaScript client

One of the main goal of Cliquet is to ease the development of REST microservices, most likely to be used in a JavaScript environment.

A client could look like this:

var client = new cliquet.Client({
    server: 'https://api.server.com',
    store: localforage
});

var articles = client.resource('/articles');

articles.create({title: "Hello world"})
  .then(function (result) {
    // success!
  });

articles.get('id-1234')
  .then(function (record) {
    // Read from local if offline.
  });

articles.filter({
    title: {'$eq': 'Hello'}
  })
  .then(function (results) {
    // List of records.
  });

articles.sync()
  .then(function (result) {
    // Synchronize offline store with server.
  })
  .catch(function (err) {
    // Error happened.
    console.error(err);
  });