6.4. Python

6.4.1. Introduction

The qdb module includes two client implementations.

The qdb.Client class uses the standard pickle module to serialize your objects to and from the quasardb cluster.

If you want to manipulate your data directly using strings or binary buffers, you should use the qdb.RawClient class. This class does not perform any transformation on the data and will store it “as is” on the quasardb cluster. This may improve performance but most of all enables you to have a cross-language data approach.

The API is BSD licensed.

6.4.2. Requirements

Python 2.6 or higher is required.

6.4.3. Installation

As the Python API relies on our quasardb C API to function, you need to download the package that matches your operating system. For example, you cannot download the FreeBSD version and compile it on Linux.

All required libraries and extensions are included in the Python package.

6.4.3.1. Windows

Installers for Python 2.7 on Windows 32-bit and 64-bit are available. You just need to download the installer and follow the on-screen instructions.

You need to download the version matching your Python architecture, not the OS. For example, you may have installed Python 2.7 32-bit on a Windows 64-bit platform, in which case you must get the Python 32-bit quasardb package.

If you have a different Python version or if you want to recompile the extension, download the source package.

To compile it, you need the appropriate Visual Studio version (e.g. Visual Studio 2008 for Python 2.7). Unpack the archive and in the directory run:

python setup.py build
python setup.py install

The install phase may require administrative privileges.

6.4.3.2. Linux and FreeBSD

Download the source package for your operating system (Linux or FreeBSD) and make sure you have both a C compiler and the Python development headers installed.

Unpack the archive and in the directory run:

python setup.py build
sudo python setup.py install

or:

python setup.py build
su
python setup.py install
exit

GCC 4.6.0 or later is required to build the extension. You can specify the compiler with the following command:

setenv CC gcc46
python setup.py build

Provided that g++46 is the name of your GCC 4.6 compiler.

It is also possible - and even recommended on FreeBSD - to build the extension with clang:

setenv CC clang
python setup.py build

6.4.3.3. Testing the installation

Once the installation is complete, you must be able to import quasardb without any error message:

Python 2.7.2 (default, Dec  5 2011, 15:17:56)
[GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import qdb
>>>

Attention

If you built the extension from the sources, do not run the above command from the sources directory as it will attempt to load the local source code instead of the properly configured extension.

If you have a server up and running, you must be able to add and access entries:

>>> c = qdb.Client(qdb.RemoteNode("127.0.0.1"))
>>> c.put("entry", "content")
>>> print c.get("entry")
content
>>>

6.4.4. Expiry

Expiry is either set at creation or through the qdb.Client.expires_at() and qdb.Client.expires_from_now() methods.

Danger

The behavior of qdb.Client.expires_from_now() is undefined if the time zone or the clock of the client computer is improperly configured.

To set the expiry time of an entry to 1 minute, relative to the call time:

c = qdb.Client(qdb.RemoteNode("127.0.0.1"))
c.put("entry", "content")
c.expires_from_now("entry", 60)

To set the expiry time of an entry to January, 1st 2020:

c.put("entry", "content")
c.expires_at("entry", datetime.datetime(year=2020, month=1, day=1))

Or alternatively:

c.put("entry", "content", datetime.datetime(year=2020, month=1, day=1))

To prevent an entry from ever expiring:

c.expires_at("entry", None);

By default, entries never expire. To obtain the expiry time of an existing entry as a datetime.datetime object:

expiry = c.get_expiry_time("entry")

6.4.6. Iteration

Iteration is supported in a pythonesque way:

c = qdb.Client(qdb.RemoteNode("127.0.0.1"))
for e in c:
    print e

Each entry will be automatically pickled as you iterate. It is currently not possible to specify the range of iteration: you can only iterate on all entries.

6.4.7. Batch operations

A batch is run by providing the run_batch method with an array of qdb.BatchRequest:

requests = [ qdb.BatchRequest(qdb.Operations.get_alloc, "my_entry1"), qdb.BatchRequest(qdb.Operations.get_alloc, "my_entry2") ]

c = qdb.Client(qdb.RemoteNode("127.0.0.1"))
successes, results = c.run_batch(requests)

The run_batch method returns a couple. The left member is the number of successful operations and the right member is an array of qdb.BatchResult. For example to get the content for the first entry:

mycontent1 = results[0].result

The error member of each qdb.BatchResult is updated to reflect the success of the operation. If the success count returned by run_batch isn’t equal to the number of requests, the error field of each entry can be inspected to isolate the faulty requests.

All batchable operations are supported in Python, namely: get (get_alloc), put, update, remove, compare and swap (cas), get and update (get_update), get and remove (get_remove) and conditional remove (remove_if).

6.4.8. Examples

Is here a first sample using the qdb.Client. This module provides save() and load() methods:

import qdb

# Assuming we have a quasardb server running on dataserver.mydomain:3001
# Note this will throw an exception if the quasardb cluster is not available.
cl = qdb.Client(qdb.RemoteNode('dataserver.mydomain', 3001))

# We want to silently create or update the object
# depending on the existence of the key in the cluster.
def save(key, obj):
    cl.update(key, obj)

# We want to simply return None if the key is not found in the cluster.
def load(key):
    try:
        return cl.get(key)
    except qdb.QuasardbException:
        return None

The second example uses the qdb.RawClient for direct binary access. This module uses a quasardb cluster as a document store, providing upload() and download() methods, without any file size limit:

import uuid
import qdb

# If the document is bigger than 10 MiB, we slice it
SIZE_LIMIT = 10 * 1024 * 1024

# Note given what we are doing here, the server should be configure with
# a limiter-max-bytes of 1 GiB at least for proper caching
cl = qdb.RawClient(qdb.RemoteNode('docserver.mydomain', 3002))

# We expect a readable file-like object
def upload(f):
    key = uuid.uuid4()

    # If we need to slice, suffix the entry key by the slice number
    # An empty slice marks the end of the file.
    current_slice = f.read(SIZE_LIMIT)
    count = 0
    while len(current_slice) > 0:
        cl.update(key.hex + str(count), current_slice)
        current_slice = f.read(SIZE_LIMIT)
        count += 1
    return key

# We expect an uuid and a writable file-like object
def download(key, f):
    try:
        count = 0
        while True:
            f.write(cl.get(key.hex + str(count)))
            count += 1

    # If count is >0, we had at least one slice, so it is ok
    # If not, we have really not found the file.
    except qdb.QuasardbException:
        if not count: raise

6.4.9. Reference

class qdb.BatchRequest(op_type, alias=None, content=None, comparand=None, expiry=None)

A request within a batch run.

cpp_type(handle)

Converts the BatchRequest into a qdb.impl.batch_request, a low level structure used for calls to the underlying C++ API. :param handle: The qdb handle used for internal allocation

Returns:qdb.impl.batch_request - A converted copy of self
pickle()

“Pickles” the content and comparand members and returns a copy of the resulting object

Returns:a copy of self with content and comparand “pickled”
class qdb.BatchResult(br)

The result from a batch run.

unpickle()

“Unpickles” the result member and returns a copy of the resulting object.

Returns:a copy of self with result “unpickled”
class qdb.Client(remote_node, *args, **kwargs)

Bases: qdb.RawClient

The client offers the same interface as the RawClient but accepts any kind of object as alias and data, provided that the object can be marshalled with the cPickle package.

class qdb.ForwardIterator(qdb_iter)

Bases: qdb.RawForwardIterator

A forward iterator that can be used to iterate on a whole cluster with a pickle interface

exception qdb.QuasardbException(error_code=0)

Bases: exceptions.Exception

The quasardb exception, based on the API error codes.

class qdb.RawClient(remote_node, *args, **kwargs)

Bases: object

The raw client takes strings as arguments for both alias and data. If you want to put and get other objects, use Client instead.

compare_and_swap(alias, new_data, comparand, expiry_time=None)

Atomically compares the alias with comparand and replaces it with new_data if it matches. If expiry_time is None, the entry never expires.

Parameters:
  • alias (str) – The alias to compare to
  • new_data (str) – The new content to put in place if the comparand matches
  • comparand (str) – The content to compare to
  • expiry_time (datetime.datetime) – The expiry time for the alias
Returns:

str – The original content

Raises:

QuasardbException

expires_at(alias, expiry_time)

Sets the expiry time of an existing entry. If the value is None, the entry never expires.

Parameters:
  • alias (str) – The alias for which the expiry time will be set
  • expiry_time (datetime.datetime) – The expiry time, must be offset aware
Raises:

QuasardbException

expires_from_now(alias, expiry_delta)

Sets the expiry time of an existing entry relative to the current time, in seconds.

Parameters:
  • alias (str) – The alias for which the expiry time will be set
  • expiry_delta (long) – The expiry delta in seconds
Raises:

QuasardbException

get(alias)

Gets the data for the given alias. It is an error to request a non-existing entry.

param alias:The alias to get
type alias:str
returns:str – The associated content
raises:QuasardbException
get_expiry_time(alias)

Returns the expiry time of an existing entry.

Parameters:alias (str) – The alias to get the expiry time from
Returns:datetime.datetime – The expiry time, offset aware
Raises:QuasardbException
get_remove(alias)

Atomically gets the data from the given alias and removes it. It is an error to call this method on a non-existing alias. If expiry_time is None, the entry never expires.

param alias:The alias to get
type alias:str
returns:str – The associated content
raises:QuasardbException
get_update(alias, data, expiry_time=None)

Updates the given alias and returns the previous value. It is an error to call this method on a non-existing alias. If expiry_time is None, the entry never expires.

param alias:The alias to get
type alias:str
param data:The new data to put in place
type data:str
param expiry_time:
 The expiry time for the alias
type expiry_time:
 datetime.datetime
returns:str – The original content
raises:QuasardbException
node_config(remote_node)

Retrieves the configuration of a given node in JSON format.

Parameters:remote_node (qdb.RemoteNode) – The node to obtain the configuration from.
Returns:dict – The requested configuration
Raises:QuasardbException
node_status(remote_node)

Retrieves the status of a given node in JSON format.

Parameters:remote_node (qdb.RemoteNode) – The node to obtain the status from.
Returns:dict – The requested status
Raises:QuasardbException
node_topology(remote_node)

Retrieves the topology of a given node in JSON format.

Parameters:remote_node (qdb.RemoteNode) – The node to obtain the topology from.
Returns:dict – The requested topology
Raises:QuasardbException
prefix_get(prefix)

Returns the list of entries whose alias start with the given prefix. If no alias matches the given prefix, the function returns an empty list and raises no error.

Parameters:prefix (str) – The prefix to use
Returns:a list of str – The list of matching aliases
Raises:QuasardbException
put(alias, data, expiry_time=None)

Puts a piece of data in the repository. It is an error to call this method on an entry that already exists. Use the update() method to update an alias. If expiry_time is None, the entry never expires.

param alias:The alias to update
type alias:str
param data:The content for the alias
type data:str
param expiry_time:
 The expiry time for the alias
type expiry_time:
 datetime.datetime
raises:QuasardbException
remove(alias)

Removes the given alias from the repository. It is an error to remove a non-existing alias.

Parameters:alias (str) – The alias to remove
Raises:QuasardbException
remove_all()

Removes all the entries from all nodes of the cluster.

Raises:QuasardbException

Caution

This method is intended for very specific usage scenarii. Use at your own risks.

remove_if(alias, comparand)

Removes the given alias from the repository if it matches comparand. The operation is atomic. It is an error to attempt to remove a non-existing alias.

Parameters:
  • alias (str) – The alias to remove
  • comparand (str) – The content to compare to
Raises:

QuasardbException

run_batch(requests)

Runs the provided requests (a collection of BatchRequest) and returns a collection of BatchResult.

Parameters:requests (a list of BatchRequest) – The requests to run
Returns:a list of BatchRequest – The list of results
Raises:QuasardbException
stop_node(remote_node, reason)

Stops a node.

Parameters:
  • remote_node (qdb.RemoteNode) – The node to stop
  • reason (str) – A string describing the reason for the stop
Raises:

QuasardbException

Caution

This method is intended for very specific usage scenarii. Use at your own risks.

update(alias, data, expiry_time=None)

Updates the given alias. If the alias is not found in the repository, the entry is created. If expiry_time is None, the entry never expires.

param alias:The alias to update
type alias:str
param data:The content for the alias
type data:str
param expiry_time:
 The expiry time for the alias
type expiry_time:
 datetime.datetime
raises:QuasardbException
class qdb.RawForwardIterator(qdb_iter)

Bases: object

A forward iterator that can be used to iterate on a whole cluster

class qdb.RemoteNode(address, port=2836)

Convenience wrapper for the low level qdb.impl.qdb_remote_node_t structure

c_type()

Converts the RemoteNode into a qdb.impl.qdb_remote_node_t, a low-level structure used for calls to the underlying C API. :returns: qdb.impl.qdb_remote_node_t – A converted copy of self

class qdb.TimeZone

Bases: datetime.tzinfo

The quasardb time zone is UTC. Please refer to the documentation for further information.

qdb.build()

Returns the build tag and build date as a string

Returns:str – The API build tag
qdb.make_error_string(error_code)

Returns a meaningful error message corresponding to the quasardb error code.

Parameters:error_code – The error code to translate
Returns:str – An error string
qdb.version()

Returns the API’s version number as a string

Returns:str – The API version number