Commit f9b88bd7 authored by Romain Courteaud's avatar Romain Courteaud

Documentation moved to

parent 60187866
* Francois Billioud
* Tristan Cavelier
* Sven Franck
* Romain Courteaud
This diff is collapsed.
For developers
Quick start
The source repository includes ready-to-use files, so in case you do
not want to build jIO yourself, just use *sha256.amd.js*, *rsvp.js*, *jio.js*
plus the storages and dependencies you need and you will be good to go.
If you want to modify or build jIO yourself, you need to
* Clone from a repository
``$ git clone``
* Install `NodeJS <>`_ (including ``npm``)
* Install the Grunt command line with ``npm``.
``# npm -g install grunt-cli``
* Install the dependencies.
``$ npm install``
* Compile the JS/CC parser.
``$ make`` (until we find out how to compile it with grunt)
* Run build.
``$ grunt``
Naming Conventions
All the code follows this :ref:`JavaScript Style Guide <style-guide>`.
How to design your own jIO Storage Library
Create a constructor:
.. code-block:: javascript
function MyStorage(storage_description) {
this._value = storage_description.value;
if (typeof this._value !== 'string') {
throw new TypeError("'value' description property " +
"is not a string");
Your storage must be added to jIO. This is done
with the ``jIO.addStorage()`` method, which requires two parameters: the storage
type (string) and a constructor (function). It is called like this:
.. code-block:: javascript
// add custom storage to jIO
jIO.addStorage('mystoragetype', MyStorage);
Please refer to *localstorage.js* implementation for a good example on how to
setup a storage and what methods are required.
Getting started
#. :ref:`Download <download-fork>` the core jIO, the storages you need and the
dependencies required for them.
#. Add the scripts to your HTML page in the following order:
.. code-block:: html
<!-- jio core + dependencies -->
<script src="rsvp.js"></script>
<script src="jio.js"></script>
<script ...>
#. jIO connects to a number of storages and allows adding handlers (or
functions) to specific storages.
You can use both handlers and available storages to build a storage
tree across which all documents will be maintained and managed by jIO.
.. code-block:: javascript
// create your jio instance
var my_jio = jIO.createJIO(storage_description);
You have to provide a ``storage_description`` object, providing location
and credentials.
Its format depends on the type of storage,
see :ref:`List of Available Storages <list-of-available-storages>`.
#. The jIO API provides ten main methods to manage documents across the storage(s) specified in your jIO storage tree.
For details on the ``document`` and ``attachment`` objects, see :ref:`What is a document? <what-is-a-document>`
======================= ======================================================
Method Example
======================= ======================================================
``.post()`` | ``, [options]);``
| Creates a new document
``.put()`` | ``my_jio.put(document, [options]);``
| Creates/Updates a document
``.putAttachment()`` | ``my_jio.putAttachement(attachment, [options]);```
| Updates/Adds an attachment to a document
``.get()`` | ``my_jio.get(document, [options]);``
| Reads a document
``.getAttachment()`` | ``my_jio.getAttachment(attachment, [options]);``
| Reads a document attachment
``.remove()`` | ``my_jio.remove(document, [options]);``
| Deletes a document and its attachments
``.removeAttachment()`` | ``my_jio.removeAttachment(attachment, [options]);``
| Deletes a document's attachment
``.allDocs()`` | ``my_jio.allDocs([options]);``
| Retrieves a list of existing documents
``.repair()`` | ``, [options]);``
| Repairs the document
======================= ======================================================
.. _download-fork:
Download & Fork
You can get latest jIO release by using on of those links:
`Full download <>`_
`Minified download <>`_
You can get latest RSVP release by using on of those links:
`Full download <>`_
`Minified download <>`_
Fork jIO
| Feel free to use the Gitlab repository:
| `GitLab <>`_: ``git clone``
Welcome to jIO
jIO is a JavaScript library that allows to manage JSON documents on local or
remote storages in asynchronous fashion. jIO is an abstracted API mapped after
CouchDB, that offers connectors to multiple storages, special handlers to
enhance functionality (replication, revisions, indexing) and a query module to
retrieve documents and specific information across storage trees.
How does it work?
jIO is composed of two parts - jIO core and storage libraries. The core
makes use of storage libraries (connectors) to interact with the associated remote
storage servers. Some queries can be used on top of the ``.allDocs()`` method to
query documents based on defined criteria.
jIO uses a job management system, so each method call adds a job into a
queue. The queue is copied in the browser's local storage (by default), so it
can be restored in case of browser crash. Jobs are invoked
asynchronously and ongoing jobs are not able to re-trigger to prevent
Copyright and license
jIO is open source and is licensed under the `LGPL <>`_ license.
jIO documentation
.. toctree::
:maxdepth: 2
jIO documentation moved to ` <>`_.
Search Keys
Features like case insensitive, accent-removing, full-text searches and more can be implemented
by customizing jIO's query behavior.
Let's start with a simple search:
.. code-block:: javascript
var query = {
type: 'simple',
key: 'someproperty',
value: comparison_value,
operator: '='
Each of the ``.someproperty`` attribute in objects' metadata is compared with
``comparison_value`` through a function defined by the '=' operator.
You can provide your own function to be used as '=' operator:
.. code-block:: javascript
var strictEqual = function (object_value, comparison_value) {
return comparison_value === object_value;
var query = {
type: 'simple',
key: {
read_from: 'someproperty',
equal_match: strictEqual
value: comparison_value
Inside ``equal_match``, you can decide to interpret the wildcard character ``%``
or just ignore it, as in this case.
If you need to convert or preprocess the values before comparison, you can provide
a conversion function:
.. code-block:: javascript
var numberType = function (obj) {
return parseFloat('3.14');
var query = {
type: 'simple',
key: {
read_from: 'someproperty',
cast_to: numberType
value: comparison_value
In this case, the operator is still the default '=' that works with strings.
You can combine ``cast_to`` and ``equal_match``:
.. code-block:: javascript
var query = {
type: 'simple',
key: {
read_from: 'someproperty',
cast_to: numberType,
equal_match: strictEqual
value: comparison_value
Now the query returns all objects for which the following is true:
.. code-block:: javascript
For a more useful example, the following function removes the accents
from any string:
.. code-block:: javascript
var accentFold = function (s) {
var map = [
[new RegExp('[àáâãäå]', 'gi'), 'a'],
[new RegExp('æ', 'gi'), 'ae'],
[new RegExp('ç', 'gi'), 'c'],
[new RegExp('[èéêë]', 'gi'), 'e'],
[new RegExp('[ìíîï]', 'gi'), 'i'],
[new RegExp('ñ', 'gi'), 'n'],
[new RegExp('[òóôõö]', 'gi'), 'o'],
[new RegExp('œ', 'gi'), 'oe'],
[new RegExp('[ùúûü]', 'gi'), 'u'],
[new RegExp('[ýÿ]', 'gi'), 'y']
map.forEach(function (o) {
var rep = function (match) {
if (match.toUpperCase() === match) {
return o[1].toUpperCase();
return o[1];
s = s.replace(o[0], rep);
return s;
cast_to: accentFold
A more robust solution to manage diacritics is recommended for production
environments, with unicode normalization, like (untested):
Overriding operators and sorting
The advantage of providing an ``equal_match`` function is that it can work with basic types;
you can keep the values as strings or, if you use a ``cast_to`` function, it can return strings,
numbers, arrays... and that's fine if all you need is the '=' operator.
It's also possible to customize the behavior of the other operators: <, >, !=...
To do that, the object returned by ``cast_to`` must contain a ``.cmp``
property, that behaves like the ``compareFunction`` described in
`Array.prototype.sort() <>`_:
.. code-block:: javascript
function myType (...) {
return {
'cmp': function (b) {
if (a < b) {
return -1;
if (a > b) {
return +1;
return 0;
cast_to: myType
If the < or > comparison makes no sense for the objects, the function should return ``undefined``.
The ``.cmp()`` property is also used, if present, by the sorting feature of queries.
Partial Date/Time match
As a real life example, consider a list of documents that have a *start_task* property.
The value of ``start_task`` can be an `ISO 8601 <>`_ string
with date and time information including fractions of a second. Which is, honestly, a bit too
much for most queries.
By using a ``cast_to`` function with custom operators, it is possible to perform queries like
"start_task > 2010-06", or "start_task != 2011". Partial time can be used as well, so
we can ask for projects started after noon of a given day: ``start_task = "2011-04-05" AND start_task > "2011-04-05 12"``
The JIODate type has been implemented on top of the `Moment.js <>`_ library, which
has a rich API with support for multiple languages and timezones. No special support for timezones
is present (yet) in JIODate.
To use JIODate, include the ``jiodate.js`` and ``moment.js`` files in your
application, then set ``cast_to = jiodate.JIODate``.
Key Schemas
Instead of providing the key object for each attribute you want to filter,
you can group all of them in a schema object for reuse:
.. code-block:: javascript
var key_schema = {
key_set: {
date_day: {
read_from: 'date',
cast_to: 'dateType',
equal_match: 'sameDay'
date_month: {
read_from: 'date',
cast_to: 'dateType',
equal_match: 'sameMonth'
cast_lookup: {
dateType: function (str) {
return new Date(str);
match_lookup: {
sameDay: function (a, b) {
return (
(a.getFullYear() === b.getFullYear()) &&
(a.getMonth() === b.getMonth()) &&
(a.getDate() === b.getDate())
sameMonth: function (a, b) {
return (
(a.getFullYear() === b.getFullYear()) &&
(a.getMonth() === b.getMonth())
With this schema, we have created two 'virtual' metadata attributes,
``date_day`` and ``date_month``. When queried, they match values that
happen to be in the same day, ignoring the time, or the same month, ignoring
both time and day.
A key_schema object can have three properties:
* ``key_set`` - required.
* ``cast_lookup`` - optional, an object of the form ``{name: function}`` that is
used if cast_to is a string. If cast_lookup is not provided,
then cast_to must be a function.
* ``match_lookup`` - optional, an object of the form ``{name: function}`` that is
used if ``equal_match`` is a string. If match_lookup is not provided,
then ``equal_match`` must be a function.
Using a schema
A schema can be used:
* In a query constructor. The same schema will be applied to all the sub-queries:
.. code-block:: javascript
jIO.QueryFactory.create({...}, key_schema).exec(...);
* In the ``jIO.createJIO()`` method. The same schema will be used
by all the queries created with the ``.allDocs()`` method:
.. code-block:: javascript
var jio = jIO.createJIO({
type: 'local',
username: '...',
application_name: '...',
key_schema: key_schema
How to manage documents?
jIO is mapped after the CouchDB APIs and extends them to provide unified, scalable
and high performance access via JavaScript to a wide variety of storage backends.
If you are not familiar with `Apache CouchDB <>`_:
it is a scalable, fault-tolerant, and schema-free document-oriented database.
It is used in large and small organizations for a variety of applications where
traditional SQL databases are not the best solution for the problem at hand.
CouchDB provides a RESTful HTTP/JSON API accessible by many programming
libraries and tools (like `curl <>`_ or `Pouchdb <>`_)
and has its own conflict management system.
.. _what-is-a-document:
What is a document?
A document is an association of metadata and attachment(s). The metadata is the
set of properties of the document and the attachments are binary (or text) objects
that represent the content of the document.
In jIO, the metadata is a dictionary with keys and values (a JSON object), and
attachments are simple strings.
.. code-block:: javascript
// document metadata
title: 'A Title!',
creator: 'Mr.Author'
:ref:`Here <metadata-head>` is a draft about metadata to use with jIO.
Basic Methods
Below you can see examples of the main jIO methods.
.. code-block:: javascript
// Create a new jIO instance
var jio_instance = jIO.createJIO(storage_description);
// create and store new document{title: 'my document'}).
then(function (response) {
// console.log(response);
// create or update an existing document
jio_instance.put('document_name', {title: 'another document'}).
then(function (response) {
// console.log(response);
// add an attachment to a document
new Blob([data], {'type' : data_mimetype});
then(function (response) {
// console.log(response);
// read a document
then(function (response) {
// console.log(response);
// read an attachment
then(function (response) {
// console.log(response);
// delete a document and its attachment(s)
then(function (response) {
// console.log(response);
// delete an attachment
then(function (response) {
// console.log(response);
// get all documents
jio_instance.allDocs().then(function (response) {
// console.log(response);
Each jIO method (with the exception of ``.createJIO()``) returns a Promise object, which allows us to get responses into
callback parameters and to chain callbacks with other returned values.
jIO uses a custom version of `RSVP.js <>`_, adding canceler and progression features.
You can read more about promises:
* `RSVP.js <>`_ on GitHub
* `Promises/A+ <>`_
* `CommonJS Promises <>`_
Method Options and Callback Responses
To retrieve jIO responses, you have to provide callbacks like this:
.. code-block:: javascript, [options]).
then([responseCallback], [errorCallback], [progressionCallback]);
* On command success, ``responseCallback`` is called with the jIO response as first parameter.
* On command error, ``errorCallback`` is called with the jIO error as first parameter.
* On command notification, ``progressionCallback`` is called with the storage notification.
Here is a list of responses returned by jIO according to methods and options:
============================================== ================== ===============================================
Available for Option Response (Callback first parameter)
============================================== ================== ===============================================
``.post()``, ``.put()``, ``.remove()`` Any id of the document affected (string)
``.putAttachment()``, ``.removeAttachment()`` Any no specific value
``.get()`` Any document_metadata (object)
``.getAttachment()`` Any .. code-block:: javascript
new Blob([data], {"type": content_type})
``.allDocs()`` No option .. code-block:: javascript
total_rows: 1,
rows: [{
id: 'mydoc',
value: {},
``.allDocs()`` include_docs: true .. code-block:: javascript
total_rows: 1,
rows: [{
id: 'mydoc',
value: {
// Here, 'mydoc' metadata
============================================== ================== ===============================================
In case of error, the ``errorCallback`` first parameter looks like:
.. code-block:: javascript
status_code: 404,
message: 'Unable to get the requested document'
How to store binary data
The following example creates a new jIO in localStorage and then posts a document with two attachments.
.. code-block:: javascript
// create a new jIO
var jio_instance = jIO.createJIO({type: 'indexeddb'});
// post the document 'myVideo'
jio_instance.put( 'metadata', {
title : 'My Video',
type : 'MovingImage',
format : 'video/ogg',
description : 'Images Compilation'
.push(undefined, function(err) {
// pushes error handler with RSVP.Queue push method
// nothing to do with JIO itself
return alert('Error posting the document metadata');
// post a thumbnail attachment
new Blob([my_image], {type: 'image/jpeg'})
).push(undefined, function(err) {
return alert('Error attaching thumbnail');
// post video attachment
new Blob([my_video], {type: 'video/ogg'})
).push(undefined, function(err) {
return alert('Error attaching video');
alert('Video Stored');
indexedDB Storage now contains:
.. code-block:: javascript
"/myVideo/": {
"title": "My Video",
"type": "MovingImage",
"format": "video/ogg",
"description": "Images Compilation",
"digest": "md5-3ue...",
"content_type": "image/jpeg",
"length": 17863
"digest": "md5-0oe...",
"content_type": "video/ogg",
"length": 2840824
"/myVideo/thumbnail": "...",
"/myVideo/video": "..."
This diff is collapsed.
jIO Query
What are Queries?
In jIO, a query can ask a storage server to select, filter, sort, or limit a
document list before sending it back. If the server is not able to do so, the
jio query tool can do the filtering by itself on the client. Only the
``.allDocs()`` method can use jio queries.
A query can either be a string (using a specific language useful for writing
queries), or it can be a tree of objects (useful to browse queries). To handle
queries, jIO uses a parsed grammar file which is compiled using `JSCC
Why use JIO Queries?
JIO queries can be used like database queries, for tasks such as:
* search a specific document
* sort a list of documents in a certain order
* avoid retrieving a list of ten thousand documents
* limit the list to show only N documents per page
For some storages (like localStorage), jio queries can be a powerful tool to
query accessible documents. When querying documents on a distant storage, some
server-side logic should be run to avoid returning too many documents to the
client. If distant storages are static, an alternative would be to use an
indexStorage with appropriate indices as jio queries will always try to run the
query on the index before querying documents itself.
How to use Queries with jIO?
Queries can be triggered by including the option named **query** in the ``.allDocs()`` method call.
.. code-block:: javascript
var options = {};
// search text query
options.query = '(creator:"John Doe") AND (format:"pdf")';
// OR query tree