updated documentation

renamed "revision_storage.rst" "replicate_storage.rst"
you need to "compile" the files (go in doc/ directory and type "make")
to have the html files.

docs/developers.rst

@@ -12,7 +12,7 @@ If you want to modify or build jIO yourself, you need to
 
 * Clone from a repository
 
-  ``$ git clone https://github.com/nexedi/jio.git``
+  ``$ git clone https://lab.nexedi.com/nexedi/jio.git``
 
 * Install `NodeJS <http://nodejs.org/>`_ (including ``npm``)
 
@@ -53,8 +53,8 @@ Create a constructor:
       }
     }
 
-Create 10 methods: ``post``, ``put``, ``putAttachment``, ``get``, ``getAttachment``,
-``remove``, ``removeAttachment``, ``allDocs``, ``check`` and ``repair``.
+Create 9 methods: ``post``, ``put``, ``putAttachment``, ``get``, ``getAttachment``,
+``remove``, ``removeAttachment``, ``allDocs``. ``repair`` method is optional.
 
 .. code-block:: javascript
 
@@ -81,66 +81,26 @@ Create 10 methods: ``post``, ``put``, ``putAttachment``, ``get``, ``getAttachmen
 
 (To help you design your methods, some tools are provided by jIO.util.)
 
-The first parameter command provides some methods to act on the jIO job:
-
-* ``success``, to tell jIO that the job is successfully terminated
-
-  ``command.success(status[Text], [{custom key to add to the response}]);``
-
-* ``resolve``, is equal to success
-
-* ``error``, to tell jIO that the job cannot be done
-
-  ``command.error(status[Text], [reason], [message], [{custom key to add to the response}])``
-
-* ``retry``, to tell jIO that the job cannot be done now, but can be retried later. (same API than error)
-
-* ``reject``, to tell jIO that the job cannot be done, let jIO to decide whether to retry or not. (same API than error)
-
-
 The second parameter ``metadata`` or ``param`` is the first parameter provided by the jIO user.
 
 The third parameter ``option`` is the option parameter provided by the jIO user.
 
-Methods should return the following objects:
+Methods should return:
 
-* **post()** --> ``success("created", {"id": new_generated_id})``
+* **post()**, **put()**, **remove()** --> id of the document affected (string)
 
-* **put()**, **remove()**, **putAttachment()** or **removeAttachment()** --> ``success(204)``
+* **putAttachment()** or **removeAttachment()** --> no specific value
 
-* **get()** --> ``success("ok", {"data": document_metadata})``
+* **get()** --> document_metadata (object)
 
 * **getAttachment()** -->
 
   .. code-block:: javascript
 
-    success("ok", {"data": binary_string, "content_type": content_type})
-    // or
-    success("ok", {"data": new Blob([data], {"type": content_type})})
-
-* **allDocs()** --> ``success("ok", {"data": row_object})``
-
-* **check()** -->
-
-  .. code-block:: javascript
-
-    // if metadata provides "_id" -> check document state
-    // if metadata doesn't promides "_id" -> check storage state
-    success("no_content")
-    // or
-    error("conflict", "corrupted", "incoherent document or storage")
-
-* **repair()** -->
+    new Blob([data], {"type": content_type})
 
-  .. code-block:: javascript
+* **allDocs()** --> list of all documents (restricted by a query, if given). (object)
 
-    // if metadata provides "_id" -> repair document state
-    // if metadata doesn't promides "_id" -> repair storage state
-    success("no_content")
-    // or
-    error("conflict", "corrupted", "impossible to repair document or storage")
-    // DON'T DESIGN STORAGES IF THERE IS NO WAY
-    // TO REPAIR INCOHERENT STATES
 
 After creating all methods, your storage must be added to jIO. This is done
 with the ``jIO.addStorage()`` method, which requires two parameters: the storage
@@ -154,160 +114,3 @@ type (string) and a constructor (function). It is called like this:
 
 Please refer to *localstorage.js* implementation for a good example on how to
 setup a storage and what methods are required.
-
-Also keep in mind that jIO is a job-based library: whenever you trigger a method,
-a job is created, which will later return a response, after being processed.
-
-Job rules
----------
-
-The jIO job manager follows several rules set at the creation of a new jIO
-instance. When you try to call a method, jIO will create a job and will make
-sure the job is really necessary and will be executed. Thanks to these job
-rules, jIO knows what to do with the new job before adding it to the queue. You
-can also add your own rules, as we're going to see now.
-
-These are the jIO **default rules**:
-
-.. code-block:: javascript
-
-    var jio_instance = jIO.createJIO(storage_description, {
-      "job_rules": [{
-        "code_name": "readers update",
-        "conditions": [
-          "sameStorageDescription",
-          "areReaders",
-          "sameMethod",
-          "sameParameters",
-          "sameOptions"
-        ],
-        "action": "update"
-      }, {
-        "code_name": "metadata writers update",
-        "conditions": [
-          "sameStorageDescription",
-          "areWriters",
-          "useMetadataOnly",
-          "sameMethod",
-          "haveDocumentIds",
-          "sameParameters"
-        ],
-        "action": "update"
-      }, {
-        "code_name": "writers wait",
-        "conditions": [
-          "sameStorageDescription",
-          "areWriters",
-          "haveDocumentIds",
-          "sameDocumentId"
-        ],
-        "action": "wait"
-      }]
-    });
-
-
-The following actions can be used:
-
-* ``ok`` - accept the job
-* ``wait`` - wait until the end of the selected job
-* ``update`` - bind the selected job to this one
-* ``deny`` - reject the job
-
-The following condition functions can be used:
-
-* ``sameStorageDescription`` - check if the storage descriptions are different.
-* ``areWriters`` - check if the commands are ``post``, ``put``, ``putAttachment``, ``remove``, ``removeAttachment``, or ``repair``.
-* ``areReaders`` - check if the commands are ``get``, ``getAttachment``, ``allDocs`` or ``check``.
-* ``useMetadataOnly`` - check if the commands are ``post``, ``put``, ``get``, ``remove`` or ``allDocs``.
-* ``sameMethod`` - check if the commands are equal.
-* ``sameDocumentId`` - check if the document ids are equal.
-* ``sameParameters`` - check if the metadata or param are equal (deep comparison).
-* ``sameOptions`` - check if the command options are equal.
-* ``haveDocumentIds`` - test if the two commands contain document ids.
-
-Create Job Condition
---------------------
-
-You can create two types of function: job condition, and job comparison.
-
-.. code-block:: javascript
-
-    // Job Condition
-    // Check if the job is a get command
-
-    jIO.addJobRuleCondition("isGetMethod", function (job) {
-      return job.method === 'get';
-    });
-
-
-    // Job Comparison
-    // Check if the jobs have the same 'title' property
-    // only if they are strings
-
-    jIO.addJobRuleCondition("sameTitleIfString",
-        function (job, selected_job) {
-          if (typeof job.kwargs.title === 'string' &&
-              typeof selected_job.kwargs.title === 'string') {
-            return job.kwargs.title === selected_job.kwargs.title;
-          }
-          return false;
-        });
-
-
-Add job rules
--------------
-
-You just have to define job rules in the jIO options:
-
-.. code-block:: javascript
-
-    // Do not accept to post or put a document which title is equal
-    // to another already running post or put document title
-
-    var jio_instance = jIO.createJIO(storage_description, {
-      "job_rules": [{
-        "code_name": "avoid similar title",
-        "conditions": [
-          "sameStorageDescription",
-          "areWriters",
-          "sameTitleIfString"
-        ],
-        "action": "deny",
-        "before": "writers update" // optional
-        // "after": also exists
-      }]
-    });
-
-
-Clear/Replace default job rules
--------------------------------
-
-If a job's ``code_name`` is equal to ``readers update``, then adding this rule will replace it:
-
-.. code-block:: javascript
-
-    var jio_instance = jIO.createJIO(storage_description, {
-      "job_rules": [{
-        "code_name": "readers update",
-        "conditions": [
-          "sameStorageDescription",
-          "areReaders",
-          "sameMethod",
-          "haveDocumentIds"
-          "sameParameters"
-          // sameOptions is removed
-        ],
-        "action": "update"
-      }]
-    });
-
-Or you can just clear all rules before adding new ones:
-
-.. code-block:: javascript
-
-    var jio_instance = jIO.createJIO(storage_description, {
-      "clear_job_rules": true,
-      "job_rules": [{
-        // ...
-      }]
-    });

docs/getting_started.rst

@@ -79,8 +79,6 @@ Getting started
                              |  Deletes a document's attachment
     ``.allDocs()``           |  ``my_jio.allDocs([options]);``
                              |  Retrieves a list of existing documents
-    ``.check()``             |  ``my_jio.check(document, [options]);``
-                             |  Checks the document state
     ``.repair()``            |  ``my_jio.repair(document, [options]);``
                              |  Repairs the document
     =======================  ======================================================
@@ -122,29 +120,29 @@ Storage dependencies
 Storage connectors
 ^^^^^^^^^^^^^^^^^^
 
-* localstorage.js
-* davstorage.js
-* searchableencryptionstorage.js (depends on sjcl) (WIP)
-* s3storage.js (depends on sha1, jQuery) (WIP)
+* Localstorage
+* MemoryStorage
+* IndexedDB
+* WebSQL
+* DavStorage
+* Dropbox
+* Google Drive
+* ERP5Storage
+* s3storage.js (WIP)
 * xwikistorage.js (depends on jQuery) (WIP)
-* erp5storage.js (WIP)
-* restsqlstorage.js (depends on jQuery) (WIP)
-* mioga2storage.js (depends on jQuery) (WIP)
 
 Storage handlers
 ^^^^^^^^^^^^^^^^
 
-* indexstorage.js
-* gidstorage.js
-* splitstorage.js (WIP)
-* replicatestorage.js (WIP)
-
-Revision based storage handlers
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-* revisionstorage.js (depends on sha256)
-* replicaterevisionstorage.js
-
+* Zipstorage
+* ShaStorage
+* UUIDStorage
+* QueryStorage
+* CryptStorage
+* UnionStorage
+* FileSystemBridgeStorage
+* Document Storage
+* Replicate Storage
 
 Unit tests
 ^^^^^^^^^^
@@ -155,9 +153,5 @@ the test suite with each release.
 Fork jIO
 ^^^^^^^^
 
-The same source code is kept in three synchronized repositories.
-Feel free to use any of them.
-
-* `GitHub <https://github.com/nexedi/jio>`_: ``git clone https://github.com/nexedi/jio.git``
-* `Gitorius <https://gitorious.org/nexedi/jio>`_: ``git clone https://git.gitorious.org/nexedi/jio.git``
-* `Git Erp5 <http://git.erp5.org/gitweb/jio.git>`_ (read only): ``git clone http://git.erp5.org/repos/jio.git``
+| Feel free to use the Gitlab repository:
+| `GitLab <https://lab.nexedi.com/nexedi/jio.git>`_: ``git clone https://lab.nexedi.com/nexedi/jio.git``

docs/index.rst

@@ -38,9 +38,8 @@ jIO documentation
 
     getting_started
     manage_documents
-    revision_storages
+    replicate_storage
     available_storages
-    gid_storage
     query
     keys
     metadata

docs/metadata.rst

@@ -58,11 +58,6 @@ List of metadata to use
 Identification
 ^^^^^^^^^^^^^^
 
-* **_id**
-
-  A specific jIO metadata which helps the storage to find a document
-  (can be a real path name, a dc:identifier, a uuid, ...). **String Only**
-
 * **identifier**
 
   | ``{"identifier": "http://domain/jio_home_page"}``

docs/replicate_storage.rst

+.. _replicate-storage-conflicts-and-resolution:
+
+Replicate Storage: Conflicts and Resolution
+===========================================
+
+
+Why Conflicts can Occur
+-----------------------
+
+Using jIO you can store documents in multiple locations. With an
+increasing number of users working on a document and some storages not being
+available or responding too slow, conflicts are more likely to occur. jIO
+defines a conflict as multiple versions of a document existing in a storage
+tree and a user trying to save on a version that does not match the latest
+version of the document.
+
+To keep track of document versions a replicate storage must be used. When doing
+so, jIO creates a document tree file for every document. This file contains all
+existing versions and their status and is modified whenever a version is
+added/updated/removed or when storages are being synchronized.
+
+How conflicts are handled
+-------------------------
+
+The RemoteStorage takes in parameter two substorages, one "local" and one "remote".
+The "local" storage can be remote, but it will be used for all the requests
+like **get()**, **getAttachment()**, **allDocs()**...
+
+Using the document tree, jIO tries to make every version of a document
+available on the two storages. When multiple versions of a document exist,
+Jio will follow the rule set by the conflict_handling option, given at storage creation.
+This option can one of the following numbers:
+
+* 0: no conflict resolution (throws an error when conflict is occuring)
+* 1: keep the local state. (overwrites the remote document with local content)
+* 2: keep the remote state. (overwrites the local document with remote content)
+* 3: keep both copies (leave documents untouched, no signature update)
+
+
+
+Simple Conflict Example
+-----------------------
+
+.. TODO this is a little confusing
+
+You are keeping a namecard file on your PC updating from your smartphone. Your
+smartphone ran out of battery and is offline when you update your namecard on
+your PC with your new email adress. Someone else changes this email from your PC
+and once your smartphone is recharged, you go back online and the previous
+update is executed.
+
+#. Set up the replicate storage:
+
+  .. code-block:: javascript
+
+    var jio_instance = jIO.createJIO({
+      // replicate storage
+      type: 'replicate',
+      local_sub_storage : {
+          type: 'local',
+          ...
+        }
+        remote_sub_storage: {
+          type: 'dav',
+          ...
+        }
+        conflict_handling: ...
+    });
+
+
+#. 1) Create the namecard on your smartphone:
+
+   .. code-block:: javascript
+
+     jio_instance.put("myNameCard", {
+       email: 'jb@td.com'
+      }).then(function (response) {
+       // response -> 'myNameCard'
+     });
+
+   This will create the document on your WebDAV and local storage
+
+#. 2) Someone else updates your shared namecard on WebDAV:
+
+   .. code-block:: javascript
+
+     jio_instance.put(myNameCard, {
+       email: 'kyle@td.com',
+     }).then(function (response) {
+       // response -> 'myNameCard'
+     });
+
+   Your smartphone is offline, so now you will have one version on
+   your smartphone and another version on WebDAV on your PC.
+
+#. 3) Later, your smartphone is online and you modify your email:
+
+   .. code-block:: javascript
+
+     jio_instance.get('myNameCard').then(function (response) {
+       // response.email -> 'jb@td.com'
+       // the get() method checks only on your local storage
+       // and doesn't warn you about remote modifications.
+
+       return jio_instance.put('myNameCard', {
+         email: 'jack@td.com'
+       })
+      .then(function (response) {
+       // response -> 'myNameCard'
+     });
+
+| Your latest modification of the email is: "jack@td.com"
+| The modification from the other user is: "kyle@td.com"
+
+If your conflict_handling option was:
+
+* | 0: the email is:
+  | -"kyle@td.com" on WebDAV
+  | -"jack@td.com" on your local storage
+  | The storage rejects your latest modification,
+  | you get an error because local and remote documents are desynchronized.
+  | The documents in local and remote state are left untouched.
+
+* | 1: the email is: "jack@td.com" on both storages
+  | The storage pushes the local modification, which is yours.
+
+* | 2: the email is: "kyle@td.com" on both storages
+  | The storage keeps the remote modification, which is from the other user.
+  | Your local storage is modified to fit the state of the remote storage.
+
+* | 3: the email is: "jack@td.com" on both storages
+  | The storage doesn't do synchronization, and pushes your modification
+  | without checking if the remote storage has been changed or not