/reviewed-on nexedi/erp5!139

@Tyagov review, please. I'm creating a test suite now and will post about the test results as soon as they are available. 

- An environment object was implemented to help us deal with the multiprocess
architecture of ERP5 and objects that cannot be easily stored in the ZODB.
It stores definition of functions, classes and variables as string. The implementation
uses a dumb Environment class to allow users to make `define` and `undefine` calls,
which are captured and processed by an AST transformer before code execution.

- Along with the environment object, an automatic "import fixer" was created. It does
not allow users to import modules as they normally would, because this may cause
collateral effects on other users' code. A good example is the plot settings in the
matplotlib module. It will fix normal imports, make them use the environment object
mentione earlier automatically and warn the user about it.

Some bugs were fixed with this new implementation: 

- https://nexedi.erp5.net/bug_module/20160318-7098DD, which reports an inconsistency
on portal catalog queries between Jupyter and Python (Script) objects. Probably an
issue with user context storage in ActiveProcess

- https://nexedi.erp5.net/bug_module/20160330-13AA193, which reports an error related
to acquisition when trying to plot images, which happened in other situations, although
this is not officially reported in Nexedi's ERP5. This probably also was happening because
of old user context storage.


/reviewed-on nexedi/erp5!131

Variables are investigated, recursively in case of container objects (like lists, for example),
to detect if they can be stored in the ZODB.
In this investigation persistent objects are identified by being an instance of the object
class and implementing a `__getstate__` method that raises no exception. If the variable is
not a Persistent object then we try to pickle and load it.

While developing the pickleable object identification a complication was found. It seems that
the code cannot capture cPickle.PicklingError in the usual way, `except cPickle.PicklingError`.
It's consequence of some weirdness with regards to pickle/cPickle modules exceptions classes and
more about it can be read at http://bugs.python.org/issue1457119. So, the workaround for this complication
was to catch all exceptions and check the exception class name as string.

The whole check for zodb persistence was moved into an utility function for the sake of readability
and code maintenance.

The Base_executeJupyter script object was transformed into an extension to be able to properly handle
transaction errors and render them correctly inside Jupyter.

Avoid getting all related values when we jump to a module.

Getter of content translations were setting an empty translation dict. So
there was ZODB pollution by doing useless changes.

erp5_web_renderjs_ui_test: Isolate testRenderjsLogout from other tests, otherwise next test fails because of unauthorized error.

erp5_ui_test_core: Don't use Base_doLanguage. Base_doLanguage opens another selenium test runner in the bottom frame of the original selenium test runner by redirect and another selenium test runner does the same thing recursively.

Up to now, once all test result lines in draft were processed,
test result lines already started where affected to all test nodes.
It was designed like this in case the initial affected test node was
unable to finish is work (test node or machine could die for various
reasons). But having a testnode dying should be rare, thus optimisation
should not consider that this happens all the time, even though we
must take into account that this could happen.

This was leading to cases where a testnode, instead of quiting a test
suite to process another was affected a test already affected. So it
happened that we loosed one hour of a testnode while it could do much
more useful work than repeating the work of another testnode.

Thus, consider that testnodes are usually able to process their work,
and make testnodes immediately work on another test suite once all tests
of a test result are started.

Then, run regularly an alarm looking for stuck test to restart them
in order to affect work already affected only when required.

This change should make the system more reactive when things are working
(wich is the majority of time). Not working cases would still finish
to work, but in a less reactive way. If we wait urgently for a test result
and we see that a test is stuck, there is also possibility to unblock
it by hand (if we do not want to wait the alarm).

Cloning a movement as same effect as creating a new one and editing it, so make
sure interaction usually called when editing a movement are called when a movement
is cloned. Otherwise we can get parts of simulation tree missing.

I missed it in commit b0209091

This fixes a regression in commit b9c41350

Commit b45914fa changed the return value of
getFunction() to return more information. But it's easy to preserve the
previous API for any code that would use it, and anyway, it became misnamed.

simplify error handling and avoid code duplication (thanks Romain for advices)

Instead of trying without and retry in case of TypeError, we now have a reliable
rule: aq_parent is passed if and only if the name of first argument is 'self'.

The main optimization is the replacement of
  erp5.component.extension.find_load_module
by a simple
  __import__
like it was before commit c2ce3ba0

Use it on PythonScript & ExternalMethod.

- Code refactoring and small optimizations.
- Really fix unwanted acquisition.
- ExternalMethod: fix security declarations.
- Fix role/permission checking for ExternalMethod called by a PythonScript
  with proxy roles.
- When editing an existing guard, modify it instead of always recreate one.
  no very useful here, but that's good practice: this is nicer for the ZODB
  and it's easier to browse the history. BT should do the same when upgrading.

If for some reasons, this gadget does not get results (usually due to network failures or
service unavailable for other reasons), do not fail and retry until it works again

erp5_data_notebook: moves variables and setup storage from ActiveProcess to `notebook_context` property

Notebook's variables and setup were being stored in an ActiveProcess, which is
not needed anymore, because now everything related to user context can be
safely stored in the ZODB.
They are stored in a `notebook_context` property of the Data Notebook
itself. Code and tests were updated properly. The old `process` property was removed.

All the references to *_variable_dict were renamed to  *_notebook_context, documentation
and tests were updated. Related objects like scripts and external methods were renamed
too.

To store objects in the `notebook_context` property we do 2 different tests: the first
to check if the object can be serialized by `ZODB.serialize.ObjectWriter`. If the first
test fails we test if the object can be serialized by cPickle. For the second test we
need to dump & load the object to be completely sure that it can be correctly loaded
later.

The function called by the Base_runJupyterCode external method was renamed from
Base_compileJupyterCode to Base_runJupyterCode be more consistent and avoid confusion.

All errors while running setup functions and now properly propagated to the user
interface in Jupyter and code execution is aborted.

- An environment object was implemented to help us deal with the multiprocess
architecture of ERP5 and objects that cannot be easily stored in the ZODB.
It stores definition of functions, classes and variables as string. The implementation
uses a dumb Environment class to allow users to make `define` and `undefine` calls,
which are captured and processed by an AST transformer before code execution.

- Along with the environment object, an automatic "import fixer" was created. It does
not allow users to import modules as they normally would, because this may cause
collateral effects on other users' code. A good example is the plot settings in the
matplotlib module. It will fix normal imports, make them use the environment object
mentione earlier automatically and warn the user about it.

A few bugs were fixed with this implementation, for example:

- https://nexedi.erp5.net/bug_module/20160318-7098DD, which reports an inconsistency
on portal catalog queries between Jupyter and Python (Script) objects. Probably an
issue with user context storage in ActiveProcess

- https://nexedi.erp5.net/bug_module/20160330-13AA193, which reports an error related
to acquisition when trying to plot images, which happened in other situations, although
this is not officially reported in Nexedi's ERP5. This probably also was happening because
of old user context storage.