1. 08 Jul, 2016 1 commit
    • Douglas's avatar
      erp5 kernel: improved detection of variables from user context that cannot be put in the zodb · 23e06437
      Douglas authored
      Variables are investigated, recursively in case of container objects (like lists, for example),
      to detect if they can be stored in the ZODB.
      In this investigation persistent objects are identified by being an instance of the object
      class and implementing a `__getstate__` method that raises no exception. If the variable is
      not a Persistent object then we try to pickle and load it.
      
      While developing the pickleable object identification a complication was found. It seems that
      the code cannot capture cPickle.PicklingError in the usual way, `except cPickle.PicklingError`.
      It's consequence of some weirdness with regards to pickle/cPickle modules exceptions classes and
      more about it can be read at http://bugs.python.org/issue1457119. So, the workaround for this complication
      was to catch all exceptions and check the exception class name as string.
      
      The whole check for zodb persistence was moved into an utility function for the sake of readability
      and code maintenance.
      
      The Base_executeJupyter script object was transformed into an extension to be able to properly handle
      transaction errors and render them correctly inside Jupyter.
      23e06437
  2. 29 Jun, 2016 2 commits
    • Douglas's avatar
      erp5_data_notebook: moves variables and setup storage from ActiveProcess to... · 11eb5168
      Douglas authored
      erp5_data_notebook: moves variables and setup storage from ActiveProcess to `notebook_context` property
      
      Notebook's variables and setup were being stored in an ActiveProcess, which is
      not needed anymore, because now everything related to user context can be
      safely stored in the ZODB.
      They are stored in a `notebook_context` property of the Data Notebook
      itself. Code and tests were updated properly. The old `process` property was removed.
      
      All the references to *_variable_dict were renamed to  *_notebook_context, documentation
      and tests were updated. Related objects like scripts and external methods were renamed
      too.
      
      To store objects in the `notebook_context` property we do 2 different tests: the first
      to check if the object can be serialized by `ZODB.serialize.ObjectWriter`. If the first
      test fails we test if the object can be serialized by cPickle. For the second test we
      need to dump & load the object to be completely sure that it can be correctly loaded
      later.
      
      The function called by the Base_runJupyterCode external method was renamed from
      Base_compileJupyterCode to Base_runJupyterCode be more consistent and avoid confusion.
      
      All errors while running setup functions and now properly propagated to the user
      interface in Jupyter and code execution is aborted.
      11eb5168
    • Douglas's avatar
      erp5_data_notebook: environment object implementation and refactoring to ERP5 kernel · 5eb44e82
      Douglas authored
      - An environment object was implemented to help us deal with the multiprocess
      architecture of ERP5 and objects that cannot be easily stored in the ZODB.
      It stores definition of functions, classes and variables as string. The implementation
      uses a dumb Environment class to allow users to make `define` and `undefine` calls,
      which are captured and processed by an AST transformer before code execution.
      
      - Along with the environment object, an automatic "import fixer" was created. It does
      not allow users to import modules as they normally would, because this may cause
      collateral effects on other users' code. A good example is the plot settings in the
      matplotlib module. It will fix normal imports, make them use the environment object
      mentione earlier automatically and warn the user about it.
      
      A few bugs were fixed with this implementation, for example:
      
      - https://nexedi.erp5.net/bug_module/20160318-7098DD, which reports an inconsistency
      on portal catalog queries between Jupyter and Python (Script) objects. Probably an
      issue with user context storage in ActiveProcess
      
      - https://nexedi.erp5.net/bug_module/20160330-13AA193, which reports an error related
      to acquisition when trying to plot images, which happened in other situations, although
      this is not officially reported in Nexedi's ERP5. This probably also was happening because
      of old user context storage.
      5eb44e82
  3. 17 Jun, 2016 5 commits
  4. 15 Jun, 2016 2 commits
  5. 14 Jun, 2016 6 commits
  6. 13 Jun, 2016 5 commits
  7. 10 Jun, 2016 5 commits
  8. 09 Jun, 2016 14 commits