but running it shows a lot of errors. Will investigate

This reverts commit 27d502dc.

This reverts commit 6083a0f1.

This reverts commit 394dfa82.

This reverts commit d2898b06.

This reverts commit c9dd5d55.

version up : OpenBLAS 0.2.15. First try with auto-detected target and if it fails try TARGET=GENERIC.

You can also spcify the target explicitly by build-ext-options.

reviewd-by @Tyagov (on nexedi/slapos!42).

This patch series teaches ERP5 software release to automatically instantiate
Jupyter notebook web UI and tune it to connect to ERP5 by default. When Jupyter
is enabled, it also installs on-server erp5_data_notebook bt5 (see
nexedi/erp5!29 and nexedi/erp5@f662b5a2) which handles code execution requested
for Jupyter.

For ERP5 - for security and backward compatibility reasons - Jupyter
instantiation and erp5_data_notebook bt5 install happen only if jupyter is
explicitly enabled in instance parameters. The default is not to have Jupyter
out of the box.

On the other hand for Wendelin SR, which inherits from ERP5 SR, the
default is to have Jupyter out of the box, because Wendelin SR is fresh
enough without lots of backward compatibility needs, and Jupyter is
usually very handy for people who use Wendelin.

--------

NOTE Currently erp5-data-notebook bt5 has the following limitations (see
details on nexedi/slapos!43 and nexedi/erp5!29):

- errors are not reported properly to users;
- state is not fully saved to ZODB.

the latter point means notebook works only if it is connected to Zope family
with only 1 zope process. Hopefully this will be resolved some day.

Technical overview about how the integration is done itself on slapos part and
other notes are in 0a446263.

/proposed-for-review-on nexedi/slapos!43

Query strings used on GET requests have size limitations
on servers and this causes big code cells to not be executed
at all, returning only an Internal Server Error with no
further explanation.

/reviewed-by @kirr, @Tyagov (on nexedi/slapos!43)

Pandas and scikit-learn are handy to have for data processing which we
are going to do more and more in ERP5 context. Matplotlib is very handy
to have when one works with Jupyter, but also can be used by just
backend code to generate graphs.

From this point of view it makes sense to have this eggs always
available out of the box. To do so move them from Wendelin to ERP5.

/suggested-by @tiwariayush
/reviewed-by @Tyagov (on nexedi/slapos!43)

This patch teaches ERP5 software release to automatically instantiate Jupyter
notebook web UI and tune it to connect to ERP5 by default. When Jupyter is
enabled, it also installs on-server erp5_data_notebook bt5 (nexedi/erp5!29)
which handles code execution requested for Jupyter.

For ERP5 - for security and backward compatibility reasons - Jupyter
instantiation and erp5_data_notebook bt5 install happen only if jupyter is
explicitly enabled in instance parameters. The default is not to have Jupyter
out of the box.

On the other hand for Wendelin SR, which inherits from ERP5 SR, the
default is to have Jupyter out of the box, because Wendelin SR is fresh
enough without lots of backward compatibility needs, and Jupyter is
usually very handy for people who use Wendelin.

~~~~

For integration, we reuse already established in ERP5 infrastructure, to
request various slave instances, and request Jupyter in a way so it
automatically tunes and connects to balancer of one of Zope family.

Jupyter code itself is compiled by reusing
software/ipython_notebook/software.cfg, and Jupyter instance code is
reused by hooking software/ipython_notebook/instance.cfg.in into ERP5 SR
properly (the idea to override instance-jupyter not to render into
default template.cfg is taken from previous work by @tiwariayush).

~~~~

I tested this patch inside webrunner with create-erp5-site software type and
various configurations (whether to have or not have jupyter, to which zope
family to connect it, etc).

I have not tested frontend instantiation fully - because tests were done only
in webrunner, but I've tried to make sure generated buildout code is valid for
cases with frontend.

NOTE the code in this patch depends erp5_data_notebook bt5 (nexedi/erp5!29) which just got merged to erp5.git recently (see nexedi/erp5@f662b5a2)

NOTE even when erp5_data_notebook bt5 is installed, on a freshly installed ERP5, it
is required to "check site consistency" first, so that initial bt5(s) are
actually installed and erp5 is ready to function.

/cc @vpelletier, @Tyagov, @klaus, @Camata, @tiwariayush, @Kreisel, @jerome, @nexedi
/proposed-for-review-on nexedi/slapos!43

Gitlab uses github-markup to render various text-based markups
(markdown, rst, ...) into html. For rst github-markup wants to run
python and have docutils egg available:

    https://github.com/github/markup/blob/5393ae93/lib/github/markups.rb#L36

as we were not having docutils installed and path to proper python
interpreter setup, rst documents were not automatically rendered and
were show just as plain text.

We do a lot of documents in rst - that case is important for us.

So fix it by providing gitlab with properly setup python interpreter
with all needed eggs installed.

/cc @kazuhiko, @jerome
/proposed-for-review-on nexedi/slapos!39

The reason is: we are going to integrate Jupyter into ERP5 SR, and in
ERP5 SR code, there is already established infrastructure, which
requests slave instances. That infrastructure passes parameters in
serialized (= json) form, that's why we need to switch.

On the other hand, slapos.cookbook:slapconfiguration.serialised handles
non-json instance parameters as well: when there parameters format is
not

    {'_' -> <json>}

it decodes them as slapos.cookbook:slapconfiguration would do:

    https://lab.nexedi.com/nexedi/slapos/blob/69229988/slapos/recipe/slapconfiguration.py#L232
    https://lab.nexedi.com/nexedi/slapos/blob/69229988/slapos/recipe/librecipe/__init__.py#L51

So from this point of view, though we don't really need it here (Jupyter
SR is relatively new), we are staying backward compatible.

/cc @tiwariayush

Both numpy and ipython are included in ERP5 SR which wendelin inherits
from, and are pinned there. Here are e.g. latest pin-ups for numpy and
ipython in erp5:

    e3144a8a    (version up eggs.)
    135570c9    (version up eggs.)

Furthermore: this is not only a cleanup. As e3144a8a shows current
version of numpy in erp5 is 1.10.4 and in wendelin we still have 1.9.2
which is unintentional downgrade compared to erp5.

Don't do that.

/cc @kazuhiko
/reviewed-by @Tyagov  (on nexedi/slapos!41)

@rafael added this in 971d0bb7 (erp5: Make possible extent the list of
initial business templates to install), but we dropped that change while
merging erp5-cluster to master - see:

    6bbb61a8 "Merge branch 'master' into erp5-cluster", and
    e84d5e83 "Merge branch 'erp5-cluster'"

6bbb61a8 claimed that it

    Dropped commit 971d0bb7
    ("erp5: Make possible extent the list of initial business templates to install").

but it actually dropped changes only under stack/erp5/ , not software/wendelin/

Fix it.

/cc @rafael, @jm, @Tyagov
/reviewed-by TrustMe

In slapos.package.git/obs, we need to build binaries for specific paths,
without changing where buildout actually install them.