stack.erp5: Stop using slapos.cookbook:generic.zope.zeo.client .

Such complex recipes prevent evolution. Split it into individual
components already covered by existing recipes.

stack/erp5/buildout.cfg

@@ -184,10 +184,10 @@ md5sum = 5691b7f12d22bfb8a926dfcb592f12c8
 filename = zope.conf.in
 md5sum = 77ab4f229a92e02603028a0bd3772edd
 
-[template-runzope-userhosts-preloaded]
+[site-zcml]
 <= download-base
-filename = runzope_userhosts_preloaded.in
-md5sum = bc6048b85b410693e60e5a77399dd1b7
+filename = site.zcml
+md5sum = d32417746fcf671d4e86a70379815039
 
 [template-my-cnf]
 <= download-base
@@ -234,7 +234,7 @@ recipe = slapos.recipe.template:jinja2
 # XXX: "template.cfg" is hardcoded in instanciation recipe
 rendered = ${buildout:directory}/template.cfg
 template = ${:_profile_base_location_}/instance.cfg.in
-md5sum = 10d02ec69c875e6a55afe3bf79db7bba
+md5sum = 83aaa4def66c5667049c1fd025949125
 mode = 640
 context =
     key mariadb_link_binary template-mariadb:link-binary
@@ -288,6 +288,7 @@ context =
     key pixman_location pixman:location
     key postfix_location postfix:location
     key root_common root-common:target
+    key site_zcml site-zcml:target
     key sixtunnel_location 6tunnel:location
     key slapos_core_version versions:slapos.core
     key stunnel_location stunnel:location
@@ -308,7 +309,6 @@ context =
     key template_postfix_aliases template-postfix-aliases:target
     key template_postfix_main_cf template-postfix-main-cf:target
     key template_postfix_master_cf template-postfix-master-cf:target
-    key template_runzope_userhosts_preloaded template-runzope-userhosts-preloaded:target
     key template_zeo template-zeo:target
     key template_zope template-zope:target
     key template_zope_conf template-zope-conf:target
@@ -341,7 +341,7 @@ md5sum = b0cb0ee97cddc79112a718e065806037
 [template-zope]
 <= download-base
 filename = instance-zope.cfg.in
-md5sum = 83304f1940e0d7555bd678e2cea4c738
+md5sum = c42bb374f41654433fb9470c1d4e4483
 link-binary =
   ${aspell:location}/bin/aspell
   ${dmtx-utils:location}/bin/dmtxwrite

stack/erp5/instance-zope.cfg.in

@@ -162,20 +162,6 @@ template = inline: {{ '
 rendered = ${directory:etc}/hosts
 context = key host_dict hosts-parameter:host-dict
 
-[preload-userhosts-runzope-parameter]
-runzope-binary = {{ bin_directory }}/runzope
-userhosts = {{ parameter_dict['userhosts'] }}
-shell-path = {{ parameter_dict['dash'] }}/bin/dash
-hosts = ${hosts:rendered}
-hostaliases = ${hostaliases:rendered}
-
-[preload-userhosts-runzope]
-recipe = slapos.recipe.template:jinja2
-rendered = ${directory:bin}/runzope_userhosts_preloaded
-context = section parameter_dict preload-userhosts-runzope-parameter
-template = {{ parameter_dict['runzope-userhosts-preloaded-template'] }}
-mode = 755
-
 {# Hack to deploy SSL certs via instance parameters -#}
 {% for zodb in zodb_dict.itervalues() -%}
 {%   set storage_dict = zodb.setdefault('storage-dict', {}) -%}
@@ -197,19 +183,31 @@ pem = {{dumps(storage_dict.pop(k))}}
 {% endfor -%}
 {# endhack -#}
 
-[zope-base]
-recipe = slapos.cookbook:generic.zope.zeo.client
-inituser = ${directory:instance}/inituser
-user = {{ dumps(slapparameter_dict['inituser-login']) }}
-password = {{ dumps(slapparameter_dict['inituser-password']) }}
-timezone = {{ dumps(slapparameter_dict['timezone']) }}
-tmp-path = ${directory:tmp}
-instancehome-path = ${directory:instance}
-home-path = ${buildout:directory}
-bin-path = ${directory:bin}:{{ parameter_dict['coreutils'] }}/bin
-site-zcml = ${directory:instance-etc}/site.zcml
-runzope-binary = ${preload-userhosts-runzope:rendered}
-bt5-repository =
+[runzope-base]
+recipe = slapos.cookbook:wrapper
+environment =
+  HOSTALIASES=${hostaliases:rendered}
+  HOSTS=${hosts:rendered}
+  TMP=${directory:tmp}
+  TMPDIR=${directory:tmp}
+  HOME=${buildout:directory}
+  PATH=${directory:bin}:{{ parameter_dict['coreutils'] }}/bin
+  TZ={{ slapparameter_dict['timezone'] }}
+  INSTANCE_HOME=${directory:instance}
+parameters-extra = true
+command-line = '{{ parameter_dict['userhosts'] }}' '{{ bin_directory }}/runzope' -C '${:configuration-file}'
+
+[{{ section('zcml') }}]
+recipe = slapos.cookbook:symbolic.link
+target-directory = ${directory:instance-etc}
+link-binary = {{ parameter_dict['site-zcml'] }}
+
+[{{ section('zope-inituser') }}]
+recipe = slapos.recipe.template:jinja2
+rendered = ${directory:instance}/inituser
+template = inline:{{ slapparameter_dict['inituser-login'] }}:{SHA}{{ hashlib.sha1(slapparameter_dict['inituser-password']).digest().encode('base64').rstrip() }}
+mode = 600
+once = ${:rendered}_done
 
 [zope-conf-parameter-base]
 ip = {{ ipv4 }}
@@ -309,8 +307,8 @@ context =
   section parameter_dict {{ conf_parameter_name }}
 
 [{{ section(name) }}]
-< = zope-base
-wrapper = ${directory:service-on-watch}/{{ name }}
+< = runzope-base
+wrapper-path = ${directory:service-on-watch}/{{ name }}
 configuration-file = {{ '${' ~ conf_name ~ ':rendered}' }}
 
 [{{ section("promise-" ~ name) }}]

stack/erp5/instance.cfg.in

@@ -122,7 +122,7 @@ dash = {{ dash_location }}
 jsl = {{ jsl_location }}
 link-binary = {{ dumps(zope_link_binary) }}
 userhosts = {{ userhosts_location }}
-runzope-userhosts-preloaded-template = {{ template_runzope_userhosts_preloaded }}
+site-zcml = {{ site_zcml }}
 template-monitor = {{ dumps(template_monitor) }}
 extra-path-list = {{ dumps(extra_path_list) }}
 
@@ -134,6 +134,7 @@ extra-context =
     key buildout_directory buildout:directory
     section parameter_dict dynamic-template-zope-parameters
     import urlparse urlparse
+    import hashlib hashlib
 
 [dynamic-template-kumofs-parameters]
 dash-location = {{ dash_location }}

stack/erp5/runzope_userhosts_preloaded.in

-#!{{ parameter_dict['shell-path'] }}
-HOSTALIASES='{{ parameter_dict['hostaliases'] }}' HOSTS='{{ parameter_dict['hosts'] }}' exec '{{ parameter_dict['userhosts'] }}' '{{ parameter_dict['runzope-binary'] }}' "$@"

stack/erp5/site.zcml

+<configure
+    xmlns="http://namespaces.zope.org/zope"
+    xmlns:meta="http://namespaces.zope.org/meta"
+    xmlns:five="http://namespaces.zope.org/five">
+
+  <include package="Products.Five" />
+  <meta:redefinePermission from="zope2.Public" to="zope.Public" />
+
+
+  <!-- Load the meta -->
+  <include files="package-includes/*-meta.zcml" />
+  <five:loadProducts file="meta.zcml"/>
+
+  <!-- Load the configuration -->
+  <include files="package-includes/*-configure.zcml" />
+  <five:loadProducts />
+
+  <!-- Load the configuration overrides-->
+  <includeOverrides files="package-includes/*-overrides.zcml" />
+  <five:loadProductsOverrides />
+
+
+  <securityPolicy
+      component="Products.Five.security.FiveSecurityPolicy" />
+
+</configure>

[Nicolas Wavrant]

@vpelletier, @rafael, @alain.takoudjou : Beware, this section breaks the processing of the zope slappart in the case it was instanciated with a version of slapos.git older than this commit .

ie : for ERP5s inside webrunners, when the software.cfg of a project does :

[buildout]
extends =
  https://lab.nexedi.com/nexedi/slapos/raw/master/software/erp5/software.cfg

Then the clones fail to process the slappart when restoring the backup with the following error message :

[2017-01-18 11:30:27,995] INFO     While:
[2017-01-18 11:30:27,995] INFO       Installing zcml.
[2017-01-18 11:30:27,995] INFO     Error: Target '/srv/slapgrid/slappartXX/srv/runner/instance/slappart6/srv/erp
5shared/etc/site.zcml' already exists but is not a link

I don't know if the correct solution would be to enforce projects to set a revision when extending repositories with buildout, force developers to write migration compatibility code (not very good with buildout, as it would end by using the command recipe which sucks considering indempotency), completely delete/recreate slapparts in the case of rebuilding the backup in webrunner clones (would be the all-cases proof solution), or let the vifib team manually fix it as this kind of errors happen very infrequently.

[Nicolas Wavrant]

After a second thought : I knew the last idea was bad (let vifib team do once...), but in fact it's even just impossible. Until ERP5 is not redeployed on the webrunner-export with the new version of slapos, resiliency will always export site.zcml as a file, so the instance processing on webrunner-import will always fail (as site.zcml already exists but is not a link).

[Vincent Pelletier]

this section breaks the processing of the zope slappart in the case it was instanciated with a version of slapos.git older than this commit .

Yes, because the previous recipe had a bug (it should really have reported that file to buildout so it would get deleted when the section changes).

I had a fix for that yesterday, which I now see I forgot to push. I now pushed 0f62d562 .

enforce projects to set a revision when extending repositories with buildout

This is not a good idea solution for this issue beyond technical limitation: in-place migration must work, at least outside of (hopefully rare) conscious compatibility breaks. This was not a conscious compatibility break, but the effect of a bogus recipe.