slaprunner: improves the import script.

It includes more debugging lines, not deploying a slapproxy (no need anymore thanks
the free-port selection recipe), and not delegating some work to a cron job.

software/slaprunner/common.cfg

@@ -68,7 +68,7 @@ recipe = hexagonit.recipe.download
 ignore-existing = true
 url = ${:_profile_base_location_}/template/runner-import.sh.jinja2
 download-only = true
-md5sum = d3ce78b35cb47dcb647772891a1bf814
+md5sum = 10317dadb2d06befd154bc5e68beab06
 filename = runner-import.sh.jinja2
 mode = 0644
 
@@ -76,7 +76,7 @@ mode = 0644
 recipe = slapos.recipe.template
 url = ${:_profile_base_location_}/instance-runner-import.cfg.in
 output = ${buildout:directory}/instance-runner-import.cfg
-md5sum = 856eda762461264b21e46d17318e2bab
+md5sum = 1921c10aa819205f1b0ec7309173de4e
 mode = 0644
 
 [template-runner-export-script]

software/slaprunner/instance-runner-import.cfg.in

@@ -63,11 +63,15 @@ ip = $${slaprunner:ipv4}
 [supervisord]
 port = $${supervisord-free-port:port}
 
-# Idem for some other services
 [runner-sshd-port]
 minimum = 22232
 maximum = 22241
 
+# Deactivate the call to prepareSoftware, and let the importer script
+# handle the build&run of the instance.
+[cron-entry-prepare-software]
+recipe =
+
 [importer]
 recipe = slapos.recipe.template:jinja2
 template = ${template-runner-import-script:location}/${template-runner-import-script:filename}
@@ -79,6 +83,7 @@ restore-exit-code-file=$${directory:srv}/importer-exit-code-file
 context =
   key backend_url slaprunner:access-url
   key proxy_host slaprunner:ipv4
+  key proxy_port slaprunner:proxy_port
   section directory directory
   raw  output_log_file $${directory:log}/resilient.log
   raw  shell_binary ${bash:location}/bin/bash

software/slaprunner/template/runner-import.sh.jinja2

@@ -62,29 +62,22 @@ $SQLITE3 $DATABASE "update software11 set url='$SOFTWARE_RELEASE' where url='$OL
 # Change slapproxy database to have all instances stopped
 $SQLITE3 $DATABASE "update partition11 set slap_state='stopped';"
 
-# Run slapproxy on different port (in case of running inside of erp5testnode
-# with only one IP and port 50000 already taken by slapproxy of main instance)
-HOST="{{ proxy_host }}"
-PORT="50001"
-URL="http://$HOST:$PORT"
-$SLAPOS proxy start --cfg $HOME/etc/slapos.cfg --port $PORT >/dev/null 2>&1 &
-SLAPPROXY_PID=$!
-trap "kill $SLAPPROXY_PID" EXIT TERM INT
-sleep 5
+MASTERURL="http://{{ proxy_host }}:{{ proxy_port }}"
 
 echo "Building newest software..."
-$SLAPOS node software --cfg $HOME/etc/slapos.cfg --all --master-url=$URL --logfile $HOME/srv/runner/software.log --pidfile $HOME/var/run/slapos-node-software.pid >/dev/null 2>&1 ||
-$SLAPOS node software --cfg $HOME/etc/slapos.cfg --all --master-url=$URL --logfile $HOME/srv/runner/software.log --pidfile $HOME/var/run/slapos-node-software.pid >/dev/null 2>&1 ||
-$SLAPOS node software --cfg $HOME/etc/slapos.cfg --all --master-url=$URL --logfile $HOME/srv/runner/software.log --pidfile $HOME/var/run/slapos-node-software.pid >/dev/null 2>&1
+$SLAPOS node software --cfg $HOME/etc/slapos.cfg --all --master-url=$MASTERURL --logfile $HOME/srv/runner/software.log --pidfile $HOME/var/run/slapos-node-software.pid >/dev/null 2>&1 ||
+$SLAPOS node software --cfg $HOME/etc/slapos.cfg --all --master-url=$MASTERURL --logfile $HOME/srv/runner/software.log --pidfile $HOME/var/run/slapos-node-software.pid >/dev/null 2>&1 ||
+$SLAPOS node software --cfg $HOME/etc/slapos.cfg --all --master-url=$MASTERURL --logfile $HOME/srv/runner/software.log --pidfile $HOME/var/run/slapos-node-software.pid >/dev/null 2>&1
 # Remove defined scripts to force buildout to recreate them to have updated paths
 rm $srv_directory/runner/instance/slappart*/srv/runner-import-restore || true
 echo "Running slapos node instance..."
 # XXX hardcoded
-$SLAPOS node instance --cfg $HOME/etc/slapos.cfg --master-url=$URL --logfile $HOME/srv/runner/instance.log --pidfile $HOME/var/run/slapos-node-instance.pid >/dev/null 2>&1 || true
-$SLAPOS node instance --cfg $HOME/etc/slapos.cfg --master-url=$URL --logfile $HOME/srv/runner/instance.log --pidfile $HOME/var/run/slapos-node-instance.pid >/dev/null 2>&1 || true
-$SLAPOS node instance --cfg $HOME/etc/slapos.cfg --master-url=$URL --logfile $HOME/srv/runner/instance.log --pidfile $HOME/var/run/slapos-node-instance.pid >/dev/null 2>&1 || true
+$SLAPOS node instance --cfg $HOME/etc/slapos.cfg --master-url=$MASTERURL --logfile $HOME/srv/runner/instance.log --pidfile $HOME/var/run/slapos-node-instance.pid >/dev/null 2>&1 || true
+$SLAPOS node instance --cfg $HOME/etc/slapos.cfg --master-url=$MASTERURL --logfile $HOME/srv/runner/instance.log --pidfile $HOME/var/run/slapos-node-instance.pid >/dev/null 2>&1 || true
+$SLAPOS node instance --cfg $HOME/etc/slapos.cfg --master-url=$MASTERURL --logfile $HOME/srv/runner/instance.log --pidfile $HOME/var/run/slapos-node-instance.pid >/dev/null 2>&1 || true
 
 # Invoke defined scripts for each partition inside of slaprunner
+echo "Invoke custom import scripts defined by each instances..."
 for partition in $srv_directory/runner/instance/slappart*/
 do
   script=$partition/srv/runner-import-restore
@@ -95,9 +88,11 @@ do
 done
 
 # Change back slapproxy database to have all instances started
+echo "Start instances..."
 $SQLITE3 $DATABASE "update partition11 set slap_state='started';"
 
 # Write exit code to an arbitrary file that will be checked by promise/monitor
+echo "Write status file... End"
 RESTORE_EXIT_CODE_FILE="{{ restore_exit_code_file }}"
 echo $RESTORE_EXIT_CODE > $RESTORE_EXIT_CODE_FILE
 exit $RESTORE_EXIT_CODE

[Rafael Monnerat]

why this removal? what starts slapproxy now?

[Nicolas Wavrant]

The slapproxy is started by the webrunner. As the comment states, it was necessary some time ago, in the case of erp5testnode, because the slapproxy of the export instance was already using the standard slapproxy port. Now, thanks to the free_port recipe, there is no more conflict and both slapproxy can run at the same time. So no need to start one just for the time of running the import script

[Rafael Monnerat]

Ins't it better have the output of the slapos runs exposed?

What guarantees that slapos node software and slapos node instance succeeded before continue? and how to debug if the command fails? SSH and read logs?

[Nicolas Wavrant]

Hello,

Right, I didn't think that the script is better dying than keeping running if some steps fail. I should re-use the pattern of the "slapos node software" commands here too.

If the commands fail, there is a promise to test that the SR is correctly built, but nothing in the case of instances. If anything fail, there is a resilient.log accessible from the monitoring interface : software.log and instance.log won't be accessible, but thanks to this commit, we can know which step of this script failed. A promise returning an error message returned by this script could indeed be very interesting.

[Rafael Monnerat]

Know if the software or instance finished fine might not be enough if I cannot see why (this is the current problem, never know why things fail).

So it is better have the "tail -n 300 log" if something fail, inside the resilience.log or maybe the full log of everything when import (anyway, instance.log and software.log are not usefull on this case for debug resilience).

[Rafael Monnerat]

Better log/echo the result for each partition individually to know exactly what fail on which partition.

[Nicolas Wavrant]

True