Generalize how nxdtest python script is generated into pyprog recipe macro

In e328aa49 (component/nxdtest: Prepare for nexedi/nxdtest!13)
I reworked how nxdtest script is generated and splitted it into nxdtest itself
and .nxdtest.pyexe python interpreter, so that sys.executable could be used to
correctly spawn other python scripts:

    3) rework how nxdtest script is generated and split it into .nxdtest.pyexe and
       nxdtest itself. .nxdtest.pyexe is python interpreter via which nxdtest is run.
       This interpreter has all eggs required by nxdtest in sys.path, so that
       nxdtest could spawn its trun.py via sys.executable. If we don't care to have
       properly setup sys.executable, trun.py will fail when importing any module that
       nxdtest.py could already successfully import.

       Initially I tried to workaround this issue via adjusting $PYTHONPATH <-
       sys.path in main nxdtest script, but @jerome points out that, $PYTHONPATH,
       if set, also affects processes that trun.py spawns, which is not good:

       nexedi/slapos!1095 (comment 146799)

       -> so fix this via running nxdtest via environment where sys.executable is
       properly setup python interpreter with path for all eggs that nxdtest has
       access to.

       Because we already have half-way workarounds for similar problem in several
       places, and because running a script with correctly setup sys.executable is
       generally better, I would say it should be a good idea to rework
       zc.recipe.egg:scripts to generate all scripts to work this way, but I do not
       want to fight about it.

       So let's leave this scheme nxdtest-specific for now.

This patch addresses the last paragraph and provides a general pyprog buildout
macro that could be used to generate python script for any entry point to run
with correctly set sys.executable.

/cc @jerome

component/nxdtest/buildout.cfg

@@ -9,15 +9,11 @@ extends =
 # nxdtest is bin/ program to run nxdtest.
 # use ${nxdtest:exe} to run it.
 [nxdtest]
-recipe  = slapos.recipe.template:jinja2
+<= pyprog
 exe     = ${buildout:bin-directory}/nxdtest
-rendered= ${:exe}
-mode    = 755
-eggs    = ${.nxdtest.pyexe:eggs}
-template=
-  inline:
-  #!${.nxdtest.pyexe:exe}
-
+entry   = nxdtest:main
+eggs    = ${nxdtest-egg:egg}
+initialization =
   # $PATH for unshare and mount
   import os
   path = os.environ.get('PATH', '')
@@ -25,19 +21,6 @@ template=
     path = ':' + path
     os.environ['PATH'] = '${util-linux:location}/bin' + path
 
-  from nxdtest import main; main()
-
-
-# .nxdtest.pyexe is python interpreter used by nxdtest.
-# the interpreter is located at ${.nxdtest.pyexe:exe}.
-[.nxdtest.pyexe]
-<= python-interpreter
-eggs +=
-  ${pygolang:egg}
-  ${nxdtest-egg:egg}
-interpreter = ${:_buildout_section_name_}
-exe     = ${buildout:bin-directory}/${:interpreter}
-
 
 [nxdtest-egg]
 recipe  = zc.recipe.egg:develop

component/pygolang/buildout.cfg

@@ -49,5 +49,63 @@ initialization =
 scripts = ${:interpreter}
 
 
+# pyprog provides macro recipe to build python programs.
+#
+# Contrary to zc.recipe.egg:scripts it generates scripts that are run with
+# sys.executable being correctly set. In particular it is valid to spawn
+# sys.executable from inside the program and assume that all specified eggs are
+# still accessible and can be imported.
+#
+# Usage example:
+#
+#   [myprog]
+#   <= pyprog
+#   exe = ${buildout:bin-directory}/myprog
+#   entry = my.py.mod:main
+#   eggs = ...
+[pyprog]
+recipe = slapos.recipe.build
+initialization =
+depends =
+_name = ${:_buildout_section_name_}
+init =
+  name   = options['_name']  # options['_buildout_section_name_'] does not work
+  exe    = options['exe']
+  entry  = options['entry']
+  eggs   = options['eggs']
+  pyinit = options['initialization']
+
+  options['depends'] += '$${.%s.pyprog:recipe}' % name
+
+  # mod:func -> 'from mod import func; func()'
+  mod, func = entry.split(':')
+  entry_run = 'from %s import %s; %s()' % (mod, func, func)
+
+  # indent pyinit with '  '
+  __pyinit = '\n'.join(['  '+_ for _ in pyinit.splitlines()])
+
+  self.buildout.parse("""
+  # .X.pyprog is python program to start and run entry
+  # it uses .X.pyexe as underlying python interpreter
+  [.%(name)s.pyprog]
+  recipe  = slapos.recipe.template:jinja2
+  exe     = %(exe)s
+  rendered= $${:exe}
+  mode    = 755
+  template=
+    inline:
+    #!$${.%(name)s.pyexe:exe}
+  %(__pyinit)s
+    %(entry_run)s
+
+  # .X.pyexe is python interpreter used by .X.pyprog
+  [.%(name)s.pyexe]
+  <= python-interpreter
+  eggs += %(eggs)s
+  interpreter = $${:_buildout_section_name_}
+  exe = $${buildout:bin-directory}/$${:interpreter}
+  """ % locals())
+
+
 [versions]
 pygolang = 0.0.9