Commit e3566a11 authored by Kirill Smelkov's avatar Kirill Smelkov

Generalize how nxdtest python script is generated into pyprog recipe macro

In e328aa49 (component/nxdtest: Prepare for nexedi/nxdtest!13)
I reworked how nxdtest script is generated and splitted it into nxdtest itself
and .nxdtest.pyexe python interpreter, so that sys.executable could be used to
correctly spawn other python scripts:

    3) rework how nxdtest script is generated and split it into .nxdtest.pyexe and
       nxdtest itself. .nxdtest.pyexe is python interpreter via which nxdtest is run.
       This interpreter has all eggs required by nxdtest in sys.path, so that
       nxdtest could spawn its trun.py via sys.executable. If we don't care to have
       properly setup sys.executable, trun.py will fail when importing any module that
       nxdtest.py could already successfully import.

       Initially I tried to workaround this issue via adjusting $PYTHONPATH <-
       sys.path in main nxdtest script, but @jerome points out that, $PYTHONPATH,
       if set, also affects processes that trun.py spawns, which is not good:

       nexedi/slapos!1095 (comment 146799)

       -> so fix this via running nxdtest via environment where sys.executable is
       properly setup python interpreter with path for all eggs that nxdtest has
       access to.

       Because we already have half-way workarounds for similar problem in several
       places, and because running a script with correctly setup sys.executable is
       generally better, I would say it should be a good idea to rework
       zc.recipe.egg:scripts to generate all scripts to work this way, but I do not
       want to fight about it.

       So let's leave this scheme nxdtest-specific for now.

This patch addresses the last paragraph and provides a general pyprog buildout
macro that could be used to generate python script for any entry point to run
with correctly set sys.executable.

/cc @jerome
parent dd7038fe
...@@ -9,15 +9,11 @@ extends = ...@@ -9,15 +9,11 @@ extends =
# nxdtest is bin/ program to run nxdtest. # nxdtest is bin/ program to run nxdtest.
# use ${nxdtest:exe} to run it. # use ${nxdtest:exe} to run it.
[nxdtest] [nxdtest]
recipe = slapos.recipe.template:jinja2 <= pyprog
exe = ${buildout:bin-directory}/nxdtest exe = ${buildout:bin-directory}/nxdtest
rendered= ${:exe} entry = nxdtest:main
mode = 755 eggs = ${nxdtest-egg:egg}
eggs = ${.nxdtest.pyexe:eggs} initialization =
template=
inline:
#!${.nxdtest.pyexe:exe}
# $PATH for unshare and mount # $PATH for unshare and mount
import os import os
path = os.environ.get('PATH', '') path = os.environ.get('PATH', '')
...@@ -25,19 +21,6 @@ template= ...@@ -25,19 +21,6 @@ template=
path = ':' + path path = ':' + path
os.environ['PATH'] = '${util-linux:location}/bin' + path os.environ['PATH'] = '${util-linux:location}/bin' + path
from nxdtest import main; main()
# .nxdtest.pyexe is python interpreter used by nxdtest.
# the interpreter is located at ${.nxdtest.pyexe:exe}.
[.nxdtest.pyexe]
<= python-interpreter
eggs +=
${pygolang:egg}
${nxdtest-egg:egg}
interpreter = ${:_buildout_section_name_}
exe = ${buildout:bin-directory}/${:interpreter}
[nxdtest-egg] [nxdtest-egg]
recipe = zc.recipe.egg:develop recipe = zc.recipe.egg:develop
......
...@@ -49,5 +49,63 @@ initialization = ...@@ -49,5 +49,63 @@ initialization =
scripts = ${:interpreter} scripts = ${:interpreter}
# pyprog provides macro recipe to build python programs.
#
# Contrary to zc.recipe.egg:scripts it generates scripts that are run with
# sys.executable being correctly set. In particular it is valid to spawn
# sys.executable from inside the program and assume that all specified eggs are
# still accessible and can be imported.
#
# Usage example:
#
# [myprog]
# <= pyprog
# exe = ${buildout:bin-directory}/myprog
# entry = my.py.mod:main
# eggs = ...
[pyprog]
recipe = slapos.recipe.build
initialization =
depends =
_name = ${:_buildout_section_name_}
init =
name = options['_name'] # options['_buildout_section_name_'] does not work
exe = options['exe']
entry = options['entry']
eggs = options['eggs']
pyinit = options['initialization']
options['depends'] += '$${.%s.pyprog:recipe}' % name
# mod:func -> 'from mod import func; func()'
mod, func = entry.split(':')
entry_run = 'from %s import %s; %s()' % (mod, func, func)
# indent pyinit with ' '
__pyinit = '\n'.join([' '+_ for _ in pyinit.splitlines()])
self.buildout.parse("""
# .X.pyprog is python program to start and run entry
# it uses .X.pyexe as underlying python interpreter
[.%(name)s.pyprog]
recipe = slapos.recipe.template:jinja2
exe = %(exe)s
rendered= $${:exe}
mode = 755
template=
inline:
#!$${.%(name)s.pyexe:exe}
%(__pyinit)s
%(entry_run)s
# .X.pyexe is python interpreter used by .X.pyprog
[.%(name)s.pyexe]
<= python-interpreter
eggs += %(eggs)s
interpreter = $${:_buildout_section_name_}
exe = $${buildout:bin-directory}/$${:interpreter}
""" % locals())
[versions] [versions]
pygolang = 0.0.9 pygolang = 0.0.9
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment