• Bryton Lacquement's avatar
    slapgrid-sr: do not rebootstrap unnecessarily · 7f279836
    Bryton Lacquement authored
    Currently, bootstrapBuildout is called unconditionally, and since
    Software Releases use slapos.reboostrap, we end up with the
    following behaviour in the best case:
    
        Processing software releases...
        Installing software release /srv/slapgrid/slappart9/srv/testnode/aai/software.cfg...
        Generated script '/srv/slapgrid/slappart9/srv/testnode/aai/soft/08f8010370c519147fe23fed3170be9e/bin/buildout'.
        slapos.rebootstrap: Make sure that the section 'python2.7' won't be reinstalled after rebootstrap.
        Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
        Updating xz-utils.
        Updating patch.
        ...
        Updating gcc.
        Updating python2.7.
        Stripping binaries ...
        Done.
        slapos.rebootstrap:
        ************ REBOOTSTRAP: IMPORTANT NOTICE ************
        bin/buildout is being reinstalled right now, as new python:
          /srv/slapgrid/slappart9/srv/testnode/aai/soft/08f8010370c519147fe23fed3170be9e/parts/python2.7/bin/python2.7
        is available, and buildout is using another python:
          /opt/slapgrid/b9f9e0967ab8491759399f306d65239a/parts/python2.7/bin/python2.7
        Buildout will be restarted automatically to have this change applied.
        ************ REBOOTSTRAP: IMPORTANT NOTICE ************
    
        Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
        Updating xz-utils.
        Updating patch.
        ...
        Updating gcc.
        Updating python2.7.
        Updating autoconf.
        ...
    
    Which means that bin/buildout always runs twice: updating parts is usually fast
    but loading extends and checking that binaries are stripped can take a while.
    
    The idea of this commit is minimize the amount of work when bin/buildout
    already exists, in particular if it is already changed by slapos.reboostrap
    to use the built Python. We also hope this will avoid complete rebuilds when
    building a different version of Python:
    
        Installing software release /srv/slapgrid/slappart1/srv/testnode/bsu/software.cfg...
        Generated script '/srv/slapgrid/slappart1/srv/testnode/bsu/soft/90adf14823b5e2bc7dd89ccfbd9388df/bin/buildout'.
        slapos.rebootstrap: Make sure that the section 'python3.5' won't be reinstalled after rebootstrap.
        Uninstalling python3.5.
        Uninstalling file.
        Uninstalling file-msooxml.
        Uninstalling gettext.
        Uninstalling lunzip.
        Uninstalling libxml2.
        Uninstalling sqlite3.
        Uninstalling openssl.
        Uninstalling ca-certificates.
        Uninstalling perl.
        Uninstalling gdbm.
        Uninstalling bzip2.
        Uninstalling libffi.
        Uninstalling libexpat.
        Uninstalling readline.
        Uninstalling ncurses.
        Uninstalling zlib.
        Uninstalling patch.
        Uninstalling xz-utils.
        Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
        Installing xz-utils.
        ...
        Stripping binaries ...
        Done.
        slapos.rebootstrap:
        ************ REBOOTSTRAP: IMPORTANT NOTICE ************
        bin/buildout is being reinstalled right now, as new python:
          /srv/slapgrid/slappart1/srv/testnode/bsu/soft/90adf14823b5e2bc7dd89ccfbd9388df/parts/python3.5/bin/python3.5
        is available, and buildout is using another python:
          /opt/slapgrid/b9f9e0967ab8491759399f306d65239a/parts/python2.7/bin/python2.7
        Buildout will be restarted automatically to have this change applied.
        ************ REBOOTSTRAP: IMPORTANT NOTICE ************
    
        Uninstalling template.
        ...
        Uninstalling m4.
        Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
        Updating xz-utils.
        ...
        Updating python3.5.
        Installing m4.
        ...
    
    About the implementation, we could merely add a `if not os.path.exists(...):`.
    I fear cases where a previous run may have left bin/buildout in a non-working
    state, so I suggest the use of a marker to force bootstrap if a previous run
    failed whereas the existing bin/buildout was reused.
    
    /reviewed-on !54
    7f279836
SlapObject.py 38.7 KB