- ensure that gpg key for opensuse repo is present
 - pre-fill cache with specific node to faster bootstrap in re6stnet-upgrader
 - re6stnet-upgrader doesn't fail if previous version of package was also a re6st-node
 - repositories keys have to be installed before updating package cache
 - re6stnet-upgrader ensures that service is enabled
 - only update once package cache in re6stnet-upgrader

Makefile was so horrible and unreliable.
I tried https://waf.io/ which is nice but not suitable for this
(see the example 'wscript' at the end).

Functional improvements:
- better detection of what needs to be rebuilt or not
- reproducible tarballs, except for the re6stnet egg
  (and the main tarball if the egg is rebuilt)
- fewer temporary files

And support for OSC is back.

_______________________________________________________________________________

import os, shutil, subprocess, urllib
from waflib import Node, Utils

PREFIX = "opt/re6st"
BOOTSTRAP_URL = "http://downloads.buildout.org/1/bootstrap.py"
repo_dict = dict(
    re6stnet="http://git.erp5.org/repos/re6stnet.git",
    slapos="http://git.erp5.org/repos/slapos.git",
)

def configure(ctx):
    for name, url in repo_dict.iteritems():
        if ctx.path.find_node(name) is None:
            ctx.exec_command(("git", "clone", url), stdout=None, stderr=None)

def cfg(task):
    o, = task.outputs
    bld = task.generator.bld
    o.write(task.inputs[0].read() % dict(
        SLAPOS=bld.srcnode.search_node("slapos").abspath(),
        ROOT="${buildout:directory}/" + bld.bldnode.path_from(o.parent),
        TARGET="/"+PREFIX))

def bootstrap(task):
    b = task.outputs[0]
    d = b.parent.parent
    bootstrap = urllib.urlopen(BOOTSTRAP_URL).read()
    for cache in "download-cache", "extends-cache":
        cache = d.make_node(cache).abspath()
        if os.path.exists(cache):
            shutil.rmtree(cache)
        os.mkdir(cache)
    cwd = d.abspath()
    task.exec_command(("python2", "-S"), input=bootstrap, cwd=cwd)
    task.exec_command((b.abspath(), "buildout:parts=python"), cwd=cwd)

def sdist(task):
    r, p = task.inputs
    d = p.find_node("../../download-cache/dist")
    for x in d.ant_glob("re6stnet-*", quiet=True):
        x.delete()
    task.exec_command((p.abspath(), "setup.py", "sdist", "-d", d.abspath()),
                      cwd=r.abspath())

def build(bld):
    b = bld.bldnode.make_node(PREFIX)
    buildout_cfg = b.make_node("buildout.cfg")
    bld(source="buildout.cfg.in", target=buildout_cfg, rule=cfg)
    tg = bld(source=(buildout_cfg, "slapos"), rule=bootstrap,
             target=map(b.make_node, ("bin/buildout", "bin/python")))
    buildout, python = tg.target
    r = bld.path.find_node("re6stnet")
    tg = bld(source=(r, python), rule=sdist, update_outputs=True,
             target=r.make_node("re6stnet.egg-info/PKG-INFO"))
    bld(name="buildout", source=(buildout_cfg, tg.target),
        target=b.make_node(".installed.cfg"),
        rule=lambda task: task.exec_command((buildout.abspath(),),
            stdout=None, stderr=None, cwd=b.abspath()))

 ###

def h_file(fname, h_file=Utils.h_file):
    if os.path.isdir(fname):
        m = Utils.md5(fname)
        n = len(fname) + 1
        for dirpath, dirs, files in os.walk(fname):
            if dirpath.endswith(("/.git", ".egg-info")):
                del dirs[:]
                continue
            dirs.sort()
            files.sort()
            m.update(dirpath[n:] + '\0')
            for fname in files:
                m.update("%s\0%s\0" % (h_file(os.path.join(dirpath, fname)),
                                       fname))
        return m.digest()
    return h_file(fname)
Utils.h_file = h_file

def find_resource(self, lst):
    if isinstance(lst, str):
        lst = [x for x in Node.split_path(lst) if x and x != '.']
    return self.get_bld().search_node(lst) \
        or self.get_src().find_node(lst)
Node.Node.find_resource = find_resource

- With a Makefile instead of a shell script to make build files,
  it's easier to only redo the necessary parts.
- Do not bootstrap buildout in advance. This caused build failures on old
  dists when this was done with Python 2.7 (parts/buildout/site.py imported
  a module that does not exist on Python 2.6). Now everything can be prepared
  on a recent dist.
- Minimize list of build-deps. We don't need setuptools anymore.
  We also don't rely on system chrpath anymore because SLE_12 does not have it.
- Refactoring with upstream packaging.
- Debian: fix upgrade path from 're6stnet' package.
- Package re6st manpages and logrotate configuration.
- systemd integration on RPM-based dists.
- Fix shebang of OpenVPN/NetworkManager hooks.
- Do never ship *.py[co] files in packages. Remove them on uninstallation
  (not done yet for Arch).
- chrpath: Suse dists use RUNPATH instead of PATH
- Stop stripping ELF files.
- Greatly reduce size of packages by removing eggs/parts that were only useful
  during the build, like Perl.

TODO: integrate with osc working copy ?

For the record, here is an example to strip ELF files:

strip:
	cd $(DESTDIR)/$(TARGET) \
	&& s() { chmod u+w $$x && strip -v -R .comment -R .note "$$@" $$x; } \
	&& x=`find -type f -executable ! -name '*.so*' |xargs -r file | \
		sed -n 's/:.*ELF.*\(executable\|shared\).*//p'` && s \
	&& x=`find -type f -name '*.so*' |xargs -r file | \
		sed -n 's/:.*ELF.*shared.*//p'` && s --strip-unneeded

 This is important for legacy machines w/o upgrader or ansible installed.

  Don't upgrade slapos or re6st for now vifib upgrader, only update ntp and include vifib team ssh keys

  Spin-off Channels for SlapOS/VIFIB/Grandenet from VM related ones.
  Fixes for dev/ansible-only installation
  Include more index.html with autoredirect.
  Include README file with explanation for the channels.

  Those changes are required to create the package at the OBS VMs
  environment and they are not needed for average buildout installations.

Logging

See merge request !18