1. 04 Apr, 2018 2 commits
    • Kirill Smelkov's avatar
      slapgrid: Do not leak file descriptors to subprocesses when running 'node software' · 2f5ade0a
      Kirill Smelkov authored
      Some context: I'm trying to build Go 1.10 under slapos and get failures
      like:
      
      	ok   os      0.782s
      	PASS
      	testing: can't write /tmp/go-build511454695/b682/testlog.txt: write /tmp/go-build511454695/b682/testlog.txt: broken pipe
      	FAIL os/exec 0.947s
      	...
      	ok   cmd/vet/internal/cfg    0.002s
      	2018/04/03 18:13:29 Failed: exit status 1
      	golang1.10: Command failed with exit code 1:   cd src && ls -l /proc/self/fd && ./all.bash && cp -alf .. /srv/slapgrid/slappart5/srv/runner/software/099a7368550e95d120639cb03b6d47c8/parts/golang1.10
      	golang1.10: Compilation error. The package is left as is at /srv/slapgrid/slappart5/srv/runner/software/099a7368550e95d120639cb03b6d47c8/parts/golang1.10__compile__/go where you can inspect what went wrong
      	While:
      	  Installing golang1.10.
      	Error: System error
      
      This was traced to https://github.com/golang/go/issues/24285, which in turn was
      traced by me to the fact that golang os/exec tests close all file descriptors >
      4 assuming that only stdin, stdout, stderr, epoll and testlog file descriptors
      are there:
      
      	https://github.com/golang/go/blob/26e0e8a840/src/os/exec/exec_test.go#L402
      	https://github.com/golang/go/blob/26e0e8a840/src/os/exec/exec_test.go#L415
      
      However when go build is run under slapos it starts with the following
      descriptors inherited in the environment:
      
      	lr-x------ 1 slapuser5 slapuser5 64 Apr  3 17:57 0 -> pipe:[2969903608]
      	l-wx------ 1 slapuser5 slapuser5 64 Apr  3 17:57 1 -> pipe:[2969903609]
      	l-wx------ 1 slapuser5 slapuser5 64 Apr  3 17:57 2 -> pipe:[2969903609]
      	l-wx------ 1 slapuser5 slapuser5 64 Apr  3 17:57 3 -> /srv/slapgrid/slappart5/srv/runner/software.log
      
      i.e. the file descriptor for software.log (under `slapos node software`) is
      passed opened to spawned buildout commands -> and oops - go build failure.
      
      I'm not saying go behaviour is correct. However for slapos it is also not good
      to leak e.g. log file descriptors to spawned processes (e.g. what if spawned
      child writes there by mistake?).
      
      Since fixing on Go side is not very easy, because it will probably require
      non-small testing infrastructure refactoring, I decided to fix on SlapOS side,
      because it is a) easier, and b) correct & good to do in any way.
      
      ----
      
      To fix we just pass close_fds=True to subprocess.Popen run from under
      SlapPopen, which means that popen will close all file descriptors besides
      stdin, stdout & stderr for child. close_fds default is already True for python3
      
      	https://www.python.org/dev/peps/pep-0433/#subprocess-close
      
      for the reason that it is frequent mistake for programmers to not leak file
      descriptors to spawned processes.
      
      However on python2, which we are still using, close_fds default is False.
      
      So let's fix the default on our side.
      
      /reviewed-by @jerome, @rafael
      /reviewed-on nexedi/slapos.core!42
      2f5ade0a
    • Jérome Perrin's avatar
      grid: do not hide `$USER` when running buildout · 66847322
      Jérome Perrin authored
      Hiding such environment variable prevent easy access to the user
      owning the software or the user owning the instance.
      
      Some instances needs to have the actual username of the instance in their
      config files and some softwares needs access to the username of user
      building the software. This is - in theory - easy to do in zc.buildout
      thanks to https://collective.github.io/collective.recipe.grp/ , but the
      way slapos run buildout with some environment variables hidden made it
      not possible.
      
      Fixes [#20180402-716ACB](https://nexedi.erp5.net/bug_module/20180402-716ACB/)
      
      /reviewed-on !41
      66847322
  2. 29 Mar, 2018 4 commits
  3. 28 Mar, 2018 7 commits
  4. 27 Mar, 2018 7 commits
  5. 22 Mar, 2018 1 commit
  6. 19 Mar, 2018 3 commits
  7. 16 Mar, 2018 5 commits
  8. 14 Mar, 2018 11 commits