slapos:438bfe91929b828599b8ad97c3635228cb1959bc commitshttps://lab.node.vifib.com/nexedi/slapos/-/commits/438bfe91929b828599b8ad97c3635228cb1959bc2018-06-22T16:27:11+02:00https://lab.node.vifib.com/nexedi/slapos/-/commit/438bfe91929b828599b8ad97c3635228cb1959bcapache-frontend: Do not remove 'secure' from cookies2018-06-22T16:27:11+02:00Lukasz Nowakluke@nexedi.com
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/346" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/346" data-link="false" data-link-reference="true" data-project="15" data-merge-request="2158" data-project-path="nexedi/slapos" data-iid="346" data-mr-title="apache-frontend: Do not remove 'secure' from cookies" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!346</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/21b2d1c174bb35a49dbaee5154004f59ca385703Version up^Clapos.cookbook 1.0.662018-06-22T12:33:04+02:00Rafael Monneratrafael@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/54b0d8748e64852f01c0a0bcc554db25b525219bRelease slapos.cookbook 1.0.662018-06-22T12:31:13+02:00Rafael Monneratrafael@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/2f8fad85829cb15c3a2cad2d815cfa9fa57c03daMake update-hash file path handling portable.2018-06-22T10:36:32+09:00Vincent Pelletiervincent@nexedi.com
Filename is split as an URL path, and joined in OS-dependent manner.
Remove corresponding comment in all buildout.hash.cfg files.https://lab.node.vifib.com/nexedi/slapos/-/commit/ee6ca37f08303e905ced751ccbbe2baa009f4025slapos.recipe.erp5_test: cloudooo-retry-count should be optional.2018-06-21T18:23:54+02:00Rafael Monneratrafael@nexedi.com
This prevent outdated instances to fail resilience, so it makes this change backward compatible.https://lab.node.vifib.com/nexedi/slapos/-/commit/3da76768a75716aea8929b501f9b1e1d85e721d0apache-frontend: Set consistent username on log access2018-06-13T13:05:46+02:00Lukasz Nowakluke@nexedi.com
Do not upper filenames nor users, use them as is.
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/339" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/339" data-link="false" data-link-reference="true" data-project="15" data-merge-request="2127" data-project-path="nexedi/slapos" data-iid="339" data-mr-title="apache-frontend: Set consistent username on log access" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!339</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/38d862bb23c43a4527fef6d0a304d5b5822a57ebapache-frontend: Fix publish condition2018-06-13T12:33:56+02:00Lukasz Nowakluke@nexedi.com
No custom_http nor custom_https shall publish information normally calculated.
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/340" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/340" data-link="false" data-link-reference="true" data-project="15" data-merge-request="2128" data-project-path="nexedi/slapos" data-iid="340" data-mr-title="apache-frontend: Fix publish condition" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!340</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/fe467a41bd06e100bc4bec540441d10d662b1528gitlab: v↑ git-backup2018-06-13T09:38:57+02:00Kirill Smelkovkirr@nexedi.com
This brings the following changes of git-backup:
<a href="https://lab.nexedi.com/kirr/git-backup/compare/0b8d834b...cc6ac54f" data-original="https://lab.nexedi.com/kirr/git-backup/compare/0b8d834b...cc6ac54f" data-link="false" data-link-reference="true" data-project="96" data-commit-range="0b8d834bd00b19169f65d407eab8980076779b9d...cc6ac54f451dfa6e343d6340dcfa25aa6eac9565" data-reference-type="commit_range" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-commit_range has-tooltip">kirr/git-backup@0b8d834b...cc6ac54f</a>
the most important of which is dramatical speedup of pull and 30-50%
speedup of restore if bitmap index of backup repo is built and git is
recent enough.
Please see the following commits for details:
<a href="https://lab.nexedi.com/kirr/git-backup/commit/899103bf" data-original="https://lab.nexedi.com/kirr/git-backup/commit/899103bf" data-link="false" data-link-reference="true" data-project="96" data-commit="899103bfb69341a80695a0851b784c04364043b4" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="pull: Switch from porcelain `git fetch` to plumbing `git fetch-pack` + friends" class="gfm gfm-commit has-tooltip">kirr/git-backup@899103bf</a>
<a href="https://lab.nexedi.com/kirr/git-backup/commit/3efed898" data-original="https://lab.nexedi.com/kirr/git-backup/commit/3efed898" data-link="false" data-link-reference="true" data-project="96" data-commit="3efed898e1f4f979d382956fd500c2d4ce0141b9" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="pull: Speedup fetching by prebuilding index of objects we already have at start" class="gfm gfm-commit has-tooltip">kirr/git-backup@3efed898</a>
<a href="https://lab.nexedi.com/kirr/git-backup/commit/0ab7bbb6" data-original="https://lab.nexedi.com/kirr/git-backup/commit/0ab7bbb6" data-link="false" data-link-reference="true" data-project="96" data-commit="0ab7bbb6e2cf3f481a68ce48a8f6d6d609b3dced" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="restore: Use bitmap index from backup repo, if present" class="gfm gfm-commit has-tooltip">kirr/git-backup@0ab7bbb6</a>
/cc <a href="/alain.takoudjou" data-user="3" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Alain Takoudjou">@alain.takoudjou</a>, <a href="/rafael" data-user="11" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Rafael Monnerat">@rafael</a>, <a href="/jerome" data-user="9" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Jérome Perrin">@jerome</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/342" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/342" data-link="false" data-link-reference="true" data-project="15" data-merge-request="2139" data-project-path="nexedi/slapos" data-iid="342" data-mr-title="gitlab: v↑ git-backup" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!342</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/e8fb63239272df7b5ec868414a80c20db20f20badeploy-test: "Backport" commit "PyPI is changing"2018-06-08T14:07:31+02:00Lukasz Nowakluke@nexedi.com
Extended KVM SR does not have this hosts in allowed, and we need to stick
with it.
For reference in case if some existing Python distribution does not install,
it was detected with "git diff 8deeab.. -- stack/slapos.cfg"https://lab.node.vifib.com/nexedi/slapos/-/commit/f7303597775e0defe228422fed582956a109ff08ERP5: fix backup-periodicity=never (ZEO)2018-06-07T19:56:26+02:00Julien Muchembledjm@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/feb4d9294ff1f12616f3b6c28592753cc3f0b15eRevert "build-rina: also test on latest Debian release"2018-06-07T17:50:30+02:00Julien Muchembledjm@nexedi.com
The Debian packaging of our RINA kernel builds with GCC 4.9,
which is not available on Stretch:
# apt install ./linux-build-deps_4.1.33+rina65.g2e1cdbcb4-1_amd64.deb
...
The following packages have unmet dependencies:
linux-build-deps : Depends: gcc-4.9 but it is not installablehttps://lab.node.vifib.com/nexedi/slapos/-/commit/b8be395053829591febe5690ad5dc2ba3f03a4b7fixup! slaprunner: Support resilience on computers with more them 100 partitions2018-06-06T15:59:29+02:00Tristan Caveliertristan.cavelier@nexedi.com
The regexp was matching "slappart10" in `/path/to/slappart10/software.cfg`,
was also matching "slappart10" in `/path/to/aaslappart10aa/software.cfg`
and was not matching "slappart1" in `/path/to/slappart1/software.cfg`
or "slappart1000" in `/path/to/slappart1000/software.cfg`.
The fix allows to match "slappartX" in `/path/to/slappartX/software.cfg`
where X can be between 0 to infinite. Also, it protects from matching
"slappartX" in `/path/to/aaaslappartXaaa/software.cfg`.https://lab.node.vifib.com/nexedi/slapos/-/commit/e419ece6d7ecd0865af100edf6e12c10a116aafbbuild-rina: also test on latest Debian release2018-06-06T11:11:55+02:00Julien Muchembledjm@nexedi.com
Now that the test suite passes again for Jessie, let's try Stretch.https://lab.node.vifib.com/nexedi/slapos/-/commit/976ab53325742aafafabc5ca186253fc9eff4a0abuild-rina: make it work with erp5.util >= 0.4.472018-06-05T20:50:38+02:00Julien Muchembledjm@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/2eafd4b14656b1534d3fa7b0f1fedf45b5137293monitor: add missing python file in bin directory2018-06-05T15:52:27+02:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/220b5e1d29e7d79f2aba8f6960da112a5d8a8676gitlab sr: update git-backup revision2018-06-05T11:25:21+02:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/c7864eac52877ce6380ff97fd0c7225690587299gitlab sr: update gitlab-workhorse revision2018-06-05T09:59:38+02:00Alain Takoudjoualain.takoudjou@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/0bb9c19e1fcbb401e420e2e25598d0b4147db692Override previous NayuOS build image2018-06-04T11:11:20+02:00Lorenzo Martinicolorenzo.martinico@nexedi.com
If two NayuOS images are built on the same day, the second one will not get zipped. The -f flag in gzip can be used to overwrite the previous image
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/336" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/336" data-link="false" data-link-reference="true" data-project="15" data-merge-request="2118" data-project-path="nexedi/slapos" data-iid="336" data-mr-title="Override previous NayuOS build image" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!336</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/fdc9120aacdd8bcb9a10751383a96758f5541731ProFTPd: fixes to support running in a "system level" slapos2018-06-01T08:18:12+02:00Jérome Perrinjerome@nexedi.com
This SR was developed and tested in environments where both software and instance where owned by the same unix user. These are fixes for problems discovered when running on a "system level" slapos.
This cannot easily be tested without actually using different users (especially the mistake on the mode of the executable script), but this MR introduce some tests to check that service running in instance does not open files or unix sockets outside of instance. It also introduce a utility method to access the supervisor RPC API, it can be useful if tests needs to start/stop a process or get process PID to inspect this process.
I ran into problems, because proftpd sets [`PR_SET_DUMPABLE`](<a href="http://man7.org/linux/man-pages/man2/prctl.2.html" rel="nofollow noreferrer noopener" target="_blank">http://man7.org/linux/man-pages/man2/prctl.2.html</a>) to 0, which disables any possibility of inspecting a running program. See <a href="/tomo/slapos/-/commit/18e28bbb0757984a4d7c823e11c749b8a3fb3c46" data-original="18e28bbb0757984a4d7c823e11c749b8a3fb3c46" data-link="false" data-link-reference="false" data-project="870" data-commit="18e28bbb0757984a4d7c823e11c749b8a3fb3c46" data-reference-type="commit" data-container="body" data-placement="top" data-html="true" title="component/proftpd: allow dumping process" class="gfm gfm-commit has-tooltip">18e28bbb</a> for details about the chosen approach.
/cc <a href="/rafael" data-user="11" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Rafael Monnerat">@rafael</a> <a href="/Nicolas" data-user="34" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Nicolas Wavrant">@Nicolas</a> <a href="/luke" data-user="302" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Łukasz Nowak">@luke</a> @Eteri <a href="/vpelletier" data-user="23" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Vincent Pelletier">@vpelletier</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/334" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/334" data-link="false" data-link-reference="true" data-project="15" data-merge-request="2095" data-project-path="nexedi/slapos" data-iid="334" data-mr-title='ProFTPd: fixes to support running in a "system level" slapos' data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!334</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/46a771a69d9726868d0b19478a7f1bb62f3f3d9aerp5, mariadb: Allow users to set additional environment variables to mysqld...2018-05-25T19:04:07+09:00Yusei Taharayusei@nexedi.comerp5, mariadb: Allow users to set additional environment variables to mysqld to use third party ODBC libraries for CONNECT storage engine.
https://lab.node.vifib.com/nexedi/slapos/-/commit/8cf1b1c414cbce79ac30475319c2355aabe9e741fixup! NEO: the test suite needs mysqldump2018-05-24T16:08:49+02:00Julien Muchembledjm@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/c19f82866d8638de80d1201581d5187357405820NEO: the test suite needs mysqldump2018-05-24T15:48:38+02:00Julien Muchembledjm@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/a3f09d4d42f81c718e66d37b0f6278e481cb5c25erp5 buildout: remove onlyoffice2018-05-23T09:46:59+02:00Xiaowu Zhangxiaowu.zhang@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/2d255aa227c06df1f9b78aee30d8b473e45b9257erp5 buildout: remove libpng12 used by cloudooo2018-05-23T09:46:59+02:00Xiaowu Zhangxiaowu.zhang@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/350ab63e81d415a074df8e1358faaedace05da1cstack/erp5: remove cloudooo2018-05-23T09:46:59+02:00Tristan Caveliertristan.cavelier@nexedi.com
more discussion at <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/266" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/266" data-link="false" data-link-reference="true" data-project="15" data-merge-request="1726" data-project-path="nexedi/slapos" data-iid="266" data-mr-title="WIP: stack/erp5: completely remove cloudooo (part 2)" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">!266</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/61b8e9723e2cc7bef0093837e36e5d58c98ffb2fproftpd/test: test that files are created only in instance folder2018-05-23T09:29:51+02:00Jérome Perrinjerome@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/134c47de8d6e5382c2b42c514195a9230e22a34fSlapOSInstanceTestCase: add getSupervisorRPCServer2018-05-23T09:29:51+02:00Jérome Perrinjerome@nexedi.com
an utility method to control and query supervisor instance used by
slapos using <a href="http://supervisord.org/api.html" rel="nofollow noreferrer noopener" target="_blank">http://supervisord.org/api.html</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/18e28bbb0757984a4d7c823e11c749b8a3fb3c46component/proftpd: allow dumping process2018-05-23T09:29:51+02:00Jérome Perrinjerome@nexedi.com
Unless compiled with --enable-devel=coredump, proftpd's mod_rlimit will
use a prctl(PR_SET_DUMPABLE, 0) syscall[1] which prevents us from
inspecting files open by this process[2]
We don't want to enable debug build and I did not find a way to disable
only this PR_SET_DUMPABLE flag setting.
[1] <a href="https://github.com/proftpd/proftpd/blob/v1.3.6/modules/mod_rlimit.c#L580-L583" rel="nofollow noreferrer noopener" target="_blank">https://github.com/proftpd/proftpd/blob/v1.3.6/modules/mod_rlimit.c#L580-L583</a>
[2] <a href="http://man7.org/linux/man-pages/man5/proc.5.html" rel="nofollow noreferrer noopener" target="_blank">http://man7.org/linux/man-pages/man5/proc.5.html</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/afac6e04a5c1eccc96f033e4584b8b65396a05b2proftpd: create all files in instance2018-05-23T09:02:40+02:00Jérome Perrinjerome@nexedi.com
Some files and sockets where created in their default location, which
was in the software.
As a result, this software was working only on webrunner or testnodes,
which uses same unix user for software and instance.https://lab.node.vifib.com/nexedi/slapos/-/commit/ac4b1df56a1b4d823590bcbcc4869aedf6a78b15fixup! proftpd: new component2018-05-23T09:02:40+02:00Jérome Perrinjerome@nexedi.com
To be usable in a "system level" slapos with multiple users,
executables scripts must be 0755https://lab.node.vifib.com/nexedi/slapos/-/commit/eb2a4161e6542c44f7182af224ddfc3b65dd2fd6Fixes for ProFTPd test suite2018-05-23T02:44:09+02:00Jérome Perrinjerome@nexedi.com
!311 was merged a bit too fast. After test suite was repaired ( the test node was stuck updating the git repository ), [test](<a href="https://nexedi.erp5.net/test_result_module/20180507-19395E1F/2" rel="nofollow noreferrer noopener" target="_blank">https://nexedi.erp5.net/test_result_module/20180507-19395E1F/2</a>) was not passing on testnode, because path was too deep.
While debugging this, I realized this check was wrong, because supervisor [also append the pid](<a href="https://github.com/Supervisor/supervisor/blob/70720a0311d868fbf58a31cd26a7147921dcff29/supervisor/http.py#L564-L574" rel="nofollow noreferrer noopener" target="_blank">https://github.com/Supervisor/supervisor/blob/70720a0311d868fbf58a31cd26a7147921dcff29/supervisor/http.py#L564-L574</a>) to this path, so the path is even longer. I just checked that we have room for 7 digit pids.
Because path was too long on test node, test now run with `SLAPOS_TEST_WORKING_DIR` set to a slightly shorter path that's enough for this case.
There are some other minor fixes, see individual commits.
/cc <a href="/Nicolas" data-user="34" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Nicolas Wavrant">@Nicolas</a> <a href="/luke" data-user="302" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Łukasz Nowak">@luke</a> <a href="/rafael" data-user="11" data-reference-type="user" data-container="body" data-placement="top" data-html="true" class="gfm gfm-project_member" title="Rafael Monnerat">@rafael</a>
/reviewed-on <a href="https://lab.nexedi.com/nexedi/slapos/merge_requests/325" data-original="https://lab.nexedi.com/nexedi/slapos/merge_requests/325" data-link="false" data-link-reference="true" data-project="15" data-merge-request="2058" data-project-path="nexedi/slapos" data-iid="325" data-mr-title="Fixes for ProFTPd test suite" data-reference-type="merge_request" data-container="body" data-placement="top" data-html="true" title="" class="gfm gfm-merge_request">nexedi/slapos!325</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/778cc621b0306f66a9b724f310c05470d07c62c3testsuite/proftpd: set SLAPOS_TEST_WORKING_DIR2018-05-22T08:50:09+02:00Jérome Perrinjerome@nexedi.com
For supervisord AF_UNIX sockethttps://lab.node.vifib.com/nexedi/slapos/-/commit/11f65adcba842b685a42272eb6d3792ca35b3983testsuite/proftpd: instance must be named instance.cfg.in2018-05-22T08:50:09+02:00Jérome Perrinjerome@nexedi.com
This is a convention from
<a href="http://community.slapos.org/slapos_community/wiki/slapos-Wiki.Home/developer-SlapOS.Conventions.Guidelines" rel="nofollow noreferrer noopener" target="_blank">http://community.slapos.org/slapos_community/wiki/slapos-Wiki.Home/developer-SlapOS.Conventions.Guidelines</a>https://lab.node.vifib.com/nexedi/slapos/-/commit/225849850f4537b5bae4bb2afb8e3e6ae505b1fbfixup! fixup! proftpd: instance test suite2018-05-22T08:50:09+02:00Jérome Perrinjerome@nexedi.com
fix all getInstanceParameterDict typoshttps://lab.node.vifib.com/nexedi/slapos/-/commit/a4c16900620410dff4ea89e44bed54599de9af31proftpd/test/utils: fix check for AF_UNIX path too long2018-05-22T08:50:09+02:00Jérome Perrinjerome@nexedi.com
supervisord creates a socket with pid in the name, so we have to account
for the space of pid.
Because it's not so easy to know the max size of pids, we just assume 7
places.https://lab.node.vifib.com/nexedi/slapos/-/commit/bd53d6b69776c5c5301f9d43cb8bc64acf230787proftpd/test: make sure we teardown even if error in setup2018-05-22T08:50:09+02:00Jérome Perrinjerome@nexedi.com
otherwise we leak zombie slapproxy processeshttps://lab.node.vifib.com/nexedi/slapos/-/commit/47c586423f4a44c41cabd0ae0e021554f6ac242ccomponent/wget: version up 1.19.5, including security fixes.2018-05-20T18:08:30+02:00Kazuhiko Shiozakikazuhiko@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/18a59b273c247c3820410f3000ef0ecf883cd0e2component/mariadb: version up MariaDB 10.1.33, including security fixes.2018-05-20T18:00:37+02:00Kazuhiko Shiozakikazuhiko@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/5fed2698a9608d124e1ab29fb1c25d674eb46ba8component/pcre: version 8.42 for the recent MariaDB.2018-05-20T17:59:44+02:00Kazuhiko Shiozakikazuhiko@nexedi.comhttps://lab.node.vifib.com/nexedi/slapos/-/commit/939193258d06268e1988527a53e3137649656230component/curl: version 7.60, including security fixes.2018-05-20T17:59:11+02:00Kazuhiko Shiozakikazuhiko@nexedi.com