An error occurred fetching the project authors.
- 02 Aug, 2016 6 commits
-
-
Kirill Smelkov authored
We'll need to invoke psql connected to gitlab db in another place, so before doing it let's factor out the code to call psql as connected to a separate function.
-
Kirill Smelkov authored
Like for 0a72505e $ git diff 8.5.1+ce.0-1-ge732b39..8.6.5+ce.0-0-g342f8be -- files/gitlab-cookbooks/gitlab/templates/default/sv-sidekiq-run.erb is empty.
-
Kirill Smelkov authored
I manually reviewed $ git diff 8.5.1+ce.0-1-ge732b39..8.6.5+ce.0-0-g342f8be -- \ files/gitlab-config-template/gitlab.rb.template \ files/gitlab-cookbooks/gitlab/attributes/default.rb in omnibus-gitlab, and module proxy_cache and http2 changes, which we already handled in 2 previous patches, there is nothing more interesting for us.
-
Kirill Smelkov authored
Almost no changes this time: we only comment-out Nginx cache. See details for why we do not need it in comments and in f6f97d72.
-
Kirill Smelkov authored
Update GitLab software to - gitlab-ce 8.6.9 + NXD patches nexedi/gitlab-ce!1 - gitlab-shell to 2.6.12 + 1 patch to remove unneeded hooks.old in *.git nexedi/gitlab-shell!1 - gitlab-workhorse 0.7.1 + NXD patches. nexedi/gitlab-workhorse!1 ( download speedup patches were reworked because of upstream changes. Please see details in the above MR and in fixup commits ) This only updates software and begins SR update to 8.6 - for now gitlab instance becomes non-working -- we'll pull in configuration files updates and fixups in the following patches.
-
Kirill Smelkov authored
gitlab-workhorse works perfectly fine with it, so switch to current stable golang.
-
- 19 Jul, 2016 1 commit
-
-
Jérome Perrin authored
@jerome says at nexedi/slapos@5f5d5102 (comment 17119): before f4e51f77, we had: `~/srv/runner/instance/slappart0/bin/gitlab-rake` containing: ```python ... if __name__ == '__main__': sys.exit(slapos.recipe.librecipe.execute.generic_exec((['/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/bin/bundle', 'exec', 'sh', '-c', 'cd /srv/slapgrid/slappart16/srv/runner/instance/slappart0/gitlab-work && rake "$@"', 'rake'], None, {'BUNDLE_GEMFILE': '/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/parts/gitlab/Gemfile', 'HOME': '/srv/slapgrid/slappart16/srv/runner/instance/slappart0', 'SIDEKIQ_MEMORY_KILLER_MAX_RSS': '1000000', 'RAILS_ENV': 'production'}))) ``` after, `~/srv/runner/instance/slappart0/bin/gitlab-rake` contains: ```shell #!/bin/bash COMMAND=/srv/slapgrid/slappart16/srv/runner/instance/slappart0/bin/gitlab-rake.py # If the wrapped command uses a shebang, execute the referenced # executable passing the script path as first argument. # This is to workaround the limitation of 127 characters in #! if [[ -f $COMMAND && x$(head -c2 "$COMMAND") = x"#!" ]]; then SHEBANG=$(head -1 "$COMMAND") INTERPRETER=( ${SHEBANG#\#!} ) COMMAND="${INTERPRETER[@]} $COMMAND" fi exec $COMMAND ``` which is a wrapper around `gitlab-rake.py` containing: ```python ... if __name__ == '__main__': sys.exit(slapos.recipe.librecipe.execute.generic_exec((['/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/bin/bundle', 'exec', 'sh', '-c', 'cd /srv/slapgrid/slappart16/srv/runner/instance/slappart0/gitlab-work && rake "$@"', 'rake'], None, {'BUNDLE_GEMFILE': '/srv/slapgrid/slappart16/srv/runner/software/fffb3c99781923d3adb8bc53eb6c027a/parts/gitlab/Gemfile', 'HOME': '/srv/slapgrid/slappart16/srv/runner/instance/slappart0', 'SIDEKIQ_MEMORY_KILLER_MAX_RSS': '1000000', 'RAILS_ENV': 'production'}))) ``` `gitlab-rake.py` after is same as `gitlab-rake` before. This [slapos.cookbook:wrapper](https://lab.nexedi.com/nexedi/slapos/blob/cd9faac0/slapos/recipe/wrapper.py#L39) has an argument *parameters-extra* which if set to true, propagate command line arguments to the wrapped script. The default value for this parameter is false. Before f4e51f77, the generated wrapper was also propagating arguments even when *parameters-extra* was not set, but since this commit, this *parameters-extra* option is now handled as expected. This is the reason for this regression. In our case, when we see `/srv/slapgrid/slappart16/srv/runner/instance/slappart0/bin/gitlab-rake assets:clean`, it just calls `rake` without arguments. So a simple patch that fix the problem would be jerome/slapos@d3d05f02 . This way, the generated wrapper becomes: ```shell ... exec $COMMAND $@ ``` and arguments are correctly propagated. Feel free to cherry-pick that patch for now, but it may be nice to rethink this *parameters-extra* option, after this debugging session, I believe it should be true by default. /cc @seb for introducing the parameter in 80bb4305 and @vpelletier for touching this code in e7083872 /reviewed-by @kirr
-
- 22 Apr, 2016 2 commits
-
-
Kirill Smelkov authored
It was failing because gzip was not on PATH for gitlab-workhorse service. 2016/04/22 18:44:56 error: handleGetArchive: start [gzip -c -n]: exec: "gzip": executable file not found in $PATH The same for bzip2. Fix it. /reported-by @jm
-
Kirill Smelkov authored
it ends up being copied to the wrapper itself, like this: {'PATH': '/srv/slapgrid/slappart6/srv/runner/software/f3701a18118c774c95b8ff0519e5de46/parts/git/bin:/srv/slapgrid/slappart6/srv/runner/software/f3701a18118c774c95b8ff0519e5de46/parts/ruby2.1/bin:%(PATH)s', My mistake from 6ddc1c30 (gitlab: Setup gitlab-workhorse service)
-
- 14 Mar, 2016 1 commit
-
-
Kirill Smelkov authored
To pick up nexedi/gitlab-workhorse@32ec77c4 /reviewed-by TrustMe (I hope)
-
- 06 Mar, 2016 1 commit
-
-
Kirill Smelkov authored
To pick up nexedi/gitlab-ce@827d3914 /reviewed-by TrustMe (I hope)
-
- 03 Mar, 2016 1 commit
-
-
Kirill Smelkov authored
To pick up nexedi/gitlab-ce@1bfb586a /reviewed-by @kazuhiko
-
- 02 Mar, 2016 1 commit
-
-
Kirill Smelkov authored
kirr/ remains as my personal development place and nexedi/ ones become official repositories. Discussed with @kazuhiko /reviewed-by TrustMe
-
- 29 Feb, 2016 1 commit
-
-
Kirill Smelkov authored
To pick up kirr/gitlab-ce@8d922ec3 kirr/gitlab-ce@5d6b7eba See: kirr/gitlab-ce@5a6e6e55 for discussion. /cc @kazuhiko, @jerome /reviewed-by TrustMe
-
- 28 Feb, 2016 5 commits
-
-
Kirill Smelkov authored
$ git diff 8.4.4+ce.0-0-g1680742..8.5.1+ce.0-1-ge732b39 -- \ files/gitlab-cookbooks/gitlab/templates/default/sv-sidekiq-run.erb shows nothing.
-
Kirill Smelkov authored
I manually reviewed $ git diff 8.4.2+ce.0-3-g68d5ee8..8.5.1+ce.0-1-ge732b39 \ files/gitlab-config-template/gitlab.rb.template \ files/gitlab-cookbooks/gitlab/attributes/default.rb in omnibus-gitlab, and module proxy_set_header change, which we already addressed in previous patch in Nginx config, there are no more changes for us.
-
Kirill Smelkov authored
- relative URL support: comment out - we do not need it - gitlab is always located at /. - Nginx-http: restore our version for proxy_set_header - upstream turned to allowing users to configure this, see e.g. https://gitlab.com/gitlab-org/omnibus-gitlab/commit/e13d5e42 https://gitlab.com/gitlab-org/omnibus-gitlab/commit/a450585e but doing this way creates more complexity for gitlab SR, so I've restored our version which essentially does the same as default in omnibus-gitlab, and if we'll need to tune it - we can do directly in Nginx config. In other words slapos version does not allow users to tune nginx headers as instance parameter.
-
Kirill Smelkov authored
Update GitLab software to - gitlab-ce 8.5.1 + NXD patches https://lab.nexedi.com/kirr/gitlab-ce/commits/8-5-nxd - gitlab-shell to 2.6.10 + 1 patch to remove unneeded hooks.old in *.git https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/40 - gitlab-workhorse 0.6.4 + NXD patches. https://lab.nexedi.com/kirr/gitlab-workhorse/commits/y/blobraw-4 https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17 ( download speedup patches got improved, and now also properly proxy _gitlab_session cookie to auth backend, so raw files for private repositories now open in browser ok ) This only updates software and begins SR update to 8.5 - for now gitlab instance becomes non-working -- we'll pull in configuration files updates and fixups in the following patches. P.S. we also pin-up rubygems version, used to build gems, along the way.
-
Kirill Smelkov authored
GitLab uses git executable by full path as defined in gitlab.yml, but not all places in code use it, e.g. here git is used just from $PATH https://gitlab.com/gitlab-org/gitlab_git/blob/2f0d3c1a/lib/gitlab_git/repository.rb#L259 So make sure to include our git into bundler-4gitlab PATH.
-
- 17 Feb, 2016 2 commits
-
-
Kirill Smelkov authored
To pick up kirr/gitlab-ce@32fa1180 /cc @kazuhiko, @jerome /reviewed-by TrustMe
-
Kirill Smelkov authored
6a4e45c5 (gitlab: Sync sidekiq service launch with upstream) claimed to sync sidekiq startup with upstream, but missed one upstream change in between 8.2 and 8.4: https://gitlab.com/gitlab-org/omnibus-gitlab/commit/2eb7634f My bad - I should have reviewed git log for https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/sv-sidekiq-run.erb and I did not noticed that change because I did not and just compared queues list visually. Pick it up. /cc @kazuhiko, @jerome /reviewed-by TrustMe (I hope)
-
- 16 Feb, 2016 1 commit
-
-
Kirill Smelkov authored
To pick up kirr/gitlab-ce@a5965b54.
-
- 13 Feb, 2016 1 commit
-
-
Kirill Smelkov authored
As it was outlined in 5a744de7 (gitlab: Compile assets on instantiation and make sure DB is properly setup/migrated before unicorn runs) we are performing DB initialization and migration in pre-action as part of unicorn startup script. But that currently has one drawback: if all services start at the same time - e.g. both PostgreSQL and Unicorn - and that is a common scenario when SR is compiled and instantiated / started, PostgreSQL is usually not yet ready to process queries from Unicorn startup script, and first-time Unicorn startup fails. Until now this problem was workarounded by manually starting unicorn second time - after some time postgresql is started and ready. But why do it manually, if we can do the same logic automatically. So fix it: in Unicorn startup script wait a bit (up to 5 seconds) for PostgreSQL to become ready. /cc @kazuhiko, @jerome
-
- 11 Feb, 2016 9 commits
-
-
Kirill Smelkov authored
Even though we do not use LFS objects, builds and build artifacts, gitlab backups corresponding directories and recreates them upon restore. Current state is: the directories are living under Rails root - that is in our gitlab-work/ inside instance, and since gitlab-work is recreated almost on every re-instantiation if we'll have some content in that directories - it will be lost. Don't allow even a chance for that to happen and move what is designed to hold data to under var/ . NOTE we create shared/lfs-objects/ and shared/artifacts/ because if we do not have this dirs, gitlab backup fails. Upstream code just assumes the directories are there. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Upstream added `pages` and `elasticsearch` queues to sidekiq https://gitlab.com/gitlab-org/omnibus-gitlab/commit/be6844ff https://gitlab.com/gitlab-org/omnibus-gitlab/commit/801d6ed7 but since those queues are used for EE-only services we do not enable them. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
As gitlab-workhorse is now serving everything, it proxies unknown-to-it requests to unicorn, and that returns redirect to /users/sign_in for /non-existent, not 403. Adjust the promise and just verify whether we can get /static.css NOTE nginx promise now effectively depends on gitlab-workhorse being up and running - because nginx is now configured to be only an SSL terminator and all requests processing is done by gitlab-workhorse and services below it. If in the future we'll need a pure nginx-is-up promise, we can introduce something to nginx config and verify that particular http endpoint. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
add `-documentRoot ...` argument to gitlab-workhorse start as upstream started to do: https://gitlab.com/gitlab-org/omnibus-gitlab/commit/d769f751 ( gitlab-workhorse now needs to know where document root is as it now serves all content - e.g. static files too this option was introduced here: https://gitlab.com/gitlab-org/gitlab-workhorse/commit/1a7009e4 ) /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
I manually reviewed files/gitlab-config-template/gitlab.rb.template files/gitlab-cookbooks/gitlab/attributes/default.rb from omnibus-gitlab 8.4.2+ce.0-3-g68d5ee8 and picked up changes: - unicorn OOM killer memory limits were raised up (because gitlab is now more memory hungry according to talks on gitlab.com) https://gitlab.com/gitlab-org/omnibus-gitlab/commit/f8ee4a19 - there is no default limit for nginx client_max_body_size - it just goes to gitlab-workhorse as is at any size https://gitlab.com/gitlab-org/omnibus-gitlab/commit/78862837 /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Direct merge for gitconfig.erb from omnibus-gitlab did not brought up updates. pack.threads=1 and receive.fsckObjects=true picked up from https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb in 264d785a stay the same. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
- there is a section for gitlab pages, which we stub-out; - there is no longer a need to add /raw/... handling to nginx - as now nginx is just an SSL terminator for gitlab-workhorse, all URL handling is done inside gitlab-workhorse and is dealt with ok by our patches. - as now nginx does not directly connect to unicorn, there is no need to pass unicorn section to nginx's template. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Slapos'ify config updates brought by sync with omnibus-gitlab (see merge in the previous patch). Changes: - default visibility levels gone away (see merge commit and https://gitlab.com/gitlab-org/omnibus-gitlab/commit/b5ebbab3 ) - there are new settings for GitLab Pages and Elasticsearch, which are EE only and thus we do not support them. - there are new settings for auxiliary cron-like jobs, which we do not support for now, since they are used either for CI (not supported by us) or EE features. Configuration files that were synced, but did not changed are also marked as updated, so it is easier to track their changes to upstream in the future. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Update GitLab software to - gitlab-ce 8.4 + NXD patches https://lab.nexedi.com/kirr/gitlab-ce/commits/8-4-nxd - gitlab-shell to 2.6.10 which now is pure upstream, as all NXD patches were merged. - gitlab-workhorse 0.6.X + NXD patches. https://lab.nexedi.com/kirr/gitlab-workhorse/commits/y/blobraw-3 https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17 This only updates software and begins SR update to 8.4 - for now gitlab instance becomes non-working -- we'll pull in configuration files updates and fixups in the following patches. /cc @kazuhiko, @jerome
-
- 31 Jan, 2016 1 commit
-
-
Kirill Smelkov authored
Gitlab uses github-markup to render various text-based markups (markdown, rst, ...) into html. For rst github-markup wants to run python and have docutils egg available: https://github.com/github/markup/blob/5393ae93/lib/github/markups.rb#L36 as we were not having docutils installed and path to proper python interpreter setup, rst documents were not automatically rendered and were show just as plain text. We do a lot of documents in rst - that case is important for us. So fix it by providing gitlab with properly setup python interpreter with all needed eggs installed. /cc @kazuhiko, @jerome /proposed-for-review-on nexedi/slapos!39
-
- 17 Jan, 2016 7 commits
-
-
Kirill Smelkov authored
We've reached a state where first gitlab SR version should work. So as promised let's freeze the md5 checksums. All later patches should update corresponding md5 info when they change a file. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
In slapos we do a lot of automated software rebuild constantly, and thus there is constant flow of requests to get raw blobs from git service, e.g. like this https://lab.nexedi.com/nexedi/slapos/raw/master/software/wendelin/software.cfg A lot of requests comes to slapos.git repository and currently gitlab, out of the box, cannot keep up with that load. I've prepared patches to offload raw blobs download requests handling from unicorn (ruby) to gitlab-workhorse (go), and that resulted in ~ 17x speedup - e.g. previously our std shuttle can handle ~ 70 raw-blob requests/s and with my changes it is now ~ 1200 requests/s. The patches were sent upstream https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17 and we discussed with GitLab people and made a plan how to proceed incrementally. It will probably take some time for gitlab team to fully accept the approach though. For now we can use our gitlab-workhorse fork. The patches itself are: kirr/gitlab-workhorse@1b274d0d kirr/gitlab-workhorse@2beb8c95 /cc @kazuhiko, @jerome, @jm
-
Kirill Smelkov authored
GitLab Nexedi Edition is currently upstream 8.2.X + the following patches: - HTTP(S) is made to be default clone protocol kirr/gitlab-ce@5c1f2fb3 and SSH info is completely removed from UI kirr/gitlab-ce@dfe9fb16 kirr/gitlab-ce@f3f84743 so essentially the only way to access a repository is via HTTP(S). - Rake check tasks are adjusted to exit with non-zero code if there is a failure kirr/gitlab-ce@a93ae418 We need this for promises to work correctly with failures being detected, not silently skipped. The patch was sent upstream: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1885 - GitLab supports setting up site's ICP License in gitlab.yml and shows it in appropriate places together with info about GitLab itself: kirr/gitlab-ce@e7e0fd88 kirr/gitlab-ce@79c127e6 + other cosmetic/minor changes. More patches will probably come (e.g. apply a single patch from a merge-request with `git am` without creating merge commit for just 1 patch, etc) but for now that's all. NOTE ICP is non-ascii text with hieroglyphs. slapos.core was taught to be able to pass parameters with non-ascii values to instance: nexedi/slapos.core@347d33d6 That patch is included in slapos.core 1.3.15, but as we currently have a lot of older slapos.core deployed (e.g. 1.3.5 on my development webrunner) a workaround is (hopefully temporarily) used to pass non-ascii values as URL-encoded strings. /cc @kazuhiko, @jerome, @rafael
-
Kirill Smelkov authored
Sidekiq[1] is used in GitLab as background jobs manager - i.e. if a request handler needs to spawn some non-light job - it adds it to sidekiq queue (in Redis) and relies on sidekiq service to later pick this job up and execute it. The service is setup with just to run bin/gitlab-sidekiq with appropriate queues (extracted from omnibus-gitlab) and appropriate settings to controlling GitLab's sidekiq Out-Of-Memory killer[2]. NOTE Unlike unicorn OOM killer, Sidekiq memory killer just makes sidekiq processes to be SIGKILL terminated and relies on managing service to restart it. In slapos we don't have mechanism to set autorestart=true, nor bang/watchdog currently work with slapproxy, so we setup to do such monitoring ourselves manually with here-introduced watcher-sigkill program. NOTE2 sidekiq promise, because it is rake/gitlab based, is slow to load/run and thus is put into etc/promise.slow/ [1] http://sidekiq.org/ [2] https://gitlab.com/gitlab-org/gitlab-ce/blob/1322bd78/doc/operations/sidekiq_memory_killer.md /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Go through nginx configuration templates and convert them to jinja2 with slapos parameters (reminder: names and default values are imported from omnibus-gitlab 8.2.3+ce.0-0-g8eda093), except commenting out features we do not want to support (yet ?). As nginx is a reverse-proxy, i.e. it integrates all internal services and works as frontend to them, our gitlab service is now ready to listen and talk to the world over (standard to slapos services backend) IPv6. Nginx also acts as SSL termination point - for it to work by default we setup self-signed certificate for the backend, which can be manually changed to proper certificate if needed. Backend certificate is used if gitlab is configured to work in HTTPS mode (and frontend certificate is another story). NOTE ssl certificate is generated with just `openssl req ...` - yes, there is slapos.cookbook:certificate_authority.request but it requires to start whole service and has up to 60 seconds latency to generate certificate. And we only need to run 1 command to do that... The features disabled are: - http -> https redirection not needed for us at nginx level - the frontend can do the redirection and also gitlab speaks HSTS on https port so when we access https port via http protocol, it gets redirected to https. - kerberos - ssl_dhparam - providing custom nginx configuration via instance parameter /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
As was described in the previous patch, we need $HOME to be propagated by this programs so that git can find partition's .gitconfig. Specifically we need the following patches to be present in our build: https://gitlab.com/gitlab-org/gitlab-shell/commit/9e087f64 https://gitlab.com/gitlab-org/gitlab-workhorse/commit/b5f1b803 They both have been applied upstream very close to revisions we previously had in software.cfg, so we only need to update the revisions to get them. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Like with Rails configuration files, hook nginx configuration files into SR / instance build process; rename *.erb -> *.in and add our header. The templates are still not valid - a lot of erb code is left there - we'll slapos'ify it incrementally in the following patches. /cc @kazuhiko, @jerome
-