Merge branch 'systemd' into 'master'

Add native systemd units

See merge request gitlab-org/gitlab!53847

Gemfile

@@ -186,6 +186,7 @@ gem 'rack-timeout', '~> 0.5.1', require: 'rack/timeout/base'
 group :puma do
   gem 'puma', '~> 5.3.1', require: false
   gem 'puma_worker_killer', '~> 0.3.1', require: false
+  gem 'sd_notify', '~> 0.1.0', require: false
 end
 
 # State machine

Gemfile.lock

@@ -1153,6 +1153,7 @@ GEM
       addressable (>= 2.3.5)
       faraday (> 0.8, < 2.0)
     scientist (1.6.0)
+    sd_notify (0.1.0)
     securecompare (1.0.0)
     seed-fu (2.3.7)
       activerecord (>= 3.1)
@@ -1605,6 +1606,7 @@ DEPENDENCIES
   rugged (~> 1.1)
   sanitize (~> 5.2.1)
   sassc-rails (~> 2.1.0)
+  sd_notify (~> 0.1.0)
   seed-fu (~> 2.3.7)
   selenium-webdriver (~> 3.142)
   sentry-raven (~> 3.1)

doc/administration/incoming_email.md

@@ -155,6 +155,50 @@ Reply by email should now be working.
 1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature
   and fill in the details for your specific IMAP server and email account (see [examples](#configuration-examples) below).
 
+If you use systemd units to manage GitLab:
+
+1. Add `gitlab-mailroom.service` as a dependency to `gitlab.target`:
+
+   ```shell
+   sudo systemctl edit gitlab.target
+   ```
+
+   In the editor that opens, add the following and save the file:
+
+   ```plaintext
+   [Unit]
+   Wants=gitlab-mailroom.service
+   ```
+
+1. If you run Redis and PostgreSQL on the same machine, you should add a
+   dependency on Redis. Run:
+
+   ```shell
+   sudo systemctl edit gitlab-mailroom.service
+   ```
+
+   In the editor that opens, add the following and save the file:
+
+   ```plaintext
+   [Unit]
+   Wants=redis-server.service
+   After=redis-server.service
+   ```
+
+1. Start `gitlab-mailroom.service`:
+
+   ```shell
+   sudo systemctl start gitlab-mailroom.service
+   ```
+
+1. Verify that everything is configured correctly:
+
+   ```shell
+   sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production
+   ```
+
+If you use the SysV init script to manage GitLab:
+
 1. Enable `mail_room` in the init script at `/etc/default/gitlab`:
 
    ```shell

doc/administration/pages/source.md

@@ -132,11 +132,11 @@ The Pages daemon doesn't listen to the outside world.
      https: false
      artifacts_server: false
      external_http: ["127.0.0.1:8090"]
-     secret_file: /home/git/gitlab/gitlab-pages/gitlab-pages-secret
+     secret_file: /home/git/gitlab/gitlab-pages-secret
    ```
 
 1. Add the following configuration file to
-   `/home/git/gitlab/gitlab-pages/gitlab-pages.conf`, and be sure to change
+   `/home/git/gitlab-pages/gitlab-pages.conf`, and be sure to change
    `example.io` to the FQDN from which you want to serve GitLab Pages and
    `gitlab.example.com` to the URL of your GitLab instance:
 
@@ -159,12 +159,27 @@ The Pages daemon doesn't listen to the outside world.
    sudo -u git -H openssl rand -base64 32 > /home/git/gitlab/gitlab-pages-secret
    ```
 
-1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in
-   order to enable the pages daemon:
+1. To enable the pages daemon:
 
-   ```ini
-   gitlab_pages_enabled=true
-   ```
+   - If your system uses systemd as init, run:
+
+     ```shell
+     sudo systemctl edit gitlab.target
+     ```
+
+     In the editor that opens, add the following and save the file:
+
+     ```plaintext
+     [Unit]
+     Wants=gitlab-pages.service
+     ```
+
+   - If your system uses SysV init instead, edit `/etc/default/gitlab` and set
+     `gitlab_pages_enabled` to `true`:
+
+     ```ini
+     gitlab_pages_enabled=true
+     ```
 
 1. Copy the `gitlab-pages` NGINX configuration file:
 

doc/administration/restart_gitlab.md

@@ -103,39 +103,15 @@ If you have followed the official installation guide to [install GitLab from
 source](../install/installation.md), run the following command to restart GitLab:
 
 ```shell
-sudo service gitlab restart
-```
+# For systems running systemd
+sudo systemctl restart gitlab.target
 
-The output should be similar to this:
-
-```plaintext
-Shutting down GitLab Puma
-Shutting down GitLab Sidekiq
-Shutting down GitLab Workhorse
-Shutting down GitLab MailRoom
-...
-GitLab is not running.
-Starting GitLab Puma
-Starting GitLab Sidekiq
-Starting GitLab Workhorse
-Starting GitLab MailRoom
-...
-The GitLab Puma web server with pid 28059 is running.
-The GitLab Sidekiq job dispatcher with pid 28176 is running.
-The GitLab Workhorse with pid 28122 is running.
-The GitLab MailRoom email processor with pid 28114 is running.
-GitLab and all its components are up and running.
+# For systems running SysV init
+sudo service gitlab restart
 ```
 
 This should restart Puma, Sidekiq, GitLab Workhorse, and [Mailroom](reply_by_email.md)
-(if enabled). The init service file that does all the magic can be found on
-your server in `/etc/init.d/gitlab`.
-
----
-
-If you are using other init systems, like `systemd`, you can check the
-[GitLab Recipes](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/init) repository for some unofficial services. These are
-**not** officially supported so use them at your own risk.
+(if enabled).
 
 ## Helm chart installations
 

doc/install/installation.md

@@ -423,6 +423,49 @@ echo 'unixsocket /var/run/redis/redis.sock' | sudo tee -a /etc/redis/redis.conf
 # Grant permission to the socket to all members of the redis group
 echo 'unixsocketperm 770' | sudo tee -a /etc/redis/redis.conf
 
+# Add git to the redis group
+sudo usermod -aG redis git
+```
+
+### Supervise Redis with systemd
+
+If your distribution uses systemd init and the output of the following command is `notify`,
+you do not need to make any changes:
+
+```shell
+systemctl show --value --property=Type redis-server.service
+```
+
+If the output is **not** `notify`, run:
+
+```shell
+# Configure Redis to not daemonize, but be supervised by systemd instead and disable the pidfile
+sudo sed -i \
+         -e 's/^daemonize yes$/daemonize no/' \
+         -e 's/^supervised no$/supervised systemd/' \
+         -e 's/^pidfile/# pidfile/' /etc/redis/redis.conf
+sudo chown redis:redis /etc/redis/redis.conf
+
+# Make the same changes to the systemd unit file
+sudo mkdir -p /etc/systemd/system/redis-server.service.d
+sudo tee /etc/systemd/system/redis-server.service.d/10fix_type.conf <<EOF
+[Service]
+Type=notify
+PIDFile=
+EOF
+
+# Reload the redis service
+sudo systemctl daemon-reload
+
+# Activate the changes to redis.conf
+sudo systemctl restart redis-server.service
+```
+
+### Leave Redis unsupervised
+
+If your system uses SysV init, run these commands:
+
+```shell
 # Create the directory which contains the socket
 sudo mkdir -p /var/run/redis
 sudo chown redis:redis /var/run/redis
@@ -435,9 +478,6 @@ fi
 
 # Activate the changes to redis.conf
 sudo service redis-server restart
-
-# Add git to the redis group
-sudo usermod -aG redis git
 ```
 
 ## 8. GitLab
@@ -687,48 +727,79 @@ sudo -u git -H editor config.toml
 For more information about configuring Gitaly see
 [the Gitaly documentation](../administration/gitaly/index.md).
 
-### Start Gitaly
+### Install the service
 
-Gitaly must be running for the next section.
+GitLab has always supported SysV init scripts, which are widely supported and portable, but now systemd is the standard for service supervision and is used by all major Linux distributions. You should use native systemd services if you can to benefit from automatic restarts, better sandboxing and resource control.
 
-```shell
-gitlab_path=/home/git/gitlab
-gitaly_path=/home/git/gitaly
+#### Install systemd units
 
-sudo -u git -H sh -c "$gitlab_path/bin/daemon_with_pidfile $gitlab_path/tmp/pids/gitaly.pid \
-  $gitaly_path/_build/bin/gitaly $gitaly_path/config.toml >> $gitlab_path/log/gitaly.log 2>&1 &"
-```
+Use these steps if you use systemd as init. Otherwise, follow the [SysV init script steps](#install-sysv-init-script).
 
-### Initialize Database and Activate Advanced Features
+Copy the services and run `systemctl daemon-reload` so that systemd picks them up:
 
 ```shell
 cd /home/git/gitlab
-sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production
-# Type 'yes' to create the database tables.
+sudo mkdir -p /usr/local/lib/systemd/system
+sudo cp lib/support/systemd/* /usr/local/lib/systemd/system/
+sudo systemctl daemon-reload
+```
 
-# or you can skip the question by adding force=yes
-sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production force=yes
+The units provided by GitLab make very little assumptions about where you are running Redis and PostgreSQL.
 
-# When done, you see 'Administrator account created:'
-```
+If you installed GitLab in another directory or as a user other than the default, you need to change these values in the units as well.
 
-You can set the Administrator/root password and email by supplying them in environmental variables, `GITLAB_ROOT_PASSWORD` and `GITLAB_ROOT_EMAIL` respectively, as seen below. If you don't set the password (and it is set to the default one), wait to expose GitLab to the public internet until the installation is done and you've logged into the server the first time. During the first login, you'll be forced to change the default password. An Enterprise Edition license may also be installed at this time by supplying a full path in the `GITLAB_LICENSE_FILE` environment variable.
+For example, if you're running Redis and PostgreSQL on the same machine as GitLab, you should:
+
+- Edit the Puma service:
+
+  ```shell
+  sudo systemctl edit gitlab-puma.service
+  ```
+
+  In the editor that opens, add the following and save the file:
+
+  ```plaintext
+  [Unit]
+  Wants=redis-server.service postgresql.service
+  After=redis-server.service postgresql.service
+  ```
+
+- Edit the Sidekiq service:
+
+  ```shell
+  sudo systemctl edit gitlab-sidekiq.service
+  ```
+
+  Add the following and save the file:
+
+  ```plaintext
+  [Unit]
+  Wants=redis-server.service postgresql.service
+  After=redis-server.service postgresql.service
+  ```
+
+`systemctl edit` installs drop-in configuration files at `/etc/systemd/system/<name of the unit>.d/override.conf`, so your local configuration is not overwritten when updating the unit files later. To split up your drop-in configuration files, you can add the above snippets to `.conf` files under `/etc/systemd/system/<name of the unit>.d/`.
+
+If you manually made changes to the unit files or added drop-in configuration files (without using `systemctl edit`), run the following command for them to take effect:
 
 ```shell
-sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD=yourpassword GITLAB_ROOT_EMAIL=youremail GITLAB_LICENSE_FILE="/path/to/license"
+sudo systemctl daemon-reload
 ```
 
-### Secure secrets.yml
+Make GitLab start on boot:
 
-The `secrets.yml` file stores encryption keys for sessions and secure variables.
-Backup `secrets.yml` someplace safe, but don't store it in the same place as your database backups.
-Otherwise, your secrets are exposed if one of your backups is compromised.
+```shell
+sudo systemctl enable gitlab.target
+```
+
+#### Install SysV init script
 
-### Install Init Script
+Use these steps if you use the SysV init script. If you use systemd, follow the [systemd unit steps](#install-systemd-units).
 
 Download the init script (is `/etc/init.d/gitlab`):
 
 ```shell
+cd /home/git/gitlab
 sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
 ```
 
@@ -744,6 +815,9 @@ Make GitLab start on boot:
 
 ```shell
 sudo update-rc.d gitlab defaults 21
+# or if running this on a machine running systemd
+sudo systemctl daemon-reload
+sudo systemctl enable gitlab.service
 ```
 
 ### Set up Logrotate
@@ -752,6 +826,51 @@ sudo update-rc.d gitlab defaults 21
 sudo cp lib/support/logrotate/gitlab /etc/logrotate.d/gitlab
 ```
 
+### Start Gitaly
+
+Gitaly must be running for the next section.
+
+- To start Gitaly using systemd:
+
+  ```shell
+  sudo systemctl start gitlab-gitaly.service
+  ```
+
+- To manually start Gitaly for SysV:
+
+  ```shell
+  gitlab_path=/home/git/gitlab
+  gitaly_path=/home/git/gitaly
+
+  sudo -u git -H sh -c "$gitlab_path/bin/daemon_with_pidfile $gitlab_path/tmp/pids/gitaly.pid \
+    $gitaly_path/_build/bin/gitaly $gitaly_path/config.toml >> $gitlab_path/log/gitaly.log 2>&1 &"
+  ```
+
+### Initialize Database and Activate Advanced Features
+
+```shell
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production
+# Type 'yes' to create the database tables.
+
+# or you can skip the question by adding force=yes
+sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production force=yes
+
+# When done, you see 'Administrator account created:'
+```
+
+You can set the Administrator/root password and email by supplying them in environmental variables, `GITLAB_ROOT_PASSWORD` and `GITLAB_ROOT_EMAIL` respectively, as seen below. If you don't set the password (and it is set to the default one), wait to expose GitLab to the public internet until the installation is done and you've logged into the server the first time. During the first login, you'll be forced to change the default password. An Enterprise Edition license may also be installed at this time by supplying a full path in the `GITLAB_LICENSE_FILE` environment variable.
+
+```shell
+sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD=yourpassword GITLAB_ROOT_EMAIL=youremail GITLAB_LICENSE_FILE="/path/to/license"
+```
+
+### Secure secrets.yml
+
+The `secrets.yml` file stores encryption keys for sessions and secure variables.
+Backup `secrets.yml` someplace safe, but don't store it in the same place as your database backups.
+Otherwise, your secrets are exposed if one of your backups is compromised.
+
 ### Check Application Status
 
 Check if GitLab and its environment are configured correctly:
@@ -782,9 +901,11 @@ sudo -u git -H bundle exec rake gitlab:assets:compile RAILS_ENV=production NODE_
 ### Start Your GitLab Instance
 
 ```shell
+# For systems running systemd
+sudo systemctl start gitlab.target
+
+# For systems running SysV init
 sudo service gitlab start
-# or
-sudo /etc/init.d/gitlab restart
 ```
 
 ## 9. NGINX
@@ -857,6 +978,10 @@ nginx: configuration file /etc/nginx/nginx.conf test failed
 ### Restart
 
 ```shell
+# For systems running systemd
+sudo systemctl restart nginx.service
+
+# For systems running SysV init
 sudo service nginx restart
 ```
 
@@ -889,7 +1014,10 @@ earlier and login. After login, you can change the username if you wish.
 
 **Enjoy!**
 
-You can use `sudo service gitlab start` and `sudo service gitlab stop` to start and stop GitLab.
+To start and stop GitLab when using:
+
+- systemd units: use `sudo systemctl start gitlab.target` or `sudo systemctl stop gitlab.target`.
+- The SysV init script: use `sudo service gitlab start` or `sudo service gitlab stop`.
 
 ## Advanced Setup Tips
 

doc/install/relative_url.md

@@ -101,8 +101,9 @@ Make sure to follow all steps below:
    gitlab_url: http://127.0.0.1/gitlab
    ```
 
-1. Make sure you have copied the supplied init script and the defaults file
-   as stated in the [installation guide](installation.md#install-init-script).
+1. Make sure you have copied either the supplied systemd services, or the init
+   script and the defaults file, as stated in the
+   [installation guide](installation.md#install-the-service).
    Then, edit `/etc/default/gitlab` and set in `gitlab_workhorse_options` the
    `-authBackend` setting to read like:
 

doc/update/index.md

@@ -306,8 +306,8 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap
 ### 14.5.0
 
 When `make` is run, Gitaly builds are now created in `_build/bin` and no longer in the root directory of the source directory. If you
-are using a source install, update paths to these binaries in your init scripts by
-[following the documentation](upgrading_from_source.md#init-script).
+are using a source install, update paths to these binaries in your [systemd unit files](upgrading_from_source.md#configure-systemd-units)
+or [init scripts](upgrading_from_source.md#configure-sysv-init-script) by [following the documentation](upgrading_from_source.md).
 
 ### 14.4.0
 

doc/update/patch_versions.md

@@ -20,6 +20,10 @@ It's useful to make a backup just in case things go south. Depending on the inst
 ### 1. Stop server
 
 ```shell
+# For systems running systemd
+sudo systemctl stop gitlab.target
+
+# For systems running SysV init
 sudo service gitlab stop
 ```
 
@@ -108,6 +112,11 @@ Please follow the [install instruction](../integration/elasticsearch.md#install-
 ### 9. Start application
 
 ```shell
+# For systems running systemd
+sudo systemctl start gitlab.target
+sudo systemctl restart nginx.service
+
+# For systems running SysV init
 sudo service gitlab start
 sudo service nginx restart
 ```

doc/update/upgrading_from_source.md

@@ -54,6 +54,10 @@ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
 ### 2. Stop server
 
 ```shell
+# For systems running systemd
+sudo systemctl stop gitlab.target
+
+# For systems running SysV init
 sudo service gitlab stop
 ```
 
@@ -230,7 +234,29 @@ ActionMailer::Base.delivery_method = :smtp
 
 See [`smtp_settings.rb.sample`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/initializers/smtp_settings.rb.sample#L13) as an example.
 
-#### Init script
+#### Configure systemd units
+
+If using the SysV init script, see [Configure SysV init script](#configure-sysv-init-script).
+
+Check if the systemd units have been updated:
+
+```shell
+cd /home/git/gitlab
+
+git diff origin/PREVIOUS_BRANCH:lib/support/systemd origin/BRANCH:lib/support/systemd
+```
+
+Copy them over:
+
+```shell
+sudo mkdir -p /usr/local/lib/systemd/system
+sudo cp lib/support/systemd/* /usr/local/lib/systemd/system/
+sudo systemctl daemon-reload
+```
+
+#### Configure SysV init script
+
+If using systemd units, see [Configure systemd units](#configure-systemd-units).
 
 There might be new configuration options available for
 [`gitlab.default.example`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/support/init.d/gitlab.default.example).
@@ -250,7 +276,7 @@ cd /home/git/gitlab
 sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
 ```
 
-For Ubuntu 16.04.1 LTS:
+If you are using the init script on a system running systemd as init, because you have not switched to native systemd units yet, run:
 
 ```shell
 sudo systemctl daemon-reload
@@ -342,6 +368,11 @@ sudo -u git -H make
 ### 15. Start application
 
 ```shell
+# For systems running systemd
+sudo systemctl start gitlab.target
+sudo systemctl restart nginx.service
+
+# For systems running SysV init
 sudo service gitlab start
 sudo service nginx restart
 ```

lib/support/systemd/gitlab-gitaly.service

+[Unit]
+Description=GitLab Gitaly
+ReloadPropagatedFrom=gitlab.target
+PartOf=gitlab.target
+After=network.target
+
+[Service]
+Type=simple
+User=git
+WorkingDirectory=/home/git/gitlab
+ExecStart=/home/git/gitaly/_build/bin/gitaly /home/git/gitaly/config.toml
+Restart=on-failure
+SyslogIdentifier=gitlab-gitaly
+Slice=gitlab.slice
+
+[Install]
+WantedBy=gitlab.target

lib/support/systemd/gitlab-mailroom.service

+[Unit]
+Description=GitLab Mailroom
+PartOf=gitlab.target
+After=network.target
+StartLimitIntervalSec=100s
+
+[Service]
+Type=simple
+User=git
+WorkingDirectory=/home/git/gitlab
+Environment=RAILS_ENV=production
+ExecStart=/usr/local/bin/bundle exec mail_room --log-exit-as json --quiet --config /home/git/gitlab/config/mail_room.yml
+Restart=on-failure
+RestartSec=1
+SyslogIdentifier=gitlab-mailroom
+Slice=gitlab.slice
+
+[Install]
+WantedBy=gitlab.target

lib/support/systemd/gitlab-pages.service

+[Unit]
+Description=GitLab Pages
+ReloadPropagatedFrom=gitlab.target
+PartOf=gitlab.target
+After=network.target gitlab-puma.service
+Wants=gitlab-puma.service
+
+[Service]
+Type=simple
+User=git
+WorkingDirectory=/home/git/gitlab
+ExecStart=/home/git/gitlab-pages/gitlab-pages -config /home/git/gitlab-pages/gitlab-pages.conf
+Restart=on-failure
+RestartSec=1
+SyslogIdentifier=gitlab-pages
+Slice=gitlab.slice
+
+[Install]
+WantedBy=gitlab.target

lib/support/systemd/gitlab-puma.service

+[Unit]
+Description=GitLab
+Conflicts=gitlab.service
+ReloadPropagatedFrom=gitlab.target
+PartOf=gitlab.target
+After=network.target
+StartLimitIntervalSec=100s
+
+[Service]
+Type=notify
+User=git
+WorkingDirectory=/home/git/gitlab
+Environment=RAILS_ENV=production
+ExecStart=/usr/local/bin/bundle exec puma --config /home/git/gitlab/config/puma.rb --environment production --pidfile /home/git/gitlab/tmp/pids/puma.pid
+ExecReload=/usr/bin/kill -USR2 $MAINPID
+PIDFile=/home/git/gitlab/tmp/pids/puma.pid
+# puma can be slow to start
+TimeoutStartSec=120
+WatchdogSec=10
+Restart=on-failure
+RestartSec=1
+SyslogIdentifier=gitlab-puma
+Slice=gitlab.slice
+
+[Install]
+WantedBy=gitlab.target

lib/support/systemd/gitlab-sidekiq.service

+[Unit]
+Description=GitLab Sidekiq
+ReloadPropagatedFrom=gitlab.target
+PartOf=gitlab.target
+After=network.target
+JoinsNamespaceOf=gitlab-puma.service
+
+[Service]
+Type=simple
+User=git
+WorkingDirectory=/home/git/gitlab
+Environment=RAILS_ENV=production
+ExecStart=/usr/local/bin/bundle exec sidekiq --config /home/git/gitlab/config/sidekiq_queues.yml --environment production
+ExecStop=/usr/local/bin/bundle exec sidekiqctl stop /run/gitlab/sidekiq.pid
+PIDFile=/home/git/gitlab/tmp/pids/sidekiq.pid
+Restart=on-failure
+RestartSec=1
+SyslogIdentifier=gitlab-sidekiq
+Slice=gitlab.slice
+
+[Install]
+WantedBy=gitlab.target

lib/support/systemd/gitlab-workhorse.service

+[Unit]
+Description=GitLab Workhorse
+ReloadPropagatedFrom=gitlab.target
+PartOf=gitlab.target
+After=network.target gitlab-puma.service
+Wants=gitlab-puma.service
+
+[Service]
+Type=simple
+User=git
+WorkingDirectory=/home/git/gitlab
+Environment=PATH=/home/git/gitlab-workhorse:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ExecStart=/home/git/gitlab-workhorse/gitlab-workhorse  -listenUmask 0 -listenNetwork unix -listenAddr /home/git/gitlab/tmp/sockets/gitlab-workhorse.socket -authBackend http://127.0.0.1:8080 -authSocket /home/git/gitlab/tmp/sockets/gitlab.socket -documentRoot /home/git/gitlab/public -secretPath /home/git/gitlab/.gitlab_workhorse_secret
+ExecReload=/usr/bin/kill -USR2 $MAINPID
+Restart=on-failure
+RestartSec=1
+SyslogIdentifier=gitlab-workhorse
+Slice=gitlab.slice
+
+[Install]
+WantedBy=gitlab.target

lib/support/systemd/gitlab.slice

+[Unit]
+Description=Slice to bundle all GitLab services
+Before=slices.target
+
+[Slice]
+MemoryAccounting=true
+IOAccounting=true
+CPUAccounting=true

lib/support/systemd/gitlab.target

+[Unit]
+Description=GitLab
+Wants=gitlab-gitaly.service gitlab-puma.service gitlab-sidekiq.service gitlab-workhorse.service
+
+[Install]
+WantedBy=multi-user.target

lib/system_check/app/init_script_exists_check.rb

-# frozen_string_literal: true
-
-module SystemCheck
-  module App
-    class InitScriptExistsCheck < SystemCheck::BaseCheck
-      set_name 'Init script exists?'
-      set_skip_reason 'skipped (omnibus-gitlab has no init script)'
-
-      def skip?
-        omnibus_gitlab?
-      end
-
-      def check?
-        script_path = '/etc/init.d/gitlab'
-        File.exist?(script_path)
-      end
-
-      def show_error
-        try_fixing_it(
-          'Install the init script'
-        )
-        for_more_information(
-          see_installation_guide_section('Install Init Script')
-        )
-        fix_and_rerun
-      end
-    end
-  end
-end

lib/system_check/app/systemd_unit_files_or_init_script_exist_check.rb

+# frozen_string_literal: true
+
+module SystemCheck
+  module App
+    class SystemdUnitFilesOrInitScriptExistCheck < SystemCheck::BaseCheck
+      set_name 'Systemd unit files or init script exist?'
+      set_skip_reason 'skipped (omnibus-gitlab has neither init script nor systemd units)'
+
+      def skip?
+        omnibus_gitlab?
+      end
+
+      def check?
+        unit_paths = [
+          '/usr/local/lib/systemd/system/gitlab-gitaly.service',
+          '/usr/local/lib/systemd/system/gitlab-mailroom.service',
+          '/usr/local/lib/systemd/system/gitlab-puma.service',
+          '/usr/local/lib/systemd/system/gitlab-sidekiq.service',
+          '/usr/local/lib/systemd/system/gitlab.slice',
+          '/usr/local/lib/systemd/system/gitlab.target',
+          '/usr/local/lib/systemd/system/gitlab-workhorse.service'
+        ]
+        script_path = '/etc/init.d/gitlab'
+
+        unit_paths.all? { |s| File.exist?(s) } || File.exist?(script_path)
+      end
+
+      def show_error
+        try_fixing_it(
+          'Install the Service'
+        )
+        for_more_information(
+          see_installation_guide_section('Install the Service')
+        )
+        fix_and_rerun
+      end
+    end
+  end
+end

lib/system_check/app/init_script_up_to_date_check.rb → lib/system_check/app/systemd_unit_files_or_init_script_up_to_date_check.rb

@@ -2,16 +2,25 @@
 
 module SystemCheck
   module App
-    class InitScriptUpToDateCheck < SystemCheck::BaseCheck
+    class SystemdUnitFilesOrInitScriptUpToDateCheck < SystemCheck::BaseCheck
       SCRIPT_PATH = '/etc/init.d/gitlab'
+      UNIT_PATHS = [
+        '/usr/local/lib/systemd/system/gitlab-gitaly.service',
+        '/usr/local/lib/systemd/system/gitlab-mailroom.service',
+        '/usr/local/lib/systemd/system/gitlab-puma.service',
+        '/usr/local/lib/systemd/system/gitlab-sidekiq.service',
+        '/usr/local/lib/systemd/system/gitlab.slice',
+        '/usr/local/lib/systemd/system/gitlab.target',
+        '/usr/local/lib/systemd/system/gitlab-workhorse.service'
+      ].freeze
 
-      set_name 'Init script up-to-date?'
-      set_skip_reason 'skipped (omnibus-gitlab has no init script)'
+      set_name 'Systemd unit files or init script up-to-date?'
+      set_skip_reason 'skipped (omnibus-gitlab has neither init script nor systemd units)'
 
       def skip?
         return true if omnibus_gitlab?
 
-        unless init_file_exists?
+        unless unit_files_exist? || init_file_exists?
           self.skip_reason = "can't check because of previous errors"
 
           true
@@ -19,20 +28,19 @@ module SystemCheck
       end
 
       def check?
-        recipe_path = Rails.root.join('lib/support/init.d/', 'gitlab')
-
-        recipe_content = File.read(recipe_path)
-        script_content = File.read(SCRIPT_PATH)
+        if unit_files_exist?
+          return unit_files_up_to_date?
+        end
 
-        recipe_content == script_content
+        init_file_up_to_date?
       end
 
       def show_error
         try_fixing_it(
-          'Re-download the init script'
+          'Install the Service'
         )
         for_more_information(
-          see_installation_guide_section('Install Init Script')
+          see_installation_guide_section('Install the Service')
         )
         fix_and_rerun
       end
@@ -42,6 +50,31 @@ module SystemCheck
       def init_file_exists?
         File.exist?(SCRIPT_PATH)
       end
+
+      def unit_files_exist?
+        UNIT_PATHS.all? { |s| File.exist?(s) }
+      end
+
+      def init_file_up_to_date?
+        recipe_path = Rails.root.join('lib/support/init.d/', 'gitlab')
+
+        recipe_content = File.read(recipe_path)
+        script_content = File.read(SCRIPT_PATH)
+
+        recipe_content == script_content
+      end
+
+      def unit_files_up_to_date?
+        UNIT_PATHS.all? do |unit|
+          unit_name = File.basename(unit)
+          recipe_path = Rails.root.join('lib/support/systemd/', unit_name)
+
+          recipe_content = File.read(recipe_path)
+          unit_content = File.read(unit)
+
+          recipe_content == unit_content
+        end
+      end
     end
   end
 end

lib/system_check/incoming_email/initd_configured_check.rb → lib/system_check/incoming_email/mail_room_enabled_check.rb

@@ -2,20 +2,21 @@
 
 module SystemCheck
   module IncomingEmail
-    class InitdConfiguredCheck < SystemCheck::BaseCheck
-      set_name 'Init.d configured correctly?'
+    class MailRoomEnabledCheck < SystemCheck::BaseCheck
+      include ::SystemCheck::InitHelpers
+      set_name 'Mailroom enabled?'
 
       def skip?
         omnibus_gitlab?
       end
 
       def check?
-        mail_room_configured?
+        mail_room_enabled? || mail_room_configured?
       end
 
       def show_error
         try_fixing_it(
-          'Enable mail_room in the init.d configuration.'
+          'Enable mail_room'
         )
         for_more_information(
           'doc/administration/reply_by_email.md'
@@ -25,6 +26,13 @@ module SystemCheck
 
       private
 
+      def mail_room_enabled?
+        target = '/usr/local/lib/systemd/system/gitlab.target'
+        service = '/usr/local/lib/systemd/system/gitlab-mailroom.service'
+
+        File.exist?(target) && File.exist?(service) && systemd_get_wants('gitlab.target').include?("gitlab-mailroom.service")
+      end
+
       def mail_room_configured?
         path = '/etc/default/gitlab'
         File.exist?(path) && File.read(path).include?('mail_room_enabled=true')

lib/system_check/incoming_email/mail_room_running_check.rb

@@ -3,12 +3,13 @@
 module SystemCheck
   module IncomingEmail
     class MailRoomRunningCheck < SystemCheck::BaseCheck
+      include ::SystemCheck::InitHelpers
       set_name 'MailRoom running?'
 
       def skip?
         return true if omnibus_gitlab?
 
-        unless mail_room_configured?
+        unless mail_room_enabled? || mail_room_configured?
           self.skip_reason = "can't check because of previous errors"
           true
         end
@@ -20,10 +21,10 @@ module SystemCheck
 
       def show_error
         try_fixing_it(
-          sudo_gitlab('RAILS_ENV=production bin/mail_room start')
+          'Start mail_room'
         )
         for_more_information(
-          see_installation_guide_section('Install Init Script'),
+          'doc/administration/incoming_email.md',
           'see log/mail_room.log for possible errors'
         )
         fix_and_rerun
@@ -31,6 +32,13 @@ module SystemCheck
 
       private
 
+      def mail_room_enabled?
+        target = '/usr/local/lib/systemd/system/gitlab.target'
+        service = '/usr/local/lib/systemd/system/gitlab-mailroom.service'
+
+        File.exist?(target) && File.exist?(service) && systemd_get_wants('gitlab.target').include?("gitlab-mailroom.service")
+      end
+
       def mail_room_configured?
         path = '/etc/default/gitlab'
         File.exist?(path) && File.read(path).include?('mail_room_enabled=true')

lib/system_check/incoming_email_check.rb

@@ -14,7 +14,7 @@ module SystemCheck
         end
 
         if Rails.env.production?
-          checks << SystemCheck::IncomingEmail::InitdConfiguredCheck
+          checks << SystemCheck::IncomingEmail::MailRoomEnabledCheck
           checks << SystemCheck::IncomingEmail::MailRoomRunningCheck
         end
 

lib/system_check/init_helpers.rb

+# frozen_string_literal: true
+
+require 'open3'
+
+module SystemCheck
+  module InitHelpers
+    # Return the Wants= of a unit, empty if the unit doesn't exist
+    def systemd_get_wants(unitname)
+      stdout, _stderr, status = Open3.capture3("systemctl", "--no-pager", "show", unitname)
+
+      unless status
+        return []
+      end
+
+      wantsline = stdout.lines.find { |line| line.start_with?("Wants=") }
+
+      unless wantsline
+        return []
+      end
+
+      wantsline.delete_prefix("Wants=").strip.split
+    end
+  end
+end

lib/system_check/rake_task/app_task.rb

@@ -23,8 +23,8 @@ module SystemCheck
           SystemCheck::App::UploadsDirectoryExistsCheck,
           SystemCheck::App::UploadsPathPermissionCheck,
           SystemCheck::App::UploadsPathTmpPermissionCheck,
-          SystemCheck::App::InitScriptExistsCheck,
-          SystemCheck::App::InitScriptUpToDateCheck,
+          SystemCheck::App::SystemdUnitFilesOrInitScriptExistCheck,
+          SystemCheck::App::SystemdUnitFilesOrInitScriptUpToDateCheck,
           SystemCheck::App::ProjectsHaveNamespaceCheck,
           SystemCheck::App::RedisVersionCheck,
           SystemCheck::App::RubyVersionCheck,

lib/system_check/sidekiq_check.rb

@@ -39,6 +39,12 @@ module SystemCheck
 
       if (cluster_count == 1 && worker_count > 0) || (cluster_count == 0 && worker_count == 1)
         $stdout.puts "#{cluster_count}/#{worker_count}".color(:green)
+      elsif File.symlink?('/run/systemd/units/invocation:gitlab-sidekiq.service')
+        $stdout.puts "#{cluster_count}/#{worker_count}".color(:red)
+        try_fixing_it(
+          'sudo systemctl restart gitlab-sidekiq.service'
+        )
+        fix_and_rerun
       else
         $stdout.puts "#{cluster_count}/#{worker_count}".color(:red)
         try_fixing_it(

spec/lib/system_check/incoming_email_check_spec.rb

@@ -28,7 +28,7 @@ RSpec.describe SystemCheck::IncomingEmailCheck do
       it 'runs IMAP and mailroom checks' do
         expect(SystemCheck).to receive(:run).with('Reply by email', [
           SystemCheck::IncomingEmail::ImapAuthenticationCheck,
-          SystemCheck::IncomingEmail::InitdConfiguredCheck,
+          SystemCheck::IncomingEmail::MailRoomEnabledCheck,
           SystemCheck::IncomingEmail::MailRoomRunningCheck
         ])
 
@@ -43,7 +43,7 @@ RSpec.describe SystemCheck::IncomingEmailCheck do
 
       it 'runs mailroom checks' do
         expect(SystemCheck).to receive(:run).with('Reply by email', [
-          SystemCheck::IncomingEmail::InitdConfiguredCheck,
+          SystemCheck::IncomingEmail::MailRoomEnabledCheck,
           SystemCheck::IncomingEmail::MailRoomRunningCheck
         ])