gitlab: Wait a bit for PostgreSQL to be ready in Unicorn startup

As it was outlined in 5a744de7 (gitlab: Compile assets on instantiation
and make sure DB is properly setup/migrated before unicorn runs) we are
performing DB initialization and migration in pre-action as part of
unicorn startup script. But that currently has one drawback:

    if all services start at the same time - e.g. both PostgreSQL and
    Unicorn - and that is a common scenario when SR is compiled and
    instantiated / started, PostgreSQL is usually not yet ready to
    process queries from Unicorn startup script, and first-time Unicorn
    startup fails.

Until now this problem was workarounded by manually starting unicorn
second time - after some time postgresql is started and ready. But why
do it manually, if we can do the same logic automatically. So fix it:

    in Unicorn startup script wait a bit (up to 5 seconds) for
    PostgreSQL to become ready.

/cc @kazuhiko, @jerome

software/gitlab/gitlab-unicorn-startup.in

@@ -15,11 +15,21 @@ die() {
 #   https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/recipes/postgresql.rb
 
 # initial db setup
-pgtables="$({{ psql_bin }}   \
+# ( first quering PG several times waiting a bit till postgresql is started and ready )
+tpgwait=5
+while true; do
+    pgtables="$({{ psql_bin }}   \
         -h {{ pgsql['pgdata-directory'] }}  \
         -U {{ pgsql.superuser }}            \
         -d {{ pgsql.dbname }}               \
-    -c '\d')" || die "pg query problem"
+        -c '\d')" && break
+
+    tpgwait=$(( $tpgwait - 1 ))
+    test $tpgwait = 0 && die "pg query problem"
+    echo "I: PostgreSQL is not ready (yet ?); will retry $tpgwait times..." 1>&2
+    sleep 1
+done
+echo "I: PostgreSQL ready." 1>&2
 
 if echo "$pgtables" | grep -q '^No relations found' ; then
     $RAKE db:schema:load db:seed_fu  || die "initial db setup failed"

software/gitlab/software.cfg

@@ -239,7 +239,7 @@ md5sum  = f061d529b71241d58affbf7aec5c8af1
 
 [gitlab-unicorn-startup.in]
 <= download-file
-md5sum  = 2716afaa9445c0c429c6b211356ebe8f
+md5sum  = 14c5632182d830c03f7788c85d6f4da1
 
 [gitlab.yml.in]
 <= download-template