1. 10 Nov, 2020 5 commits
    • Jérome Perrin's avatar
      stack/erp5: restart zopes on configuration changes · e2ebabf8
      Jérome Perrin authored
      We are using this pattern for most of our services since several
      months without any issue, so let's also use it for zopes. This
      makes automatic upgrade possible.
      
      Also remove "zope running current products" promise, since we
      restart we no longer need to check this.
      e2ebabf8
    • Jérome Perrin's avatar
      stack/erp5: socat wrapper to get haproxy stats · 1ccf2db8
      Jérome Perrin authored
      haproxy can be controlled with this socket, so it might be useful
      to "expose" it - it's not really expose because we only use a
      UNIX socket.
      1ccf2db8
    • Jérome Perrin's avatar
      stack/erp5: keep template-apache-backend-conf for slapos-master · 23667fc2
      Jérome Perrin authored
      Eventhough software/erp5 no longer uses this template, it is still used by
      software/slapos-master, so keep the entry point that will be empty for ERP5,
      but set only in SlapOS Master.
      23667fc2
    • Jérome Perrin's avatar
      software/erp5: remove httpd and use haproxy instead · 30058231
      Jérome Perrin authored
      Two big differences of haproxy are that haproxy does not use separate files
      for certificates and that the time in logs are in milliseconds, whereas with
      httpd it was microseconds. For certificates, when using caucase to generate
      haproxy's own server certificate we specify the same file as cert and key as
      caucase supports this and instead of maintaining directories of CA and CRL
      for client certificates (used by shared frontend), we build PEM files
      containing all CA certificates and all CRL together.
      For the logs, we use a new version of apachedex which supports `%{ms}T` for
      durations.
      
      Tests have been modified a bit, because haproxy uses HTTP/2.0 and not 1.1
      like httpd was doing several haproxy features (keep alive and gzip
      compression) are only available when backend uses HTTP/1.1, so we adjusted
      tests to use a 1.1 backend.
      
      There was also differences with logs, because of the time being in milliseconds.
      
      TestPublishedURLIsReachableMixin._checkERP5IsReachable was also updated, it
      was working by chance because when accessed behind httpd->haproxy->zope, zope
      was producing a redirect URL that was the URL of haproxy, which could be
      resolved by chance. This test was updated to access zope with a path that
      contains VirtualHostMonster magic, as the shared frontend ( with "zope" software
      type) is supposed to set.
      
      This should hopefuly solve the "502 Proxy Error" that we are observing with httpd.
      30058231
    • Jérome Perrin's avatar
      stack/caucase: make services auto restart on config change · 34b396f4
      Jérome Perrin authored
      use hash-existing-files feature of wrapper recipe to make
      sevices restart automatically when they are re-requested with different
      parameters.
      34b396f4
  2. 09 Nov, 2020 2 commits
    • Jérome Perrin's avatar
      stack/caucase: generate key / csr only once · 355aa844
      Jérome Perrin authored
      If key or csr are already present, we should not re-run this
      openssl command which generates a new key and a new CSR.
      355aa844
    • Jérome Perrin's avatar
      stack/erp5: fix balancer CSR generation · 002e4185
      Jérome Perrin authored
      Caucase rerequest uses a CSR *template* and use it to generate
      a new CSR with a new key, so we should not use the actual key to
      generate this CSR, because it is caucase rerequest job to generate
      the key.
      Also, we should be careful not to generate a new CSR every time this
      command run, otherwise a new key will be generated and a new CSR will
      be sent to caucase, but caucase will not sign it automatically (since
      we configure it to sign only one certificate).
      
      This means that the case of IP address changes is currently not
      supported automatically. To support it we would need to:
       - force generation of a new CSR template
       - force caucase rerequester to request a new certificate (by removing
        existing certificate)
       - force caucased to sign the new certificate
      
      This commit fix indentation and remove simplefile macro that is no longer used
      002e4185
  3. 04 Nov, 2020 3 commits
    • Jérome Perrin's avatar
      software/erp5/test: make forwarded for test more relaxed · 170b2992
      Jérome Perrin authored
      The test expecting that X-Forwarded-For is empty can also accept the
      case where X-Forwarded-For header is not present.
      170b2992
    • Vincent Pelletier's avatar
      stack/erp5: Make bt5 default value a string at the last step. · 3c2ce500
      Vincent Pelletier authored
      Make the value and its changes easier to read.
      3c2ce500
    • Jérome Perrin's avatar
      ERP5: Test balancer partition and use caucase certificate for balancer · af7a0208
      Jérome Perrin authored
      Revert f8f72a17 ([erp5] don't use caucase generated certificate for now, 2019-03-12) since nothing prevents us drom using caucase certificate now.
       
      Use [managed resources](nexedi/slapos.core!259) to simplify existing tests and introduce tests for:
      
      ## Access Log
      
       - [x] balancer partition should produce logs in apache "combined" log format with microsecond timing of requests.
       - [x] these logs should be rotated daily
       - [x] an [apachedex](https://lab.nexedi.com/nexedi/apachedex) report is ran on these logs daily.
      
      ## Balancing
      
       - [x] requests are balanced to multiple backends using round-robin algorithm
       - [x] if backend is down it is excluded
       - [x] a "sticky cookie" is used so that clients are associated to the same backend
          - [x] the cookie is set by balancer
          - [x] when client comes with a cookie it "sticks" on the associated backend
          - [x] if "sticked" backend is down, another backend will be used
      
      ## Content-Encoding
      
       - [x] balancer encodes responses in gzip for some configured content types.
      
      ## HTTP
      
       - [x] Server uses HTTP/1.1 or more and keep connection with clients
      
      ## TLS (server certificate)
      
      In this MR we also change apache to use a caucase managed certificate and add test coverage for:
      
       - [x] balancer listen on https with a certificate that can be verified using the CA from caucase.
       - [x] balancer uses the new certificate when its own certificate is renewed.
      
      But we don't add support for:
       -  ~~balancer can be instantiated with a certificate and key passed as SlapOS request parameters (code [here](https://lab.nexedi.com/nexedi/slapos/blob/757c1a4ddee93659d5e2649e4252d87bf9494566/stack/erp5/instance-balancer.cfg.in#L208-213))~~ this use case is the job of caucase, so we no longer support this.
      
      ## TLS (client certificate)
       - [x] balancer verifies frontend certificates from frontend caucases ( also tested in "Forwarded-For" section )
       - [x] if frontend provided a verified certificate, balancer set `remote-user` header
       - [x] balancer updates CRL from caucases ( `caucase-updater-housekeeper` )
       - (NOT TESTED) balancer updates CA certificate from caucase ( `caucase-updater-housekeeper` ). Since this is would be complex to test and basic functionality of `caucase-updater-housekeeper` for frontend caucases is covered by CRL test, we don't test this for simplicity.
      
      ## "Forwarded-For" header
      
      This was also covered by existing tests:  
      
       - [x] balancer set `X-Forwarded-For` header when frontend certificate can be verified
       - [x] balancer strips existing `X-Forwarded-For`
      
      ## Integration with the rest of ERP5 software release
      
      This was also covered by existing tests:  
      
      - [x] The https URL of each Zope family is published and replies properly
      - [x] Some https URLs are generated for `runUnitTest`, so that test run with an https certificate. This is also covered by regular ERP5 functional tests.
      
      See merge request nexedi/slapos!840
      af7a0208
  4. 02 Nov, 2020 4 commits
  5. 30 Oct, 2020 1 commit
  6. 29 Oct, 2020 5 commits
  7. 28 Oct, 2020 3 commits
  8. 27 Oct, 2020 12 commits
  9. 26 Oct, 2020 4 commits
  10. 23 Oct, 2020 1 commit