Skip to content

slapos
slapos
Commit af7a0208 authored by Jérome Perrin's avatar Jérome Perrin
Browse files
Options

ERP5: Test balancer partition and use caucase certificate for balancer 

Revert f8f72a17 ([erp5] don't use caucase generated certificate for now, 2019-03-12) since nothing prevents us drom using caucase certificate now.
 
Use [managed resources](nexedi/slapos.core!259) to simplify existing tests and introduce tests for:

## Access Log

 - [x] balancer partition should produce logs in apache "combined" log format with microsecond timing of requests.
 - [x] these logs should be rotated daily
 - [x] an [apachedex](https://lab.nexedi.com/nexedi/apachedex) report is ran on these logs daily.

## Balancing

 - [x] requests are balanced to multiple backends using round-robin algorithm
 - [x] if backend is down it is excluded
 - [x] a "sticky cookie" is used so that clients are associated to the same backend
    - [x] the cookie is set by balancer
    - [x] when client comes with a cookie it "sticks" on the associated backend
    - [x] if "sticked" backend is down, another backend will be used

## Content-Encoding

 - [x] balancer encodes responses in gzip for some configured content types.

## HTTP

 - [x] Server uses HTTP/1.1 or more and keep connection with clients

## TLS (server certificate)

In this MR we also change apache to use a caucase managed certificate and add test coverage for:

 - [x] balancer listen on https with a certificate that can be verified using the CA from caucase.
 - [x] balancer uses the new certificate when its own certificate is renewed.

But we don't add support for:
 -  ~~balancer can be instantiated with a certificate and key passed as SlapOS request parameters (code [here](https://lab.nexedi.com/nexedi/slapos/blob/757c1a4ddee93659d5e2649e4252d87bf9494566/stack/erp5/instance-balancer.cfg.in#L208-213))~~ this use case is the job of caucase, so we no longer support this.

## TLS (client certificate)
 - [x] balancer verifies frontend certificates from frontend caucases ( also tested in "Forwarded-For" section )
 - [x] if frontend provided a verified certificate, balancer set `remote-user` header
 - [x] balancer updates CRL from caucases ( `caucase-updater-housekeeper` )
 - (NOT TESTED) balancer updates CA certificate from caucase ( `caucase-updater-housekeeper` ). Since this is would be complex to test and basic functionality of `caucase-updater-housekeeper` for frontend caucases is covered by CRL test, we don't test this for simplicity.

## "Forwarded-For" header

This was also covered by existing tests:  

 - [x] balancer set `X-Forwarded-For` header when frontend certificate can be verified
 - [x] balancer strips existing `X-Forwarded-For`

## Integration with the rest of ERP5 software release

This was also covered by existing tests:  

- [x] The https URL of each Zope family is published and replies properly
- [x] Some https URLs are generated for `runUnitTest`, so that test run with an https certificate. This is also covered by regular ERP5 functional tests.

See merge request nexedi/slapos!840
parents babec085 f1173ea5
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 842 additions and 315 deletions
software/erp5/test/setup.py
View file @ af7a0208
......@@ -33,7 +33,7 @@ with open("README.md") as f:
setup(name=name,
version=version,
description="Test for SlapOS' ERP5 software releae",
description="Test for SlapOS' ERP5 software release",
long_description=long_description,
long_description_content_type='text/markdown',
maintainer="Nexedi",
......@@ -50,8 +50,9 @@ setup(name=name,
'mysqlclient',
'backports.lzma',
'cryptography',
'pexpect',
'pyOpenSSL',
],
zip_safe=True,
'typing; python_version<"3"',
],
test_suite='test',
)
software/erp5/test/test/test_balancer.py
View file @ af7a0208
This diff is collapsed.
software/erp5/test/test/test_erp5.py
View file @ af7a0208
......@@ -31,6 +31,7 @@ import glob
import urlparse
import socket
import time
import tempfile
import psutil
import requests
......@@ -43,7 +44,7 @@ setUpModule # pyflakes
class TestPublishedURLIsReachableMixin(object):
"""Mixin that checks that default page of ERP5 is reachable.
"""
def _checkERP5IsReachable(self, url):
def _checkERP5IsReachable(self, url, verify):
# What happens is that instanciation just create the services, but does not
# wait for ERP5 to be initialized. When this test run ERP5 instance is
# instanciated, but zope is still busy creating the site and haproxy replies
......@@ -51,7 +52,7 @@ class TestPublishedURLIsReachableMixin(object):
# erp5 site is not created, with 500 when mysql is not yet reachable, so we
# retry in a loop until we get a succesful response.
for i in range(1, 60):
r = requests.get(url, verify=False) # XXX can we get CA from caucase already ?
r = requests.get(url, verify=verify)
if r.status_code != requests.codes.ok:
delay = i * 2
self.logger.warn("ERP5 was not available, sleeping for %ds and retrying", delay)
......@@ -62,19 +63,36 @@ class TestPublishedURLIsReachableMixin(object):
self.assertIn("ERP5", r.text)
def _getCaucaseServiceCACertificate(self):
ca_cert = tempfile.NamedTemporaryFile(
prefix="ca.crt.pem",
mode="w",
delete=False,
)
ca_cert.write(
requests.get(
urlparse.urljoin(
self.getRootPartitionConnectionParameterDict()['caucase-http-url'],
'/cas/crt/ca.crt.pem',
)).text)
self.addCleanup(os.unlink, ca_cert.name)
return ca_cert.name
def test_published_family_default_v6_is_reachable(self):
"""Tests the IPv6 URL published by the root partition is reachable.
"""
param_dict = self.getRootPartitionConnectionParameterDict()
self._checkERP5IsReachable(
urlparse.urljoin(param_dict['family-default-v6'], param_dict['site-id']))
urlparse.urljoin(param_dict['family-default-v6'], param_dict['site-id']),
self._getCaucaseServiceCACertificate())
def test_published_family_default_v4_is_reachable(self):
"""Tests the IPv4 URL published by the root partition is reachable.
"""
param_dict = self.getRootPartitionConnectionParameterDict()
self._checkERP5IsReachable(
urlparse.urljoin(param_dict['family-default'], param_dict['site-id']))
urlparse.urljoin(param_dict['family-default'], param_dict['site-id']),
self._getCaucaseServiceCACertificate())
class TestDefaultParameters(ERP5InstanceTestCase, TestPublishedURLIsReachableMixin):
......
software/seleniumserver/test/test.py
View file @ af7a0208
......@@ -34,8 +34,10 @@ import unittest
import urlparse
import base64
import hashlib
import logging
import contextlib
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from BaseHTTPServer import BaseHTTPRequestHandler
from io import BytesIO
import paramiko
......@@ -48,81 +50,69 @@ from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from slapos.testing.testcase import makeModuleSetUpAndTestCaseClass
from slapos.testing.utils import findFreeTCPPort, ImageComparisonTestCase
from slapos.testing.utils import findFreeTCPPort, ImageComparisonTestCase, ManagedHTTPServer
setUpModule, SeleniumServerTestCase = makeModuleSetUpAndTestCaseClass(
os.path.abspath(
os.path.join(os.path.dirname(__file__), '..', 'software.cfg')))
class WebServer(ManagedHTTPServer):
class RequestHandler(BaseHTTPRequestHandler):
"""Request handler for our test server.
The implemented server is:
- submit q and you'll get a page with q as title
- upload a file and the file content will be displayed in div.uploadedfile
"""
def do_GET(self):
self.send_response(200)
self.end_headers()
self.wfile.write(
'''
<html>
<title>Test page</title>
<body>
<style> p { font-family: Arial; } </style>
<form action="/" method="POST" enctype="multipart/form-data">
<input name="q" type="text"></input>
<input name="f" type="file" ></input>
<input type="submit" value="I'm feeling lucky"></input>
</form>
<p>the quick brown fox jumps over the lazy dog</p>
</body>
</html>''')
def do_POST(self):
form = cgi.FieldStorage(
fp=self.rfile,
headers=self.headers,
environ={
'REQUEST_METHOD': 'POST',
'CONTENT_TYPE': self.headers['Content-Type'],
})
self.send_response(200)
self.end_headers()
file_data = 'no file'
if form.has_key('f'):
file_data = form['f'].file.read()
self.wfile.write(
'''
<html>
<title>%s</title>
<div>%s</div>
</html>
''' % (form['q'].value, file_data))
log_message = logging.getLogger(__name__ + '.WebServer').info
class WebServerMixin(object):
"""Mixin class which provides a simple web server reachable at self.server_url
"""
def setUp(self):
"""Start a minimal web server.
"""
class TestHandler(BaseHTTPRequestHandler):
"""Request handler for our test server.
The implemented server is:
- submit q and you'll get a page with q as title
- upload a file and the file content will be displayed in div.uploadedfile
"""
def log_message(self, *args, **kw):
if SeleniumServerTestCase._debug:
BaseHTTPRequestHandler.log_message(self, *args, **kw)
def do_GET(self):
self.send_response(200)
self.end_headers()
self.wfile.write(
'''
<html>
<title>Test page</title>
<body>
<style> p { font-family: Arial; } </style>
<form action="/" method="POST" enctype="multipart/form-data">
<input name="q" type="text"></input>
<input name="f" type="file" ></input>
<input type="submit" value="I'm feeling lucky"></input>
</form>
<p>the quick brown fox jumps over the lazy dog</p>
</body>
</html>''')
def do_POST(self):
form = cgi.FieldStorage(
fp=self.rfile,
headers=self.headers,
environ={
'REQUEST_METHOD': 'POST',
'CONTENT_TYPE': self.headers['Content-Type'],
})
self.send_response(200)
self.end_headers()
file_data = 'no file'
if form.has_key('f'):
file_data = form['f'].file.read()
self.wfile.write(
'''
<html>
<title>%s</title>
<div>%s</div>
</html>
''' % (form['q'].value, file_data))
super(WebServerMixin, self).setUp()
ip = os.environ.get('SLAPOS_TEST_IPV4', '127.0.1.1')
port = findFreeTCPPort(ip)
server = HTTPServer((ip, port), TestHandler)
self.server_process = multiprocessing.Process(target=server.serve_forever)
self.server_process.start()
self.server_url = 'http://%s:%s/' % (ip, port)
def tearDown(self):
self.server_process.terminate()
self.server_process.join()
super(WebServerMixin, self).tearDown()
self.server_url = self.getManagedResource('web_server', WebServer).url
class BrowserCompatibilityMixin(WebServerMixin):
......
software/slapos-sr-testing/software.cfg
View file @ af7a0208
......@@ -316,3 +316,4 @@ funcsigs = 1.0.2
mysqlclient = 1.3.12
pexpect = 4.8.0
ptyprocess = 0.6.0
typing = 3.7.4.3
stack/erp5/buildout.hash.cfg
View file @ af7a0208
......@@ -90,7 +90,7 @@ md5sum = 2f3ddd328ac1c375e483ecb2ef5ffb57
[template-balancer]
filename = instance-balancer.cfg.in
md5sum = bb9a953ce22f7d5188385f0171b6198e
md5sum = ecf119142e6b5cd85a2ba397552d2142
[template-haproxy-cfg]
filename = haproxy.cfg.in
......
stack/erp5/instance-balancer.cfg.in
View file @ af7a0208
......@@ -18,19 +18,52 @@ per partition. No more (undefined result), no less (IndexError).
recipe = slapos.recipe.template:jinja2
mode = 644
[balancer-csr-request-config]
< = jinja2-template-base
template = inline:
[req]
prompt = no
req_extensions = req_ext
distinguished_name = dn
[ dn ]
CN = example.com
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
IP.1 = {{ ipv4 }}
{% if ipv6_set -%}
IP.2 = {{ ipv6 }}
{% endif %}
rendered = ${buildout:parts-directory}/${:_buildout_section_name_}/${:_buildout_section_name_}.txt
[balancer-csr-request]
recipe = plone.recipe.command
command = {{ parameter_dict["openssl"] }}/bin/openssl req \
-newkey rsa:2048 \
-batch \
-new \
-nodes \
-keyout '${apache-conf-ssl:key}' \
-config '${balancer-csr-request-config:rendered}' \
-out '${:csr}'
stop-on-error = true
csr = ${directory:etc}/${:_buildout_section_name_}.csr.pem
{{ caucase.updater(
prefix='caucase-updater',
buildout_bin_directory=parameter_dict['bin-directory'],
updater_path='${directory:services-on-watch}/caucase-updater',
url=ssl_parameter_dict['caucase-url'],
data_dir='${directory:srv}/caucase-updater',
crt_path='${apache-conf-ssl:caucase-cert}',
crt_path='${apache-conf-ssl:cert}',
ca_path='${directory:srv}/caucase-updater/ca.crt',
crl_path='${directory:srv}/caucase-updater/crl.pem',
key_path='${apache-conf-ssl:caucase-key}',
key_path='${apache-conf-ssl:key}',
on_renew='${apache-graceful:output}',
max_sleep=ssl_parameter_dict.get('max-crl-update-delay', 1.0),
template_csr_pem=ssl_parameter_dict.get('csr'),
template_csr=None if ssl_parameter_dict.get('csr') else '${balancer-csr-request:csr}',
openssl=parameter_dict['openssl'] ~ '/bin/openssl',
)}}
{% do section('caucase-updater') -%}
......@@ -176,9 +209,6 @@ hash-files = ${haproxy-cfg:rendered}
[apache-conf-ssl]
cert = ${directory:apache-conf}/apache.crt
key = ${directory:apache-conf}/apache.pem
# XXX caucase certificate is not supported by caddy for now
caucase-cert = ${directory:apache-conf}/apache-caucase.crt
caucase-key = ${directory:apache-conf}/apache-caucase.pem
{% if frontend_caucase_url_list -%}
depends = ${caucase-updater-housekeeper-run:recipe}
ca-cert-dir = ${directory:apache-ca-cert-dir}
......@@ -201,19 +231,6 @@ context = key content {{content_section_name}}:content
mode = {{ mode }}
{%- endmacro %}
[apache-ssl]
{% if ssl_parameter_dict.get('key') -%}
key = ${apache-ssl-key:rendered}
cert = ${apache-ssl-cert:rendered}
{{ simplefile('apache-ssl-key', '${apache-conf-ssl:key}', ssl_parameter_dict['key']) }}
{{ simplefile('apache-ssl-cert', '${apache-conf-ssl:cert}', ssl_parameter_dict['cert']) }}
{% else %}
recipe = plone.recipe.command
command = "{{ parameter_dict['openssl'] }}/bin/openssl" req -newkey rsa -batch -new -x509 -days 3650 -nodes -keyout "${:key}" -out "${:cert}"
key = ${apache-conf-ssl:key}
cert = ${apache-conf-ssl:cert}
{%- endif %}
[apache-conf-parameter-dict]
backend-list = {{ dumps(apache_dict.values()) }}
zope-virtualhost-monster-backend-dict = {{ dumps(zope_virtualhost_monster_backend_dict) }}
......@@ -225,8 +242,8 @@ access-log = ${directory:log}/apache-access.log
# Apache 2.4's default value (60 seconds) can be a bit too short
timeout = 300
# Basic SSL server configuration
cert = ${apache-ssl:cert}
key = ${apache-ssl:key}
cert = ${apache-conf-ssl:cert}
key = ${apache-conf-ssl:key}
cipher =
ssl-session-cache = ${directory:log}/apache-ssl-session-cache
{% if frontend_caucase_url_list -%}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7