Merge branch 'feature/caddy-frontend-auth-to-backend'

Frontend can now do SSL client authentication to the backend. This is turned
off by default, but easy to enable by proper switch.

In order to achieve this result haproxy has been introduced as middleman just
before the backend. Also it came with rsyslogd, as haproxy can't log by
itself.

Documentation has been updated, also changelog has been introduced.

A lot of additional tests has been added due to adding additional component
(haproxy) which required a bit different ways to approach to the request input
data.

See merge request nexedi/slapos!771

software/caddy-frontend/CHANGES.caddy_frontend.rst

+Changes
+=======
+
+Here are listed the most important changes, which might affect upgrades.
+
+1.0.XXX XXXX-XX-XX
+------------------
+
+ * manual customisation of profiles has been dropped, as not used, dropped keys are ``apache_custom_http``, ``apache_custom_https``, ``caddy_custom_http``, ``caddy_custom_https`` from slaves and ``-frontend-authorized-slave-string`` from master
+ * ``re6st-optimal-test`` has been dropped from slave
+ * QUIC is dropped, as was not used and has been superseded by HTTP/3, dropped key is ``enable-quic`` from master
+ * haproxy is used as a gateway to backends:
+
+   * ``automatic-internal-backend-client-caucase-csr`` switch for master is introduced to control it CSR signing
+   * ``proxy-try-duration`` and ``proxy-try-interval`` has been dropped, as Caddy is not used anymore to connect to the backend, and instead ``backend-connect-timeout`` and ``backend-connect-retries`` is used, as it comes from Haproxy
+   * ``backend-client-caucase-url`` is returned in master and slave, so that backends can use caucase to fetch CA from frontend cluster
+   * ``request-timeout`` is supported per slave, as now it became possible
+   * ``authenticate-to-backend`` is added for master and slave, defaulting to False, to have control over cluster default authentication, and make it possible to do it per slave
+
+1.0.149 (2020-05-05)
+--------------------
+
+ * no changes noted

software/caddy-frontend/README.caddy_frontend.rst

@@ -442,6 +442,18 @@ Then specify in the master instance parameters:
  * set ``port`` to ``443``
  * set ``plain_http_port`` to ``80``
 
+Authentication to the backend
+=============================
+
+The cluster generates CA served by caucase, available with ``backend-client-caucase-url`` return parameter.
+
+Then, each slave configured with ``authenticate-to-backend`` to true, will use a certificate signed by this CA while accessing https backend.
+
+This allows backends to:
+
+ * restrict access only from some frontend clusters
+ * trust values (like ``X-Forwarded-For``) sent by the frontend
+
 Technical notes
 ===============
 
@@ -454,11 +466,22 @@ Instantiating caddy-frontend results with a cluster in various partitions:
  * kedifa (contains kedifa server)
  * caddy-frontend-N which contains the running processes to serve sites - this partition can be replicated by ``-frontend-quantity`` parameter
 
-So it means sites are served in `caddy-frontend-N` partition, and this partition is structured as:
+It means sites are served in ``caddy-frontend-N`` partition, and this partition is structured as:
+
+ * Caddy serving the browser [client-facing-caddy]
+ * (optional) Apache Traffic Server for caching [ats]
+ * Haproxy as a way to communicate to the backend [backend-facing-haproxy]
+ * some other additional tools (6tunnel, monitor, etc)
 
- * Caddy serving the browser
- * (optional) Apache Traffic Server for caching
- * Caddy connected to the backend
+In case of slaves without cache (``enable_cache = False``) the request will travel as follows::
+
+  client-facing-caddy --> backend-facing-haproxy --> backend
+
+In case of slaves using cache (``enable_cache = True``) the request will travel as follows::
+
+  client-facing-caddy --> ats --> backend-facing-haproxy --> backend
+
+Usage of Haproxy as a relay to the backend allows much better control of the backend, removes the hassle of checking the backend from Caddy and allows future developments like client SSL certificates to the backend or even health checks.
 
 Kedifa implementation
 ---------------------
@@ -475,3 +498,32 @@ Support for X-Real-Ip and X-Forwarded-For
 -----------------------------------------
 
 X-Forwarded-For and X-Real-Ip are transmitted to the backend, but only for IPv4 access to the frontend. In case of IPv6 access, the provided IP will be wrong, because of using 6tunnel.
+
+Automatic Internal Caucase CSR
+------------------------------
+
+Cluster is composed on many instances, which are landing on separate partitions, so some way is needed to bootstrap trust between the partitions.
+
+There are two ways to achieve it:
+
+ * use default, Automatic Internal Caucase CSR used to replace human to sign CSRs against internal CAUCASEs automatic bootstrap, which leads to some issues, described later
+ * switch to manual bootstrap, which requires human to create and manage user certificate (with caucase-updater) and then sign new frontend nodes appearing in the system
+
+The issues during automatic bootstrap are:
+
+ * rouge or hacked SlapOS Master can result with adding rouge frontend nodes to the cluster, which will be trusted, so it will be possible to fetch all certificates and keys from Kedifa or to login to backends
+ * when new node is added there is short window, when rouge person is able to trick automatic signing, and have it's own node added
+
+In both cases promises will fail on node which is not able to get signed, but in case of Kedifa the damage already happened (certificates and keys are compromised). So in case if cluster administrator wants to stay on the safe side, both automatic bootstraps shall be turned off.
+
+How the automatic signing works
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Having in mind such structure:
+
+ * instance with caucase: ``caucase-instance``
+ * N instances which want to get their CSR signed: ``csr-instance``
+
+In ``caucase-instance`` CAUCASE user is created by automatically signing one user certificate, which allows to sign service certificates.
+
+The ``csr-instance`` creates CSR, extracts the ID of the CSR, exposes it via HTTP and ask caucase on ``caucase-instance`` to sign it. The ``caucase-instance`` checks that exposed CSR id matches the one send to caucase and by using created user to signs it.

software/caddy-frontend/buildout.hash.cfg

@@ -14,7 +14,7 @@
 # not need these here).
 [template]
 filename = instance.cfg.in
-md5sum = 20f7a925e686949092823595c79a0523
+md5sum = 99f2f6d8818da4a98ca48412453c4f90
 
 [template-common]
 filename = instance-common.cfg.in
@@ -22,15 +22,15 @@ md5sum = c801b7f9f11f0965677c22e6bbe9281b
 
 [template-apache-frontend]
 filename = instance-apache-frontend.cfg.in
-md5sum = 1eae89931b305d9b9e34769946203c1c
+md5sum = 23237969bbd9e974ac674b2052e8d67c
 
 [template-caddy-replicate]
 filename = instance-apache-replicate.cfg.in
-md5sum = 6d7113ebf0c46b0e4c72c128ebb647db
+md5sum = 19debfbc27c464f451b1eb5bb5ce3c84
 
 [template-slave-list]
 _update_hash_filename_ = templates/apache-custom-slave-list.cfg.in
-md5sum = 9a5919a00a166fb8e902a72707ebc407
+md5sum = e142dad44db1a25d46a31661aa2f075f
 
 [template-replicate-publish-slave-information]
 _update_hash_filename_ = templates/replicate-publish-slave-information.cfg.in
@@ -38,7 +38,7 @@ md5sum = 7e3ee70c447f8203273d78f66ab519c3
 
 [template-caddy-frontend-configuration]
 _update_hash_filename_ = templates/Caddyfile.in
-md5sum = f0faf6d2e6c187df7e25bf717676f9df
+md5sum = 2503056e35463e045db3329bb8b6fae8
 
 [caddy-backend-url-validator]
 filename = templates/caddy-backend-url-validator.in
@@ -50,11 +50,11 @@ md5sum = f20d6c3d2d94fb685f8d26dfca1e822b
 
 [template-default-slave-virtualhost]
 _update_hash_filename_ = templates/default-virtualhost.conf.in
-md5sum = 8137d3da8658d7b2d8c99c9a283a4e5e
+md5sum = 266f175dbdfc588af7a86b0b1884fe73
 
-[template-cached-slave-virtualhost]
-_update_hash_filename_ = templates/cached-virtualhost.conf.in
-md5sum = e839ca3cb308f7fcdfa06c2f1b95e93f
+[template-backend-haproxy-configuration]
+_update_hash_filename_ = templates/backend-haproxy.cfg.in
+md5sum = 8daeaaa02190d90e0df91e364b1d2fc0
 
 [template-log-access]
 _update_hash_filename_ = templates/template-log-access.conf.in
@@ -94,7 +94,7 @@ md5sum = 061cc244558fd3af2b6bacf17cae5555
 
 [template-validate-script]
 _update_hash_filename_ = templates/validate-script.sh.in
-md5sum = f26e11574f266c7437c9c89e3c93825a
+md5sum = 53e5d7ba2827bff003051f74f24ffe4f
 
 [template-configuration-state-script]
 _update_hash_filename_ = templates/configuration-state-script.sh.in
@@ -115,3 +115,7 @@ md5sum = 38792c2dceae38ab411592ec36fff6a8
 [template-kedifa]
 filename = instance-kedifa.cfg.in
 md5sum = 9d6111a5d6bc07e708116ca331925241
+
+[template-rsyslogd-conf]
+_update_hash_filename_ = templates/rsyslogd.conf.in
+md5sum = 6f0fd930ffb3230b7e8c781f88693d69

software/caddy-frontend/common.cfg

@@ -10,6 +10,8 @@ extends =
   ../../component/trafficserver/buildout.cfg
   ../../component/6tunnel/buildout.cfg
   ../../component/xz-utils/buildout.cfg
+  ../../component/rsyslogd/buildout.cfg
+  ../../component/haproxy/buildout.cfg
 
   ../../stack/caucase/buildout.cfg
 # Monitoring stack (keep on bottom)
@@ -94,6 +96,8 @@ bin_directory = ${buildout:bin-directory}
 sixtunnel = ${6tunnel:location}
 caddy = ${caddy:output}
 caddy_location = ${caddy:location}
+haproxy_executable = ${haproxy:location}/sbin/haproxy
+rsyslogd_executable = ${rsyslogd:location}/sbin/rsyslogd
 curl = ${curl:location}
 dash = ${dash:location}
 gzip = ${gzip:location}
@@ -108,7 +112,8 @@ kedifa-csr = ${:bin_directory}/kedifa-csr
 xz_location = ${xz-utils:location}
 
 monitor_template = ${monitor-template:output}
-template_cached_slave_virtualhost = ${template-cached-slave-virtualhost:target}
+template_backend_haproxy_configuration = ${template-backend-haproxy-configuration:target}
+template_rsyslogd_conf = ${template-rsyslogd-conf:target}
 template_caddy_frontend_configuration = ${template-caddy-frontend-configuration:target}
 template_graceful_script = ${template-graceful-script:target}
 template_validate_script = ${template-validate-script:target}
@@ -184,7 +189,7 @@ mode = 640
 [template-default-slave-virtualhost]
 <=download-template
 
-[template-cached-slave-virtualhost]
+[template-backend-haproxy-configuration]
 <=download-template
 
 [template-log-access]
@@ -222,3 +227,6 @@ mode = 0644
 
 [template-configuration-state-script]
 <=download-template
+
+[template-rsyslogd-conf]
+<=download-template

software/caddy-frontend/instance-apache-frontend.cfg.in

@@ -14,6 +14,8 @@ parts =
   switch-caddy-softwaretype
   caucase-updater
   caucase-updater-promise
+  backend-client-caucase-updater
+  backend-client-caucase-updater-promise
   frontend-caddy-graceful
   port-redirection
   promise-frontend-caddy-configuration
@@ -21,8 +23,6 @@ parts =
   promise-caddy-frontend-v4-http
   promise-caddy-frontend-v6-https
   promise-caddy-frontend-v6-http
-  promise-caddy-frontend-cached
-  promise-caddy-frontend-ssl-cached
 
   trafficserver-launcher
   trafficserver-reload
@@ -43,6 +43,13 @@ parts =
   monitor-caddy-server-status-wrapper
   monitor-verify-re6st-connectivity
 
+  backend-haproxy
+  backend-haproxy-graceful
+  promise-backend-haproxy-http
+  promise-backend-haproxy-https
+  promise-backend-haproxy-configuration
+  logrotate-entry-backend-haproxy
+
 # Create all needed directories
 [directory]
 recipe = slapos.cookbook:mkdirectory
@@ -57,10 +64,12 @@ template = ${buildout:directory}/template/
 backup = ${:srv}/backup
 log = ${:var}/log
 run = ${:var}/run
+backend-haproxy-rsyslogd-spool = ${:run}/backend-haproxy-rsyslogd-spool
 service = ${:etc}/service
 etc-run = ${:etc}/run
 
 ca-dir = ${:srv}/ssl
+backend-client-dir = ${:srv}/backend-client
 # BBB: SlapOS Master non-zero knowledge BEGIN
 bbb-ssl-dir = ${:srv}/bbb-ssl
 # BBB: SlapOS Master non-zero knowledge END
@@ -147,7 +156,8 @@ context =
 [software-release-path]
 template-empty = {{ parameter_dict['template_empty'] }}
 template-default-slave-virtualhost = {{ parameter_dict['template_default_slave_virtualhost'] }}
-template-cached-slave-virtualhost = {{ parameter_dict['template_cached_slave_virtualhost'] }}
+template-backend-haproxy-configuration = {{ parameter_dict['template_backend_haproxy_configuration'] }}
+template-rsyslogd-conf = {{ parameter_dict['template_rsyslogd_conf'] }}
 caddy-location = {{ parameter_dict['caddy_location'] }}
 
 [kedifa-login-config]
@@ -203,6 +213,47 @@ kedifa-updater-mapping-file = ${directory:etc}/kedifa_updater_mapping.txt
 kedifa-updater-state-file = ${directory:srv}/kedifa_updater_state.json
 slave_kedifa_information = {{ dumps(slapparameter_dict['slave-kedifa-information']) }}
 
+[backend-client-login-config]
+d = ${directory:backend-client-dir}
+template-csr = ${:d}/csr.pem
+key = ${:d}/certificate.pem
+certificate = ${:key}
+ca-certificate = ${:d}/ca.pem
+cas-ca-certificate = ${:d}/cas-ca.pem
+crl = ${:d}/crl.pem
+
+[backend-client-login-csr]
+recipe = plone.recipe.command
+organization = {{ slapparameter_dict['cluster-identification'] }}
+organizational_unit = {{ instance_parameter['configuration.frontend-name'] }}
+command =
+{% if slapparameter_dict['backend-client-caucase-url'] %}
+  if [ ! -f ${:template-csr} ] && [ ! -f ${:key} ]  ; then
+    {{ parameter_dict['openssl'] }} req -new -sha256 \
+      -newkey rsa:2048 -nodes -keyout ${:key} \
+      -subj "/O=${:organization}/OU=${:organizational_unit}" \
+      -out ${:template-csr}
+  fi
+{% endif %}
+  test -f ${:key} && test -f ${:template-csr}
+update-command = ${:command}
+template-csr = ${backend-client-login-config:template-csr}
+key = ${backend-client-login-config:key}
+stop-on-error = True
+
+{{ caucase.updater(
+     prefix='backend-client-caucase-updater',
+     buildout_bin_directory=parameter_dict['bin_directory'],
+     updater_path='${directory:service}/backend-client-login-certificate-caucase-updater',
+     url=slapparameter_dict['backend-client-caucase-url'],
+     data_dir='${directory:srv}/backend-client-caucase-updater',
+     crt_path='${backend-client-login-config:certificate}',
+     ca_path='${backend-client-login-config:ca-certificate}',
+     crl_path='${backend-client-login-config:crl}',
+     key_path='${backend-client-login-csr:key}',
+     template_csr='${backend-client-login-csr:template-csr}'
+)}}
+
 [dynamic-custom-personal-template-slave-list]
 < = jinja2-template-base
 template = {{ parameter_dict['template_slave_list'] }}
@@ -218,10 +269,11 @@ caddy_executable = {{ parameter_dict['caddy'] }}
 sixtunnel_executable = {{ parameter_dict['sixtunnel'] }}/bin/6tunnel
 organization = {{ slapparameter_dict['cluster-identification'] }}
 organizational-unit = {{ instance_parameter['configuration.frontend-name'] }}
+backend-client-caucase-url = {{ slapparameter_dict['backend-client-caucase-url'] }}
 extra-context =
     key caddy_configuration_directory caddy-directory:slave-configuration
-    key caddy_cached_configuration_directory caddy-directory:slave-with-cache-configuration
-    key slave_with_cache_configuration_directory caddy-directory:slave-with-cache-configuration
+    key backend_client_caucase_url :backend-client-caucase-url
+    import urlparse_module urlparse
     key caddy_executable :caddy_executable
     key http_port configuration:plain_http_port
     key https_port configuration:port
@@ -232,7 +284,6 @@ extra-context =
     key autocert caddy-directory:autocert
     key master_certificate caddy-configuration:master-certificate
     key caddy_log_directory caddy-directory:slave-log
-    key caddy_log_cache_direct_directory caddy-directory:slave-log-cache-direct
     key expose_csr_id_organization :organization
     key expose_csr_id_organizational_unit :organizational-unit
     key local_ipv4 :local_ipv4
@@ -240,7 +291,6 @@ extra-context =
     key global_ipv6 slap-network-information:global-ipv6
     key empty_template software-release-path:template-empty
     key template_default_slave_configuration software-release-path:template-default-slave-virtualhost
-    key template_cached_slave_configuration software-release-path:template-cached-slave-virtualhost
     key software_type :software_type
     key frontend_lazy_graceful_reload frontend-caddy-lazy-graceful:rendered
     key frontend_graceful_reload caddy-configuration:frontend-graceful-command
@@ -251,9 +301,6 @@ extra-context =
     key enable_http2_by_default configuration:enable-http2-by-default
     key global_disable_http2 configuration:global-disable-http2
     key ciphers configuration:ciphers
-    key request_timeout configuration:request-timeout
-    key proxy_try_duration configuration:proxy-try-duration
-    key proxy_try_interval configuration:proxy-try-interval
     key access_log caddy-configuration:access-log
     key error_log caddy-configuration:error-log
     key sixtunnel_executable :sixtunnel_executable
@@ -263,6 +310,11 @@ extra-context =
 # BBB: SlapOS Master non-zero knowledge BEGIN
     key apache_certificate apache-certificate:rendered
 # BBB: SlapOS Master non-zero knowledge END
+## backend haproxy
+    key template_backend_haproxy_configuration software-release-path:template-backend-haproxy-configuration
+    section backend_haproxy_configuration backend-haproxy-configuration
+## full configuration
+    section configuration configuration
 
 # Deploy Caddy Frontend with Jinja power
 [dynamic-caddy-frontend-template]
@@ -277,9 +329,6 @@ extra-context =
     key master_certificate caddy-configuration:master-certificate
     key access_log caddy-configuration:access-log
     key slave_configuration_directory caddy-directory:slave-configuration
-    key cached_port caddy-configuration:cache-through-port
-    key ssl_cached_port caddy-configuration:ssl-cache-through-port
-    key slave_with_cache_configuration_directory caddy-directory:slave-with-cache-configuration
     section frontend_configuration frontend-configuration
     key http_port configuration:plain_http_port
     key https_port configuration:port
@@ -332,11 +381,9 @@ command = ln -sf  {{ parameter_dict['template_not_found_html'] }} ${caddy-direct
 recipe = slapos.cookbook:mkdirectory
 document-root = ${directory:srv}/htdocs
 slave-configuration = ${directory:etc}/caddy-slave-conf.d/
-slave-with-cache-configuration = ${directory:etc}/caddy-slave-with-cache-conf.d/
 cache = ${directory:var}/cache
 mod-ssl = ${:cache}/httpd_mod_ssl
 slave-log = ${directory:log}/httpd
-slave-log-cache-direct = ${directory:log}/httpd-cache-direct
 autocert = ${directory:srv}/autocert
 master-autocert-dir = ${:autocert}/master-autocert
 custom-ssl-directory = ${:slave-configuration}/ssl
@@ -351,8 +398,6 @@ not-found-file = ${caddy-directory:document-root}/${not-found-html:filename}
 master-certificate = ${caddy-directory:master-autocert-dir}/master.pem
 # Communication with ATS
 cache-port = ${trafficserver-variable:input-port}
-cache-through-port = 26011
-ssl-cache-through-port = 26012
 
 # BBB: SlapOS Master non-zero knowledge BEGIN
 [get-self-signed-fallback-access]
@@ -404,9 +449,6 @@ reload-path = ${directory:etc-run}/trafficserver-reload
 local-ip = {{ instance_parameter['ipv4-random'] }}
 input-port = 23432
 hostname = ${configuration:frontend-name}
-remap = map /HTTPS/ http://{{ instance_parameter['ipv4-random'] }}:${caddy-configuration:ssl-cache-through-port}
-  map / http://{{ instance_parameter['ipv4-random'] }}:${caddy-configuration:cache-through-port}
-
 plugin-config =
 ip-allow-config = src_ip=0.0.0.0-255.255.255.255 action=ip_allow
 cache-path = ${trafficserver-directory:cache-path}
@@ -464,11 +506,18 @@ template = {{ parameter_dict['template_trafficserver_logging_config'] }}
 filename = logging.config
 
 [trafficserver-remap-config]
-< = trafficserver-jinja2-template-base
-template = {{ parameter_dict['template_empty'] }}
+<= trafficserver-jinja2-template-base
+{%- raw %}
+template = inline:
+  map /HTTPS/ http://{{ ipv4 }}:{{ https_port }}
+  map / http://{{ ipv4 }}:{{ http_port }}
+{%- endraw %}
+extra-context =
+  raw ipv4 {{ instance_parameter['ipv4-random'] }}
+  key https_port backend-haproxy-configuration:https-port
+  key http_port backend-haproxy-configuration:http-port
+
 filename = remap.config
-context =
-    key content trafficserver-variable:remap
 
 [trafficserver-plugin-config]
 < = trafficserver-jinja2-template-base
@@ -536,7 +585,7 @@ template = {{ parameter_dict['template_configuration_state_script'] }}
 rendered = ${directory:bin}/${:_buildout_section_name_}
 mode = 0700
 
-path_list = ${caddy-configuration:frontend-configuration} ${frontend-configuration:log-access-configuration} ${caddy-directory:slave-configuration}/*.conf ${caddy-directory:slave-with-cache-configuration}/*.conf ${caddy-directory:master-autocert-dir}/*.key ${caddy-directory:master-autocert-dir}/*.crt ${caddy-directory:master-autocert-dir}/*.pem ${caddy-directory:autocert}/*.pem ${caddy-directory:custom-ssl-directory}/*.proxy_ca_crt ${directory:bbb-ssl-dir}/*.crt
+path_list = ${caddy-configuration:frontend-configuration} ${frontend-configuration:log-access-configuration} ${caddy-directory:slave-configuration}/*.conf ${caddy-directory:master-autocert-dir}/*.key ${caddy-directory:master-autocert-dir}/*.crt ${caddy-directory:master-autocert-dir}/*.pem ${caddy-directory:autocert}/*.pem ${caddy-directory:custom-ssl-directory}/*.proxy_ca_crt ${directory:bbb-ssl-dir}/*.crt
 sha256sum = {{ parameter_dict['sha256sum'] }}
 
 extra-context =
@@ -568,9 +617,10 @@ template = {{ parameter_dict['template_validate_script'] }}
 rendered = ${directory:bin}/frontend-caddy-validate
 mode = 0700
 last_state_file = ${directory:run}/caddy_configuration_last_state
+validate_command = ${caddy-wrapper:rendered} -validate
 extra-context =
-    key wrapper caddy-wrapper:rendered
-    key caddy_configuration_state frontend-caddy-configuration-state-validate:rendered
+    key validate_command :validate_command
+    key configuration_state_command frontend-caddy-configuration-state-validate:rendered
     key last_state_file :last_state_file
 
 [frontend-caddy-lazy-graceful]
@@ -631,19 +681,142 @@ name = caddy_frontend_ipv6_http.py
 config-hostname = {{ instance_parameter['ipv6-random'] }}
 config-port = ${configuration:plain_http_port}
 
-[promise-caddy-frontend-cached]
+[promise-backend-haproxy-http]
 <= monitor-promise-base
 module = check_port_listening
-name = caddy_cached.py
+name = backend_haproxy_http.py
 config-hostname = {{ instance_parameter['ipv4-random'] }}
-config-port = ${caddy-configuration:cache-through-port}
+config-port = ${backend-haproxy-configuration:http-port}
 
-[promise-caddy-frontend-ssl-cached]
+[promise-backend-haproxy-https]
 <= monitor-promise-base
 module = check_port_listening
-name = caddy_ssl_cached.py
+name = backend_haproxy_https.py
 config-hostname = {{ instance_parameter['ipv4-random'] }}
-config-port = ${caddy-configuration:ssl-cache-through-port}
+config-port = ${backend-haproxy-configuration:https-port}
+
+[backend-haproxy-configuration]
+file = ${directory:etc}/backend-haproxy.cfg
+pid-file = ${directory:run}/backend-haproxy.pid
+log-socket = ${backend-haproxy-rsyslogd:log-socket}
+graceful-command = ${backend-haproxy-validate:rendered} && kill -USR2 $(cat ${:pid-file})
+http-port = ${configuration:backend-haproxy-http-port}
+https-port = ${configuration:backend-haproxy-https-port}
+# Caucase related configuration
+caucase-url = {{ slapparameter_dict['backend-client-caucase-url'] }}
+ca-certificate = ${backend-client-login-config:ca-certificate}
+certificate = ${backend-client-login-config:certificate}
+cas-ca-certificate = ${backend-client-login-config:cas-ca-certificate}
+csr = ${backend-client-caucase-updater-csr:csr}
+crl = ${backend-client-login-config:crl}
+
+[backend-haproxy]
+recipe = slapos.cookbook:wrapper
+command-line = {{ parameter_dict['haproxy_executable'] }} -f ${backend-haproxy-configuration:file}
+wrapper-path = ${directory:service}/backend-haproxy
+hash-existing-files = ${buildout:directory}/software_release/buildout.cfg
+
+[logrotate-entry-backend-haproxy]
+<= logrotate-entry-base
+name = backend-haproxy
+log = ${backend-haproxy-rsyslogd-configuration:log-file}
+rotate-num = 30
+post = kill -HUP $(cat ${backend-haproxy-rsyslogd-configuration:pid-file})
+
+[backend-haproxy-configuration-state]
+<= jinja2-template-base
+template = {{ parameter_dict['template_configuration_state_script'] }}
+rendered = ${directory:bin}/${:_buildout_section_name_}
+mode = 0700
+
+path_list = ${backend-haproxy-configuration:file} ${backend-client-login-config:certificate}
+sha256sum = {{ parameter_dict['sha256sum'] }}
+
+extra-context =
+    key path_list :path_list
+    key sha256sum :sha256sum
+    key signature_file :signature_file
+
+[backend-haproxy-configuration-state-graceful]
+<= backend-haproxy-configuration-state
+signature_file = ${directory:run}/backend_haproxy_graceful_configuration_state_signature
+
+[backend-haproxy-configuration-state-validate]
+<= backend-haproxy-configuration-state
+signature_file = ${directory:run}/backend_haproxy_validate_configuration_state_signature
+
+[backend-haproxy-graceful]
+< = jinja2-template-base
+template = {{ parameter_dict['template_graceful_script'] }}
+rendered = ${directory:etc-run}/backend-haproxy-safe-graceful
+mode = 0700
+
+extra-context =
+    key graceful_reload_command backend-haproxy-configuration:graceful-command
+    key caddy_configuration_state backend-haproxy-configuration-state-graceful:rendered
+
+[backend-haproxy-validate]
+<= jinja2-template-base
+template = {{ parameter_dict['template_validate_script'] }}
+rendered = ${directory:bin}/backend-haproxy-validate
+mode = 0700
+last_state_file = ${directory:run}/backend_haproxy_configuration_last_state
+validate_command = {{ parameter_dict['haproxy_executable'] }} -f ${backend-haproxy-configuration:file} -c
+extra-context =
+    key validate_command :validate_command
+    key configuration_state_command backend-haproxy-configuration-state-validate:rendered
+    key last_state_file :last_state_file
+
+[backend-haproxy-lazy-graceful]
+< = jinja2-template-base
+template = {{ parameter_dict['template_caddy_lazy_script_call'] }}
+rendered = ${directory:bin}/backend-haproxy-lazy-graceful
+mode = 0700
+pid-file = ${directory:run}/backend-haproxy-lazy-graceful.pid
+wait_time = 60
+extra-context =
+    key pid_file :pid-file
+    key wait_time :wait_time
+    key lazy_command backend-haproxy-configuration:graceful-command
+
+[promise-backend-haproxy-configuration]
+<= monitor-promise-base
+module = validate_frontend_configuration
+name = backend-haproxy-configuration.py
+config-verification-script = ${promise-backend-haproxy-configuration-helper:rendered}
+
+[promise-backend-haproxy-configuration-helper]
+< = jinja2-template-base
+template = {{ parameter_dict['template_empty'] }}
+rendered = ${directory:bin}/backend-haproxy-read-last-configuration-state
+mode = 0700
+content =
+  #!/bin/sh
+  exit `cat ${backend-haproxy-validate:last_state_file}`
+context =
+    key content :content
+
+[backend-haproxy-rsyslogd-configuration]
+<= jinja2-template-base
+template = {{ parameter_dict['template_rsyslogd_conf'] }}
+rendered = ${directory:etc}/backend-haproxy-rsyslogd.conf
+# Note: log-socket shall be backend-haproxy-rsyslogd.sock to refer the part name,
+#       but it results with socket path limit
+log-socket = ${directory:run}/bhlog.sck
+log-file = ${directory:log}/backend-haproxy.log
+pid-file = ${directory:run}/backend-haproxy-rsyslogd.pid
+extra-context =
+  key socket :log-socket
+  key log_file :log-file
+  key spool_directory directory:backend-haproxy-rsyslogd-spool
+
+[backend-haproxy-rsyslogd]
+recipe = slapos.cookbook:wrapper
+command-line = {{ parameter_dict['rsyslogd_executable'] }} -i ${backend-haproxy-rsyslogd-configuration:pid-file} -n -f ${backend-haproxy-rsyslogd-configuration:rendered}
+wrapper-path = ${directory:service}/backend-haproxy-rsyslogd
+hash-existing-files = ${buildout:directory}/software_release/buildout.cfg
+log-socket = ${backend-haproxy-rsyslogd-configuration:log-socket}
+log-file = ${backend-haproxy-rsyslogd-configuration:log-file}
 
 #######
 # Monitoring sections

software/caddy-frontend/instance-apache-replicate.cfg.in

 {% if slap_software_type in software_type %}
+{% set aibcc_enabled = True %}
 {% import "caucase" as caucase with context %}
 {#- SERVER_POLLUTED_KEY_LIST is a list of keys which comes from various SlapOS Master implementations, which mix request and publish keys on each slave information -#}
 {%- set SERVER_POLLUTED_KEY_LIST = ['connection-parameter-hash', 'timestamp', 'slave_title', 'slap_software_type'] -%}
 {%- set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
 {%- set GOOD_CIPHER_LIST = ['ECDHE-ECDSA-AES256-GCM-SHA384', 'ECDHE-RSA-AES256-GCM-SHA384', 'ECDHE-ECDSA-AES128-GCM-SHA256', 'ECDHE-RSA-AES128-GCM-SHA256', 'ECDHE-ECDSA-WITH-CHACHA20-POLY1305', 'ECDHE-RSA-WITH-CHACHA20-POLY1305', 'ECDHE-RSA-AES256-CBC-SHA', 'ECDHE-RSA-AES128-CBC-SHA', 'ECDHE-ECDSA-AES256-CBC-SHA', 'ECDHE-ECDSA-AES128-CBC-SHA', 'RSA-AES256-CBC-SHA', 'RSA-AES128-CBC-SHA', 'ECDHE-RSA-3DES-EDE-CBC-SHA', 'RSA-3DES-EDE-CBC-SHA'] %}
 {% set aikc_enabled = slapparameter_dict.get('automatic-internal-kedifa-caucase-csr', 'true').lower() in TRUE_VALUES %}
+{% set aibcc_enabled = slapparameter_dict.get('automatic-internal-backend-client-caucase-csr', 'true').lower() in TRUE_VALUES %}
 {# Ports 8401, 8402 and 8410+1..N are reserved for monitor ports on various partitions #}
 {% set master_partition_monitor_monitor_httpd_port = 8401 %}
 {% set kedifa_partition_monitor_httpd_port = 8402 %}
 {% set frontend_monitor_httpd_base_port = 8410 %}
+{% set caucase_host = '[' ~ instance_parameter['ipv6-random'] ~ ']' %}
+{% set caucase_netloc = caucase_host ~ ':' ~ instance_parameter['configuration.caucase_backend_client_port'] %}
+{% set caucase_url = 'http://' ~ caucase_netloc %}
 [jinja2-template-base]
 recipe = slapos.recipe.template:jinja2
 rendered = ${buildout:directory}/${:filename}
@@ -64,6 +69,7 @@ context =
 {%     endif %}
 {%   endfor %}
 {%   do config_dict.__setitem__('monitor-httpd-port', frontend_monitor_httpd_base_port + i) %}
+{%   do config_dict.__setitem__('backend-client-caucase-url', caucase_url) %}
 {%   do frontend_list.append(frontend_name) %}
 {%   do frontend_section_list.append(request_section_title) %}
 {%   do part_list.append(request_section_title) %}
@@ -131,16 +137,16 @@ context =
 {%     endfor %}
 {%     do slave.__setitem__('server-alias', ' '.join(slave_server_alias_unclashed)) %}
 {%   endif %}
-{%   if slave.get('url') %}
-{%     if subprocess_module.call([caddy_backend_url_validator, '' ~ slave['url']]) == 1 %}
-{%       do slave_error_list.append('slave url %r invalid' % (slave['url'],)) %}
-{%     endif %}
-{%   endif %}
-{%   if slave.get('https-url') %}
-{%     if subprocess_module.call([caddy_backend_url_validator, '' ~ slave['https-url']]) == 1 %}
-{%       do slave_error_list.append('slave https-url %r invalid' % (slave['https-url'],)) %}
+{%   for url_key in ['url', 'https-url'] %}
+{%     if url_key in slave %}
+{%       set url = (slave[url_key] or '').strip() %}
+{%       if subprocess_module.call([caddy_backend_url_validator, url]) == 1 or not validators.url(url) %}
+{%         do slave_error_list.append('slave %s %r invalid' % (url_key, url)) %}
+{%       elif url != slave[url_key] %}
+{%         do slave_warning_list.append('slave %s %r has been converted to %r' % (url_key, slave[url_key], url)) %}
+{%       endif %}
 {%     endif %}
-{%   endif %}
+{%   endfor %}
 {%   if 'ssl_proxy_ca_crt' in slave %}
 {%     set ssl_proxy_ca_crt = slave.get('ssl_proxy_ca_crt', '') %}
 {%     set check_popen = popen([parameter_dict['openssl'], 'x509', '-noout']) %}
@@ -213,7 +219,7 @@ software-url = {{ slapparameter_dict.pop(frontend_software_url_key) }}
 software-url = ${slap-connection:software-release-url}
 {% endif %}
 software-type = {{frontend_type}}
-return = private-ipv4 public-ipv4 slave-instance-information-list monitor-base-url csr_id-url csr_id-certificate
+return = private-ipv4 public-ipv4 slave-instance-information-list monitor-base-url backend-client-csr_id-url csr_id-url csr_id-certificate
 
 {% for section, frontend_request in request_dict.iteritems() %}
 {%   set state = frontend_request.get('state', '') %}
@@ -225,6 +231,7 @@ state = {{ state }}
 {%   endif %}
 config-slave-kedifa-information = ${request-kedifa:connection-slave-kedifa-information}
 config-kedifa-caucase-url = ${request-kedifa:connection-caucase-url}
+config-backend-client-caucase-url = {{ caucase_url }}
 config-master-key-download-url = ${request-kedifa:connection-master-key-download-url}
 config-cluster-identification = {{ cluster_identification }}
 {#   Do not send additional parameters for destroyed nodes #}
@@ -259,6 +266,7 @@ domain = {{ slapparameter_dict.get('domain') }}
 slave-amount = {{ slave_instance_list | length }}
 accepted-slave-amount = {{ authorized_slave_list | length }}
 rejected-slave-amount = {{ rejected_slave_dict | length }}
+backend-client-caucase-url = {{ caucase_url }}
 {# sort_keys are important in order to avoid shuffling parameters on each run #}
 rejected-slave-dict = {{ dumps(json_module.dumps(rejected_slave_title_dict, sort_keys=True)) }}
 rejected-slave-promise-url = ${rejected-slave-promise:config-url}
@@ -273,13 +281,24 @@ warning-list = {{ dumps(json_module.dumps(warning_list, sort_keys=True)) }}
 {# sort_keys are important in order to avoid shuffling parameters on each run #}
 warning-slave-dict = {{ dumps(json_module.dumps(warning_slave_dict, sort_keys=True)) }}
 {% endif %}
+{% if not aikc_enabled or not aibcc_enabled %}
+{% for frontend in frontend_list %}
+  {% set section_part = '${request-' + frontend %}
+{{ frontend }}-csr_id-certificate = {{ section_part }}:connection-csr_id-certificate}
+{% endfor %}
+{% endif %}
 {% if not aikc_enabled %}
 kedifa-csr_id-url = ${request-kedifa:connection-csr_id-url}
 kedifa-csr_id-certificate = ${request-kedifa:connection-csr_id-certificate}
 {% for frontend in frontend_list %}
   {% set section_part = '${request-' + frontend %}
 {{ frontend }}-csr_id-url = {{ section_part }}:connection-csr_id-url}
-{{ frontend }}-csr_id-certificate = {{ section_part }}:connection-csr_id-certificate}
+{% endfor %}
+{% endif %}
+{% if not aibcc_enabled %}
+{% for frontend in frontend_list %}
+  {% set section_part = '${request-' + frontend %}
+{{ frontend }}-backend-client-csr_id-url = {{ section_part }}:connection-backend-client-csr_id-url}
 {% endfor %}
 {% endif %}
 
@@ -371,12 +390,17 @@ kedifa = ${request-kedifa:connection-monitor-base-url}
 {{ frontend }} = {{ '${' + frontend + ':connection-monitor-base-url}' }}
 {% endfor %}
 
-{% if aikc_enabled %}
 [directory]
 recipe = slapos.cookbook:mkdirectory
-
 bin = ${buildout:directory}/bin/
 srv = ${buildout:directory}/srv/
+backup = ${:srv}/backup
+# CAUCASE directories
+caucased = ${:srv}/caucased
+backup-caucased = ${:backup}/caucased
+
+{% if aikc_enabled %}
+[directory]
 aikc = ${:srv}/aikc
 
 [aikc-config]
@@ -505,7 +529,140 @@ command =
   ${aikc-{{ csr }}-wrapper:rendered}
 update-command = ${:command}
 {% endfor %}
-{% endif %}
+{% endif %} {# if aikc_enabled #}
+
+{% if aibcc_enabled %}
+[directory]
+aibcc = ${:srv}/aibcc
+
+[aibcc-config]
+caucase-url = {{ caucase_url }}
+
+csr = ${directory:aibcc}/csr.pem
+key = ${directory:aibcc}/key.pem
+ca-certificate = ${directory:aibcc}/cas-ca-certificate.pem
+crl = ${directory:aibcc}/crl.pem
+user-ca-certificate = ${directory:aibcc}/user-ca-certificate.pem
+user-crl = ${directory:aibcc}/user-crl.pem
+user-created = ${directory:aibcc}/user-created
+csr_id = ${directory:aibcc}/csr_id
+
+[aibcc-user-csr]
+recipe = plone.recipe.command
+organization = {{ cluster_identification }}
+organizational_unit = Automatic Sign Backend Client Caucase CSR
+command =
+  if [ ! -f ${:csr} ] && [ ! -f ${:key} ]  ; then
+    {{ parameter_dict['openssl'] }} req -new -sha256 \
+      -newkey rsa:2048 -nodes -keyout ${:key} \
+      -subj "/O=${:organization}/OU=${:organizational_unit}" \
+      -out ${:csr}
+  fi
+update-command = ${:command}
+csr = ${aibcc-config:csr}
+key = ${aibcc-config:key}
+stop-on-error = True
+
+
+[aibcc-caucase-wrapper]
+{# jinja2 instead of wrapper is used with context to remove py'u' #}
+recipe = slapos.recipe.template:jinja2
+context =
+  key caucase_url aibcc-config:caucase-url
+template = inline:#!{{ parameter_dict['dash'] }}/bin/dash
+  exec {{ parameter_dict['bin_directory'] }}/caucase \
+{# raw block to use context #}
+{% raw %}
+  --ca-url {{ caucase_url }} \
+{% endraw %}
+  --ca-crt ${aibcc-config:ca-certificate} \
+  --user-ca-crt ${aibcc-config:user-ca-certificate} \
+  --user-crl ${aibcc-config:user-crl} \
+  --crl ${aibcc-config:crl} \
+  "$@"
+
+rendered = ${directory:bin}/aibcc-caucase-wrapper
+mode = 0700
+
+{% do part_list.append('aibcc-create-user') %}
+[aibcc-create-user]
+recipe = plone.recipe.command
+# the caucase for this part is provided in this profile, so we can't fail
+# as otherwise caucase will never be started...
+stop-on-error = False
+update-command = ${:command}
+command =
+  if ! [ -f ${aibcc-config:user-created} ] ; then
+    ${aibcc-caucase-wrapper:rendered} --mode user --send-csr ${aibcc-user-csr:csr} > ${aibcc-config:csr_id} || exit 1
+    cut -d ' ' -f 1 ${aibcc-config:csr_id} || exit 1
+    csr_id=`cut -d ' ' -f 1 ${aibcc-config:csr_id}`
+    sleep 1
+    ${aibcc-caucase-wrapper:rendered} --mode user --get-crt $csr_id ${aibcc-config:key} || exit 1
+    touch ${aibcc-config:user-created}
+  fi
+
+{% do part_list.append('aibcc-user-caucase-updater') %}
+{% do part_list.append('aibcc-user-caucase-updater-promise') %}
+{{ caucase.updater(
+     prefix='aibcc-user-caucase-updater',
+     buildout_bin_directory=parameter_dict['bin_directory'],
+     updater_path='${directory:service}/aibcc-user-caucase-updater',
+     url='${aibcc-config:caucase-url}',
+     data_dir='${directory:srv}/caucase-updater',
+     crt_path='${aibcc-config:key}',
+     ca_path='${aibcc-config:user-ca-certificate}',
+     crl_path='${aibcc-config:user-crl}',
+     key_path='${aibcc-config:key}',
+     mode='user',
+)}}
+
+[aibcc-check-certificate]
+recipe = slapos.recipe.template:jinja2
+rendered = ${directory:bin}/aibcc-check-certificate
+template = inline:
+  import sys
+  import ssl
+  import urlparse
+  certificate = sys.argv[2]
+  parsed = urlparse.urlparse(sys.argv[1])
+  got_certificate = ssl.get_server_certificate((parsed.hostname, parsed.port))
+  sys.exit(0) if certificate.strip() == got_certificate.strip() else sys.exit(1)
+
+{% for csr in frontend_list %}
+[aibcc-{{ csr }}-wrapper]
+{# jinja2 instead of wrapper is used with context to remove py'u' #}
+recipe = slapos.recipe.template:jinja2
+context =
+  key csr_id_url request-{{ csr }}:connection-backend-client-csr_id-url
+  key csr_id_certificate request-{{ csr }}:connection-csr_id-certificate
+template = inline:#!{{ parameter_dict['dash'] }}/bin/dash
+  test -f ${directory:aibcc}/{{ csr }}-done && exit 0
+  ${buildout:executable} ${aibcc-check-certificate:rendered} \
+{# raw block to use context #}
+{% raw %}
+  {{ csr_id_url }} \
+  """{{ csr_id_certificate }}"""
+{% endraw %}
+  if [ $? = 0 ]; then
+    csr_id=`{{ parameter_dict['curl'] }}/bin/curl -s -k -g \
+{% raw %}
+  {{ csr_id_url }} \
+{% endraw %}
+  ` || exit 1
+    ${aibcc-caucase-wrapper:rendered} --user-key ${aibcc-config:key} --sign-csr $csr_id && touch ${directory:aibcc}/{{ csr }}-done
+  fi
+rendered = ${directory:bin}/aibcc-{{ csr }}-wrapper
+mode = 0700
+
+{% do part_list.append('aibcc-%s' % (csr,)) %}
+[aibcc-{{ csr }}]
+recipe = plone.recipe.command
+stop-on-error = True
+command =
+  ${aibcc-{{ csr }}-wrapper:rendered}
+update-command = ${:command}
+{% endfor %}
+{% endif %} {# if aibcc_enabled #}
 
 [rejected-slave-json]
 recipe = slapos.recipe.template:jinja2
@@ -599,6 +756,20 @@ config-filename = ${rejected-slave-json:rendered}
 config-state = empty
 config-url = ${rejected-slave-publish:url}
 
+[caucased-backend-client]
+hash-existing-files = ${buildout:directory}/software_release/buildout.cfg
+{{  caucase.caucased(
+      prefix='caucased-backend-client',
+      buildout_bin_directory=parameter_dict['bin_directory'],
+      caucased_path='${directory:service}/caucased-backend-client',
+      backup_dir='${directory:backup-caucased}',
+      data_dir='${directory:caucased}',
+      netloc=caucase_netloc,
+      service_auto_approve_count=0,
+      user_auto_approve_count=1,
+      key_len=2048,
+)}}
+
 [buildout]
 extends =
   {{ common_profile }}
@@ -610,6 +781,8 @@ parts =
   request-kedifa
   rejected-slave-promise
   promise-rejected-slave-publish-ip-port
+  caucased-backend-client
+  caucased-backend-client-promise
 {% for part in part_list %}
 {{ '  %s' % part }}
 {% endfor %}

software/caddy-frontend/instance-caddy-input-schema.json

@@ -66,16 +66,16 @@
       "title": "Test Verification URL",
       "type": "string"
     },
-    "proxy-try-duration": {
+    "backend-connect-timeout": {
       "default": 5,
-      "description": "A time during which Caddy will try to establish connection with a backend. Setting it to 0 will result with immediate return of 502 EOF error to the browser, even if it would be possible to (re)connect to the backend during few moments. More info in https://caddyserver.com/docs/proxy try_durtion.",
-      "title": "Duration in seconds of trying a backend",
+      "description": "Time in seconds for establishing connection to the backend.",
+      "title": "Timeout for backend connection (seconds)",
       "type": "integer"
     },
-    "proxy-try-interval": {
-      "default": 250,
-      "description": "How often Caddy will try to establish connection with a backend during proxy-try-duration. More info in https://caddyserver.com/docs/proxy try_interval",
-      "title": "Interval in milliseconds of tries during proxy-try-duration",
+    "backend-connect-retries": {
+      "default": 3,
+      "description": "Amount of retries to connect to the backend. The amount of backend-connect-timeout*backend-connect-retries seconds will be spent to connect to the backend.",
+      "title": "Amount of retries to connect to the backend.",
       "type": "integer"
     },
     "automatic-internal-kedifa-caucase-csr": {
@@ -88,6 +88,16 @@
       "title": "Automatic Internal KeDiFa's Caucase CSR",
       "type": "string"
     },
+    "automatic-internal-backend-client-caucase-csr": {
+      "default": "true",
+      "description": "Automatically signs CSRs sent to Backend Client's caucase, based on csr_id and matching certificate.",
+      "enum": [
+        "true",
+        "false"
+      ],
+      "title": "Automatic Internal Backend Client's Caucase CSR",
+      "type": "string"
+    },
     "ciphers": {
       "description": "List of ciphers. Empty defaults to Caddy list of ciphers. See https://caddyserver.com/docs/tls for more information.",
       "title": "Ordered space separated list of ciphers",
@@ -98,6 +108,16 @@
       "description": "Timeout for HTTP requests.",
       "title": "HTTP Request timeout in seconds",
       "type": "integer"
+    },
+    "authenticate-to-backend": {
+      "default": "false",
+      "description": "If set to true the frontend certificate will be used as authentication certificate to the backend. Note: backend might have to know the frontend CA, available with 'backend-client-caucase-url'.",
+      "enum": [
+        "false",
+        "true"
+      ],
+      "title": "Authenticate to backend",
+      "type": "string"
     }
   },
   "title": "Input Parameters",

software/caddy-frontend/instance-output-schema.json

@@ -73,6 +73,10 @@
     "warning-list": {
       "description": "List of warning found during the request.",
       "type": "array"
+    },
+    "backend-client-caucase-url": {
+      "description": "URL to caucase used by authentication to the backend.",
+      "type": "string"
     }
   },
   "type": "object"

software/caddy-frontend/instance-slave-caddy-input-schema.json

@@ -195,10 +195,34 @@
       "title": "type:zope virtualhostroot-https-port",
       "type": "integer"
     },
+    "backend-connect-timeout": {
+      "description": "Time in seconds for establishing connection to the backend.",
+      "title": "Timeout for backend connection (seconds)",
+      "type": "integer"
+    },
+    "backend-connect-retries": {
+      "description": "Amount of retries to connect to the backend. The amount of backend-connect-timeout*backend-connect-retries seconds will be spent to connect to the backend.",
+      "title": "Amount of retries to connect to the backend.",
+      "type": "integer"
+    },
+    "request-timeout": {
+      "description": "Timeout for HTTP requests.",
+      "title": "HTTP Request timeout in seconds",
+      "type": "integer"
+    },
     "ciphers": {
       "description": "List of ciphers. Empty defaults to cluster list of ciphers, which by default are Caddy list of ciphers. See https://caddyserver.com/docs/tls for more information.",
       "title": "Ordered space separated list of ciphers",
       "type": "string"
+    },
+    "authenticate-to-backend": {
+      "description": "If set to true the frontend certificate will be used as authentication certificate to the backend. Note: backend might have to know the frontend CA, available with 'backend-client-caucase-url'.",
+      "enum": [
+        "false",
+        "true"
+      ],
+      "title": "Authenticate to backend",
+      "type": "string"
     }
   },
   "title": "Input Parameters",

software/caddy-frontend/instance-slave-output-schema.json

@@ -49,6 +49,10 @@
     "kedifa-caucase-url": {
       "description": "URL to caucase used by KeDiFa",
       "type": "string"
+    },
+    "backend-client-caucase-url": {
+      "description": "URL to caucase used by authentication to the backend.",
+      "type": "string"
     }
   },
   "type": "object"

software/caddy-frontend/instance.cfg.in

@@ -103,6 +103,7 @@ configuration.nginx_port = 9443
 configuration.kedifa_port = 7879
 # Warning: Caucase takes also cacuase_port+1
 configuration.caucase_port = 8890
+configuration.caucase_backend_client_port = 8990
 configuration.apache-key =
 configuration.apache-certificate =
 configuration.open-port = 80 443
@@ -117,5 +118,8 @@ configuration.ciphers =
 configuration.request-timeout = 600
 configuration.mpm-graceful-shutdown-timeout = 5
 configuration.frontend-name =
-configuration.proxy-try-duration = 5
-configuration.proxy-try-interval = 250
+configuration.backend-connect-timeout = 5
+configuration.backend-connect-retries = 3
+configuration.backend-haproxy-http-port = 21080
+configuration.backend-haproxy-https-port = 21443
+configuration.authenticate-to-backend = False

software/caddy-frontend/templates/Caddyfile.in

@@ -2,10 +2,8 @@
 
 import {{frontend_configuration.get('log-access-configuration')}}
 import {{ slave_configuration_directory }}/*.conf
-import {{ slave_with_cache_configuration_directory }}/*.conf
 
-{% for port in [https_port] %}
-:{{ port }} {
+:{{ https_port }} {
   tls {{ master_certificate }} {{ master_certificate }}
   bind {{ local_ipv4 }}
   status 404 /
@@ -17,10 +15,8 @@ import {{ slave_with_cache_configuration_directory }}/*.conf
     * {{ not_found_file }}
   }
 }
-{%- endfor %}
 
-{% for port in [http_port, cached_port, ssl_cached_port] %}
-:{{ port }} {
+:{{ http_port }} {
   bind {{ local_ipv4 }}
   status 404 /
   log / {{ access_log }} "{remote} - {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}" {
@@ -31,7 +27,6 @@ import {{ slave_with_cache_configuration_directory }}/*.conf
     * {{ not_found_file }}
   }
 }
-{%- endfor %}
 
 # Access to server-status Caddy-style
 https://[{{ global_ipv6 }}]:{{ https_port }}/server-status, https://{{ local_ipv4 }}:{{ https_port }}/server-status {

software/caddy-frontend/templates/apache-custom-slave-list.cfg.in

 {%- if software_type == slap_software_type %}
 {%- set kedifa_updater_mapping = [] %}
 {%- set cached_server_dict = {} %}
+{%- set backend_slave_list = [] %}
 {%- set part_list = [] %}
 {%- set cache_port = caddy_configuration.get('cache-port') %}
-{%- set cached_port = caddy_configuration.get('cache-through-port') %}
-{%- set ssl_cached_port = caddy_configuration.get('ssl-cache-through-port') %}
 {%- set cache_access = "http://%s:%s" % (local_ipv4, cache_port) %}
 {%- set ssl_cache_access = "http://%s:%s/HTTPS" % (local_ipv4, cache_port) %}
+{%- set backend_haproxy_http_url = 'http://%s:%s' % (local_ipv4, backend_haproxy_configuration['http-port']) %}
+{%- set backend_haproxy_https_url = 'http://%s:%s' % (local_ipv4, backend_haproxy_configuration['https-port']) %}
 {%- set TRUE_VALUES = ['y', 'yes', '1', 'true'] %}
 {%- set generic_instance_parameter_dict = { 'cache_access': cache_access, 'local_ipv4': local_ipv4, 'http_port': http_port, 'https_port': https_port} %}
 {%- set slave_log_dict = {} %}
@@ -43,11 +44,16 @@ create = true
 [slave-log-directory-dict]
 [slave-password]
 
-# empty section if no cached slaves are available
-[slave-log-cache-direct-directory-dict]
-
 {#- Loop thought slave list to set up slaves #}
+{%- set DEFAULT_PORT = {'http': 80, 'https': 443, '': None} %}
 {%- for slave_instance in slave_instance_list %}
+{#- prepare backend parameters #}
+{%- for key, prefix in [('url', 'http_backend'), ('https-url', 'https_backend')] %}
+{%-   set parsed = urlparse_module.urlparse(slave_instance.get(key, '').strip()) %}
+{%-   set info_dict = {'scheme': parsed.scheme, 'hostname': parsed.hostname, 'port': parsed.port or DEFAULT_PORT[parsed.scheme], 'path': parsed.path, 'fragment': parsed.fragment} %}
+{%-   do slave_instance.__setitem__(prefix, info_dict) %}
+{%- endfor %}
+{%-   do slave_instance.__setitem__('ssl_proxy_verify', ('' ~ slave_instance.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES) %}
 {#-   Manage ciphers #}
 {%-   set slave_ciphers = slave_instance.get('ciphers', '').strip().split() %}
 {%-   if slave_ciphers %}
@@ -56,10 +62,34 @@ create = true
 {%-     set slave_cipher_list = ciphers.strip() %}
 {%-   endif %}
 {%-   do slave_instance.__setitem__('cipher_list', slave_cipher_list) %}
+{#-   Manage common instance parameters #}
 {%-   set slave_type = slave_instance.get('type', '') %}
 {%-   set enable_cache = (('' ~ slave_instance.get('enable_cache', '')).lower() in TRUE_VALUES and slave_type != 'redirect') %}
 {%-   set slave_reference = slave_instance.get('slave_reference') %}
 {%-   set slave_kedifa = slave_kedifa_information.get(slave_reference) %}
+{#-   Setup backend URLs for front facing Caddy #}
+{%-   if slave_type == 'redirect' %}
+{%-     do slave_instance.__setitem__('backend-http-url', slave_instance.get('url', '').rstrip('/')) %}
+{%-     if slave_instance.get('https-url') %}
+{%-       do slave_instance.__setitem__('backend-https-url', slave_instance.get['https-url'].rstrip('/')) %}
+{%-     endif %}
+{%-   elif enable_cache %}
+{%-     if 'domain' in slave_instance %}
+{%-       if not slave_instance.get('custom_domain') %}
+{%-         do slave_instance.__setitem__('custom_domain', slave_instance.get('domain')) %}
+{%-       endif %}
+{%-     endif %}
+{%-     do slave_instance.__setitem__('backend-http-url', cache_access) %}
+{%-     if slave_instance.get('https-url') %}
+{%-       do slave_instance.__setitem__('backend-https-url', ssl_cache_access) %}
+{%-     endif %}
+{%-     do cached_server_dict.__setitem__(slave_reference, slave_configuration_section_name) %}
+{%-   else %}
+{%-     do slave_instance.__setitem__('backend-http-url', backend_haproxy_http_url) %}
+{%-     if slave_instance.get('https-url') %}
+{%-       do slave_instance.__setitem__('backend-https-url', backend_haproxy_https_url) %}
+{%-     endif %}
+{%-   endif %}
 {%-   if slave_kedifa %}
 {%-     set key_download_url = slave_kedifa.get('key-download-url') %}
 {%-   else %}
@@ -70,57 +100,39 @@ create = true
 {%-   set slave_publish_dict = {} %}
 {%-   set slave_configuration_section_name = 'slave-instance-%s-configuration' % slave_reference %}
 {%-   set slave_logrotate_section = slave_reference + "-logs" %}
-{%-   set slave_logrotate_cache_direct_section = slave_reference + "-cache-direct-logs" %}
 {%-   set slave_password_section = slave_reference + "-password" %}
 {%-   set slave_ln_section = slave_reference + "-ln" %}
 {#-   extend parts #}
 {%-   do part_list.extend([slave_ln_section]) %}
 {%-   do part_list.extend([slave_logrotate_section, slave_section_title]) %}
 {%-   set slave_log_folder = '${logrotate-directory:logrotate-backup}/' + slave_reference + "-logs" %}
-{%-   if enable_cache %}
-{%-     set slave_log_cache_direct_folder = '${logrotate-directory:logrotate-backup}/' + slave_logrotate_cache_direct_section %}
-{%-     do part_list.extend([slave_logrotate_cache_direct_section]) %}
-{%-   endif %}
 {#-   Pass HTTP2 switch #}
 {%-   do slave_instance.__setitem__('enable_http2_by_default', enable_http2_by_default) %}
 {%-   do slave_instance.__setitem__('global_disable_http2', global_disable_http2) %}
-{#-   Pass proxy_try_duration and proxy_try_interval #}
-{%-   do slave_instance.__setitem__('proxy_try_duration', proxy_try_duration) %}
-{%-   do slave_instance.__setitem__('proxy_try_interval', proxy_try_interval) %}
+{#-   Pass backend timeout values #}
+{%-   for key in ['backend-connect-timeout', 'backend-connect-retries', 'request-timeout', 'authenticate-to-backend'] %}
+{%-     if slave_instance.get(key, '') == '' %}
+{%-       do slave_instance.__setitem__(key, configuration[key]) %}
+{%-     endif %}
+{%-   endfor %}
+{%-   do slave_instance.__setitem__('authenticate-to-backend', ('' ~ slave_instance.get('authenticate-to-backend', '')).lower() in TRUE_VALUES) %}
 {#-   Set Up log files #}
 {%-   do slave_parameter_dict.__setitem__('access_log', '/'.join([caddy_log_directory, '%s_access_log' % slave_reference])) %}
 {%-   do slave_parameter_dict.__setitem__('error_log', '/'.join([caddy_log_directory, '%s_error_log' % slave_reference])) %}
 {%-   do slave_instance.__setitem__('access_log', slave_parameter_dict.get('access_log')) %}
 {%-   do slave_instance.__setitem__('error_log', slave_parameter_dict.get('error_log')) %}
-{%-   if enable_cache %}
-{%-     do slave_parameter_dict.__setitem__('access_log_cache_direct', '/'.join([caddy_log_cache_direct_directory, '%s_access_log' % slave_reference])) %}
-{%-     do slave_parameter_dict.__setitem__('error_log_cache_direct', '/'.join([caddy_log_cache_direct_directory, '%s_error_log' % slave_reference])) %}
-{%-     do slave_instance.__setitem__('access_log_cache_direct', slave_parameter_dict.get('access_log_cache_direct')) %}
-{%-     do slave_instance.__setitem__('error_log_cache_direct', slave_parameter_dict.get('error_log_cache_direct')) %}
-{%-   endif %}
 {#-   Add slave log directory to the slave log access dict #}
 {%-   do slave_log_dict.__setitem__(slave_reference, slave_log_folder) %}
 {%-   set slave_log_access_url = 'https://' + slave_reference.lower() + ':${'+ slave_password_section +':passwd}@[' + frontend_configuration.get('caddy-ipv6') + ']:' + frontend_configuration.get('caddy-https-port') + '/' + slave_reference.lower() + '/' %}
 {%-   do slave_publish_dict.__setitem__('log-access', slave_log_access_url) %}
 {%-   do slave_publish_dict.__setitem__('slave-reference', slave_reference) %}
 {%-   do slave_publish_dict.__setitem__('public-ipv4', public_ipv4) %}
+{%-   do slave_publish_dict.__setitem__('backend-client-caucase-url', backend_client_caucase_url) %}
 {#-   Set slave domain if none was defined #}
 {%-   if slave_instance.get('custom_domain', None) == None %}
 {%-     set domain_prefix = slave_instance.get('slave_reference').replace("-", "").replace("_", "").lower() %}
 {%-     do slave_instance.__setitem__('custom_domain', "%s.%s" % (domain_prefix, slapparameter_dict.get('domain'))) %}
 {%-   endif %}
-{%-   if enable_cache and 'url' in slave_instance %}
-{%-     if 'domain' in slave_instance %}
-{%-       if not slave_instance.get('custom_domain') %}
-{%-         do slave_instance.__setitem__('custom_domain', slave_instance.get('domain')) %}
-{%-       endif %}
-{%-     endif %}
-{%-     do slave_instance.__setitem__('backend_url', slave_instance.get('url')) %}
-{%-     do slave_instance.__setitem__('https_backend_url', slave_instance.get('https-url', slave_instance.get('url'))) %}
-{%-     do slave_instance.__setitem__('url', cache_access) %}
-{%-     do slave_instance.__setitem__('https-url', ssl_cache_access) %}
-{%-     do cached_server_dict.__setitem__(slave_reference, slave_configuration_section_name) %}
-{%-   endif %}
 {%-   do slave_publish_dict.__setitem__('domain', slave_instance.get('custom_domain')) %}
 {%-   do slave_publish_dict.__setitem__('url', "http://%s" % slave_instance.get('custom_domain')) %}
 {%-   do slave_publish_dict.__setitem__('site_url', "http://%s" % slave_instance.get('custom_domain')) %}
@@ -129,11 +141,6 @@ create = true
 [slave-log-directory-dict]
 {{slave_reference}} = {{ slave_log_folder }}
 
-{%-   if enable_cache %}
-[slave-log-cache-direct-directory-dict]
-{{slave_reference}}_cache_direct = {{ slave_log_cache_direct_folder }}
-{%-   endif %}
-
 [slave-password]
 {{ slave_reference }} = {{ '${' + slave_password_section + ':passwd}' }}
 
@@ -143,14 +150,6 @@ create = true
 name = ${:_buildout_section_name_}
 log = {{slave_parameter_dict.get('access_log')}} {{slave_parameter_dict.get('error_log')}}
 backup = {{ slave_log_folder }}
-{%-   if enable_cache %}
-
-[{{slave_logrotate_cache_direct_section}}]
-<= logrotate-entry-base
-name = ${:_buildout_section_name_}
-log = {{slave_parameter_dict.get('access_log_cache_direct')}} {{slave_parameter_dict.get('error_log_cache_direct')}}
-backup = {{ slave_log_cache_direct_folder }}
-{%-   endif %}
 
 {#-   integrate current logs inside #}
 
@@ -221,9 +220,6 @@ certificate = {{ certificate }}
 https_port = {{ dumps('' ~ https_port) }}
 http_port = {{ dumps('' ~ http_port) }}
 local_ipv4 = {{ dumps('' ~ local_ipv4) }}
-cached_port = {{ dumps('' ~ cached_port) }}
-ssl_cached_port = {{ ('' ~ ssl_cached_port) }}
-request_timeout = {{ ('' ~ request_timeout) }}
 {%-   for key, value in slave_instance.iteritems() %}
 {%-     if value is not none %}
 {{ key }} = {{ dumps('' ~ value) }}
@@ -282,6 +278,9 @@ recipe = slapos.cookbook:publish
 {%-   else %}
 {%-     do slave_instance_information_list.append(slave_publish_dict) %}
 {%-   endif %}
+{%-  if slave_type != 'redirect' %}
+{%-    do backend_slave_list.append(slave_instance) %}
+{%-  endif %}
 {%- endfor %} {# Slave iteration ends for slave_instance in slave_instance_list #}
 
 [slave-log-directories]
@@ -290,28 +289,9 @@ recipe = slapos.cookbook:mkdirectory
 
 {%- do part_list.append('slave-log-directories') %}
 
-[slave-log-cache-direct-directories]
-<= slave-log-cache-direct-directory-dict
-recipe = slapos.cookbook:mkdirectory
-
-{%- do part_list.append('slave-log-cache-direct-directories') %}
 {%- do part_list.append('caddy-log-access') %}
 {#- ############################################## #}
 {#- ## Prepare virtualhost for slaves using cache  #}
-{%- for slave_reference, slave_configuration_section_name in cached_server_dict.iteritems() %}
-{%-   set cached_slave_configuration_section_title = '%s-cached-virtualhost' % slave_reference %}
-{%-   do part_list.append(cached_slave_configuration_section_title) %}
-
-[{{ cached_slave_configuration_section_title }}]
-< = jinja2-template-base
-template = {{ template_cached_slave_configuration }}
-filename = {{ '%s.conf' % slave_reference }}
-rendered = {{ caddy_cached_configuration_directory }}/${:filename}
-extensions = jinja2.ext.do
-extra-context =
-    section slave_parameter {{ slave_configuration_section_name }}
-{{ '\n' }}
-{%- endfor %}
 {#- Define IPv6 to IPV4 tunneling #}
 [tunnel-6to4-base]
 recipe = slapos.cookbook:wrapper
@@ -331,16 +311,6 @@ ipv6-port = {{ http_port }}
 ipv4-port = {{ https_port }}
 ipv6-port = {{ https_port }}
 
-[tunnel-6to4-base-cached_port]
-<= tunnel-6to4-base
-ipv4-port = {{ cached_port }}
-ipv6-port = {{ cached_port }}
-
-[tunnel-6to4-base-ssl_cached_port]
-<= tunnel-6to4-base
-ipv4-port = {{ ssl_cached_port }}
-ipv6-port = {{ ssl_cached_port }}
-
 {#- Define log access #}
 
 [caddy-log-access-parameters]
@@ -376,6 +346,7 @@ slave-instance-information-list = {{ json_module.dumps(slave_instance_informatio
 {%- endif %}
 monitor-base-url = {{ monitor_base_url }}
 csr_id-url = https://[${expose-csr_id-configuration:ip}]:${expose-csr_id-configuration:port}/csr_id.txt
+backend-client-csr_id-url = https://[${expose-csr_id-configuration:ip}]:${expose-csr_id-configuration:port}/backend-haproxy-csr_id.txt
 csr_id-certificate = ${get-csr_id-certificate:certificate}
 
 [kedifa-updater]
@@ -414,6 +385,47 @@ rendered = ${:file}
 template = inline: # This file contain directives to serve directories with log files for shared instances, but no shared instances are defined yet.
 rendered = {{frontend_configuration.get('log-access-configuration')}}
 
+##<Backend haproxy>
+[backend-haproxy-configuration]
+< = jinja2-template-base
+template = {{ template_backend_haproxy_configuration }}
+rendered = ${backend-haproxy-config:file}
+backend_slave_list = {{ dumps(sorted(backend_slave_list)) }}
+extra-context =
+  key backend_slave_list :backend_slave_list
+  section configuration backend-haproxy-config
+
+[backend-haproxy-config]
+file = {{ backend_haproxy_configuration['file'] }}
+pid-file = {{ backend_haproxy_configuration['pid-file'] }}
+log-socket = {{ backend_haproxy_configuration['log-socket'] }}
+local-ipv4 = {{ dumps('' ~ local_ipv4) }}
+http-port = {{ ('' ~ backend_haproxy_configuration['http-port']) }}
+https-port = {{ ('' ~ backend_haproxy_configuration['https-port']) }}
+request-timeout = {{ dumps('' ~ configuration['request-timeout']) }}
+backend-connect-timeout = {{ dumps('' ~ configuration['backend-connect-timeout']) }}
+backend-connect-retries =  {{ dumps('' ~ configuration['backend-connect-retries']) }}
+certificate = {{ dumps('' ~ backend_haproxy_configuration['certificate']) }}
+
+[store-backend-haproxy-csr_id]
+recipe = plone.recipe.command
+
+csr_id_path = {{ directory['csr_id'] }}/backend-haproxy-csr_id.txt
+csr_work_path = {{ directory['tmp'] }}/${:_buildout_section_name_}
+
+stop-on-error = False
+update-command = ${:command}
+command =
+  {{ bin_directory }}/caucase \
+    --ca-url {{ backend_haproxy_configuration['caucase-url'] }} \
+    --ca-crt {{ backend_haproxy_configuration['cas-ca-certificate'] }} \
+    --crl {{ backend_haproxy_configuration['crl'] }} \
+    --mode service \
+    --send-csr {{ backend_haproxy_configuration['csr'] }} > ${:csr_work_path} && \
+  cut -d ' ' -f 1 ${:csr_work_path} > ${:csr_id_path}
+
+##<Backend haproxy>
+
 [buildout]
 extends =
   {{ common_profile }}
@@ -423,6 +435,7 @@ extends =
 parts +=
     kedifa-updater
     kedifa-updater-run
+    backend-haproxy-configuration
 {%- for part in part_list %}
 {{ '    %s' % part }}
 {%- endfor %}
@@ -432,8 +445,6 @@ parts +=
     publish-caddy-information
     tunnel-6to4-base-http_port
     tunnel-6to4-base-https_port
-    tunnel-6to4-base-cached_port
-    tunnel-6to4-base-ssl_cached_port
     expose-csr_id
     promise-expose-csr_id-ip-port
 
@@ -496,7 +507,9 @@ config-hostname = ${expose-csr_id-configuration:ip}
 config-port = ${expose-csr_id-configuration:port}
 
 [expose-csr_id]
-depends = ${store-csr_id:command}
+depends =
+  ${store-csr_id:command}
+  ${store-backend-haproxy-csr_id:command}
 recipe = slapos.cookbook:wrapper
 command-line = {{ caddy_executable }}
   -conf ${expose-csr_id-template:rendered}

software/caddy-frontend/templates/backend-haproxy.cfg.in

+global
+  pidfile {{ configuration['pid-file'] }}
+  # master-worker is compatible with foreground with process management
+  master-worker
+
+log {{ configuration['log-socket'] }} local0
+defaults
+  log global
+  mode http
+  option httplog
+  timeout queue 60s
+  timeout server {{ configuration['request-timeout'] }}s
+  timeout client {{ configuration['request-timeout'] }}s
+  timeout connect {{ configuration['backend-connect-timeout'] }}s
+  retries {{ configuration['backend-connect-retries'] }}
+  # tentative below https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#option%20httpclose
+  option httpclose
+
+{%- macro frontend_entry(slave_instance, scheme) %}
+{%-   set host_list = (slave_instance.get('server-alias') or  '').split() %}
+{%-   if slave_instance.get('custom_domain') not in host_list %}
+{%-     do host_list.append(slave_instance.get('custom_domain')) %}
+{%-   endif %}
+{%-   for host in host_list %}
+{%-     if host.startswith('*.') %}
+{#-     hdr_sub has to be used, as anyway something.example.com shall match *.example.com[:port], with optional port #}
+  acl is_{{ slave_instance['slave_reference'] }} hdr_sub(host) -i {{ host[2:] }}
+{%-     else %}
+  acl is_{{ slave_instance['slave_reference'] }} hdr_dom(host) -i {{ host }}
+{%-     endif %}
+{%-   endfor %}
+  use_backend {{ slave_instance['slave_reference'] }}-{{ scheme }} if is_{{ slave_instance['slave_reference'] }}
+{%- endmacro %}
+
+frontend http-backend
+  bind {{ configuration['local-ipv4'] }}:{{ configuration['http-port'] }}
+{%- for slave_instance in backend_slave_list %}
+{{ frontend_entry(slave_instance, 'http') }}
+{%- endfor %}
+
+frontend https-backend
+  bind {{ configuration['local-ipv4'] }}:{{ configuration['https-port'] }}
+{%- for slave_instance in backend_slave_list %}
+{{ frontend_entry(slave_instance, 'https') }}
+{%- endfor %}
+
+{%- for slave_instance in backend_slave_list %}
+{%-   for (scheme, prefix) in [('http', 'http_backend'), ('https', 'https_backend')] %}
+{%-     set info_dict = slave_instance[prefix] %}
+{%-     if info_dict['scheme'] == 'https' %}
+{%-       set ssl = [] %}
+{%-       if slave_instance['authenticate-to-backend'] %}
+{%-         set ssl = ['crt %s' % (configuration['certificate'],)] %}
+{%-       endif %}
+{%-       do ssl.append('ssl verify') %}
+{%-       set path_to_ssl_proxy_ca_crt = slave_instance.get('path_to_ssl_proxy_ca_crt') %}
+{%-       if slave_instance['ssl_proxy_verify'] %}
+{%-         if path_to_ssl_proxy_ca_crt %}
+{%-           do ssl.append('required ca-file %s' % (path_to_ssl_proxy_ca_crt,)) %}
+{%-         else %}
+{#-         Backend SSL shall be verified, but not CA provided, disallow connection #}
+{#-         Simply dropping hostname from the dict will result with ignoring it... #}
+{%-         do info_dict.__setitem__('hostname', '') %}
+{%-         endif %}
+{%-       else %}
+{%-         do ssl.append('none') %}
+{%-       endif %}
+{%-       set ssl = ' '.join(ssl) %}
+{%-     else %}
+{%-       set ssl = '' %}
+{%-     endif %}
+
+backend {{ slave_instance['slave_reference'] }}-{{ scheme }}
+{%-     set hostname = info_dict['hostname'] %}
+{%-     set port = info_dict['port'] %}
+{%-     set path = info_dict['path'].rstrip('/') %}
+{%-     if hostname and port %}
+  timeout server {{ slave_instance['request-timeout'] }}s
+  timeout connect {{ slave_instance['backend-connect-timeout'] }}s
+  retries {{ slave_instance['backend-connect-retries'] }}
+  server backend {{ hostname }}:{{ port }} {{ ssl }}
+{%-       if path %}
+  http-request set-path {{ path }}%[path]
+{%-       endif %}
+{%-     endif %}
+{%-   endfor %}
+{%- endfor %}

software/caddy-frontend/templates/cached-virtualhost.conf.in

-{%- set TRUE_VALUES = ['y', 'yes', '1', 'true'] %}
-{%- set server_alias_list =  slave_parameter.get('server-alias', '').split() %}
-{%- set ssl_proxy_verify = ('' ~ slave_parameter.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES %}
-{%- set host_list = [] %}
-{%- for host in [slave_parameter.get('custom_domain')] + server_alias_list %}
-{%-   if host not in host_list %}
-{%-     do host_list.append(host) %}
-{%-   endif %}
-{%- endfor %}
-{%- set http_backend_host_list = [] %}
-{%- set https_backend_host_list = [] %}
-{%- for host in host_list %}
-{%-   do http_backend_host_list.append('http://%s:%s' % (host, slave_parameter['cached_port'])) %}
-{%-   do https_backend_host_list.append('http://%s:%s' % (host, slave_parameter['ssl_cached_port'])) %}
-{%- endfor %}
-
-# SSL-disabled backends
-{{ http_backend_host_list|join(', ') }} {
-  bind {{ slave_parameter['local_ipv4'] }}
-# Rewrite part
-  proxy / {{ slave_parameter.get('backend_url', '') }} {
-    try_duration {{ slave_parameter['proxy_try_duration'] }}s
-    try_interval {{ slave_parameter['proxy_try_interval'] }}ms
-
-    header_upstream Host {host}
-{#    header_upstream -X-Forwarded-For - caddy behaviour while removing and setting header is unstable, so for now original header has to be kept, even if in that case it comes from after ATS caddy itself #}
-    header_upstream X-Forwarded-For {>X-Forwarded-For-Real}
-    header_upstream -X-Forwarded-For-Real
-    timeout {{ slave_parameter['request_timeout'] }}s
-{%- if ssl_proxy_verify %}
-{%-   if 'path_to_ssl_proxy_ca_crt' in slave_parameter %}
-    ca_certificates {{ slave_parameter['path_to_ssl_proxy_ca_crt'] }}
-{%-   endif %}
-{%- else %}
-    insecure_skip_verify
-{%- endif %}
-  }
-  log / {{ slave_parameter.get('access_log_cache_direct') }} "{remote} - {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}" {
-    rotate_size 0
-  }
-
-  errors {{ slave_parameter.get('error_log_cache_direct') }} {
-    rotate_size 0
-  }
-}
-
-# SSL-enabled backends
-{{ https_backend_host_list|join(', ') }} {
-  bind {{ slave_parameter['local_ipv4'] }}
-  proxy / {{ slave_parameter.get('https_backend_url', '') }} {
-    try_duration {{ slave_parameter['proxy_try_duration'] }}s
-    try_interval {{ slave_parameter['proxy_try_interval'] }}ms
-    header_upstream Host {host}
-{#    header_upstream -X-Forwarded-For - caddy behaviour while removing and setting header is unstable, so for now original header has to be kept, even if in that case it comes from after ATS caddy itself #}
-    header_upstream X-Forwarded-For {>X-Forwarded-For-Real}
-    header_upstream -X-Forwarded-For-Real
-    timeout {{ slave_parameter['request_timeout'] }}s
-{%- if ssl_proxy_verify %}
-{%-   if 'path_to_ssl_proxy_ca_crt' in slave_parameter %}
-    ca_certificates {{ slave_parameter['path_to_ssl_proxy_ca_crt'] }}
-{%-   endif %}
-{%- else %}
-    insecure_skip_verify
-{%- endif %}
-  }
-  log / {{ slave_parameter.get('access_log_cache_direct') }} "{remote} - {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}" {
-    rotate_size 0
-  }
-
-  errors {{ slave_parameter.get('error_log_cache_direct') }} {
-    rotate_size 0
-  }
-}

software/caddy-frontend/templates/default-virtualhost.conf.in

@@ -9,7 +9,6 @@
 {%- endif %} {#- if prefer_gzip #}
 {%- set server_alias_list =  slave_parameter.get('server-alias', '').split() %}
 {%- set enable_h2 = slave_parameter['global_disable_http2'].lower() not in TRUE_VALUES and slave_parameter.get('enable-http2', slave_parameter['enable_http2_by_default']).lower() in TRUE_VALUES %}
-{%- set ssl_proxy_verify = slave_parameter.get('ssl-proxy-verify', '').lower() in TRUE_VALUES %}
 {%- set disabled_cookie_list =  slave_parameter.get('disabled-cookie-list', '').split() %}
 {%- set https_only = slave_parameter.get('https-only', 'true').lower() in TRUE_VALUES %}
 {%- set slave_type = slave_parameter.get('type', '') %}
@@ -41,31 +40,18 @@
 {%- endif %}
 
 {%- macro proxy_header() %}
-    try_duration {{ slave_parameter['proxy_try_duration'] }}s
-    try_interval {{ slave_parameter['proxy_try_interval'] }}ms
-    timeout {{ slave_parameter['request_timeout'] }}s
-{%-   if ssl_proxy_verify %}
-{%-     if 'path_to_ssl_proxy_ca_crt' in slave_parameter %}
-    ca_certificates {{ slave_parameter['path_to_ssl_proxy_ca_crt'] }}
-{%-     endif %} {#-     if 'path_to_ssl_proxy_ca_crt' in slave_parameter #}
-{%-   else %} {#-   if ssl_proxy_verify #}
-    insecure_skip_verify
-{%-   endif %} {#-   if ssl_proxy_verify #}
+    timeout {{ slave_parameter['request-timeout'] }}s
     # force reset of X-Forwarded-For
     header_upstream X-Forwarded-For {remote}
-{%-     if enable_cache %}
-    # provide a header for other components
-    header_upstream X-Forwarded-For-Real {remote}
-{%-     endif %}
 {%- endmacro %} {# proxy_header #}
 
 {%- for tls in [True, False] %}
 {%- if tls %}
-{%- set backend_url = slave_parameter.get('https-url', slave_parameter.get('url', '')).rstrip('/') %}
+{%-   set backend_url = slave_parameter.get('backend-https-url', slave_parameter.get('backend-http-url')) %}
 # SSL enabled hosts
 {{ https_host_list|join(', ') }} {
 {%- else %}
-{%- set backend_url = slave_parameter.get('url', '').rstrip('/') %}
+{%-   set backend_url = slave_parameter['backend-http-url'] %}
 # SSL-disabled hosts
 {{ http_host_list|join(', ') }} {
 {%- endif %}
@@ -178,11 +164,13 @@
 {%- endif %}
   } {# rewrite #}
 {%- endif %} {#- if prefer_gzip #}
-{%- elif slave_type ==  'redirect' and backend_url %} {#- if slave_type ==  'zope' and backend_url #}
+{%- elif slave_type == 'redirect' %}
+{%-   if backend_url %}
   # Redirect configuration
   redir 302 {
     /  {{ backend_url }}{rewrite_uri}
-  } {# redir #}
+  }
+{%-   endif %}
 {%- elif slave_type == 'notebook' %}
   proxy / {{ backend_url }} {
 {{ proxy_header() }}
@@ -204,6 +192,8 @@
 {{ proxy_header() }}
 {%-     if websocket_transparent %}
     transparent
+{%-     else %}
+    header_upstream Host {host}
 {%-     endif %}
   }
 {%-     for websocket_path in websocket_path_list %}
@@ -212,6 +202,8 @@
     websocket
 {%-       if websocket_transparent %}
     transparent
+{%-       else %}
+    header_upstream Host {host}
 {%-       endif %}
   }
 {%-     endfor %}
@@ -221,6 +213,8 @@
     websocket
 {%-   if websocket_transparent %}
     transparent
+{%-   else %}
+    header_upstream Host {host}
 {%-   endif %}
   }
 {%-   endif %}

software/caddy-frontend/templates/rsyslogd.conf.in

+module(
+  load="imuxsock"
+  SysSock.Name="{{ socket }}")
+
+# Use traditional timestamp format.
+# To enable high precision timestamps, comment out the following line.
+$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
+
+$FileCreateMode 0600
+$DirCreateMode 0700
+$Umask 0022
+
+$WorkDirectory {{ spool_directory }}
+
+*.* {{ log_file }}

software/caddy-frontend/templates/validate-script.sh.in

@@ -10,10 +10,10 @@ if [ -f $LAST_STATE_FILE ] ; then
   old_found=$(find $LAST_STATE_FILE -mmin +120 | wc -l)
 fi
 
-if [ "$old_found" -eq 1 ] || {{ caddy_configuration_state }} ; then
+if [ "$old_found" -eq 1 ] || {{ configuration_state_command }} ; then
   # do not catch errors during validation
   set +e
-  {{ wrapper }} -validate
+  {{ validate_command }}
   echo $? > $LAST_STATE_FILE
   set -e
 fi

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlave.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_dummy-cached_access_log
-T-2/var/log/httpd-cache-direct/_dummy-cached_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_dummy-cached_access_log
 T-2/var/log/httpd/_dummy-cached_error_log

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlave.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlave.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlave.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlaveGlobalDisableHttp2.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_dummy-cached_access_log
-T-2/var/log/httpd-cache-direct/_dummy-cached_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_dummy-cached_access_log
 T-2/var/log/httpd/_dummy-cached_error_log

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlaveGlobalDisableHttp2.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlaveGlobalDisableHttp2.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultDefaultSlaveGlobalDisableHttp2.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlave.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_dummy-cached_access_log
-T-2/var/log/httpd-cache-direct/_dummy-cached_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_dummy-cached_access_log
 T-2/var/log/httpd/_dummy-cached_error_log

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlave.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlave.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlave.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlaveGlobalDisableHttp2.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_dummy-cached_access_log
-T-2/var/log/httpd-cache-direct/_dummy-cached_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_dummy-cached_access_log
 T-2/var/log/httpd/_dummy-cached_error_log

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlaveGlobalDisableHttp2.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlaveGlobalDisableHttp2.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestEnableHttp2ByDefaultFalseSlaveGlobalDisableHttp2.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestMasterRequest.test_file_list_log-CADDY.txt

@@ -5,6 +5,7 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
 T-2/var/log/httpd-csr_id/expose-csr_id.log

software/caddy-frontend/test/test_data/test.TestMasterRequest.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestMasterRequest.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestMasterRequest.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestMasterRequestDomain.test_file_list_log-CADDY.txt

@@ -5,6 +5,7 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
 T-2/var/log/httpd-csr_id/expose-csr_id.log

software/caddy-frontend/test/test_data/test.TestMasterRequestDomain.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestMasterRequestDomain.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestMasterRequestDomain.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestRe6stVerificationUrlDefaultSlave.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_default_access_log
-T-2/var/log/httpd-cache-direct/_default_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_default_access_log
 T-2/var/log/httpd/_default_error_log

software/caddy-frontend/test/test_data/test.TestRe6stVerificationUrlDefaultSlave.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestRe6stVerificationUrlDefaultSlave.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestRe6stVerificationUrlDefaultSlave.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestSlave.test_file_list_log-CADDY.txt

@@ -5,27 +5,18 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-no-cache-request_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-no-cache-request_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-via-header_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-via-header_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify-unverified_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify-unverified_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache_custom_domain_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache_custom_domain_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache_server_alias_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache_server_alias_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_Url_access_log
 T-2/var/log/httpd/_Url_error_log
+T-2/var/log/httpd/_auth-to-backend-backend-ignore_access_log
+T-2/var/log/httpd/_auth-to-backend-backend-ignore_error_log
+T-2/var/log/httpd/_auth-to-backend-not-configured_access_log
+T-2/var/log/httpd/_auth-to-backend-not-configured_error_log
+T-2/var/log/httpd/_auth-to-backend_access_log
+T-2/var/log/httpd/_auth-to-backend_error_log
 T-2/var/log/httpd/_ciphers_access_log
 T-2/var/log/httpd/_ciphers_error_log
 T-2/var/log/httpd/_custom_domain_access_log
@@ -74,6 +65,8 @@ T-2/var/log/httpd/_prefer-gzip-encoding-to-backend_access_log
 T-2/var/log/httpd/_prefer-gzip-encoding-to-backend_error_log
 T-2/var/log/httpd/_server-alias-duplicated_access_log
 T-2/var/log/httpd/_server-alias-duplicated_error_log
+T-2/var/log/httpd/_server-alias-empty_access_log
+T-2/var/log/httpd/_server-alias-empty_error_log
 T-2/var/log/httpd/_server-alias-wildcard_access_log
 T-2/var/log/httpd/_server-alias-wildcard_error_log
 T-2/var/log/httpd/_server-alias_access_log
@@ -94,6 +87,8 @@ T-2/var/log/httpd/_ssl_ca_crt_only_access_log
 T-2/var/log/httpd/_ssl_ca_crt_only_error_log
 T-2/var/log/httpd/_type-notebook_access_log
 T-2/var/log/httpd/_type-notebook_error_log
+T-2/var/log/httpd/_type-redirect-custom_domain_access_log
+T-2/var/log/httpd/_type-redirect-custom_domain_error_log
 T-2/var/log/httpd/_type-redirect_access_log
 T-2/var/log/httpd/_type-redirect_error_log
 T-2/var/log/httpd/_type-websocket-websocket-path-list-websocket-transparent-false_access_log

software/caddy-frontend/test/test_data/test.TestSlave.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-_monitor-ipv4-test-ipv4-packet-list-test.py
 T-2/etc/plugin/check-_monitor-ipv6-test-ipv6-packet-list-test.py

software/caddy-frontend/test/test_data/test.TestSlave.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestSlave.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestSlaveCiphers.test_file_list_log-CADDY.txt

@@ -5,12 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_default_ciphers_access_log
-T-2/var/log/httpd-cache-direct/_default_ciphers_error_log
-T-2/var/log/httpd-cache-direct/_own_ciphers_access_log
-T-2/var/log/httpd-cache-direct/_own_ciphers_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_default_ciphers_access_log
 T-2/var/log/httpd/_default_ciphers_error_log

software/caddy-frontend/test/test_data/test.TestSlaveCiphers.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestSlaveCiphers.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestSlaveCiphers.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestSlaveGlobalDisableHttp2.test_file_list_log-CADDY.txt

@@ -5,27 +5,18 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-no-cache-request_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-no-cache-request_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-via-header_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-disable-via-header_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify-unverified_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify-unverified_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt-unverified_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache-ssl-proxy-verify_ssl_proxy_ca_crt_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache_custom_domain_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache_custom_domain_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache_error_log
-T-2/var/log/httpd-cache-direct/_enable_cache_server_alias_access_log
-T-2/var/log/httpd-cache-direct/_enable_cache_server_alias_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_Url_access_log
 T-2/var/log/httpd/_Url_error_log
+T-2/var/log/httpd/_auth-to-backend-backend-ignore_access_log
+T-2/var/log/httpd/_auth-to-backend-backend-ignore_error_log
+T-2/var/log/httpd/_auth-to-backend-not-configured_access_log
+T-2/var/log/httpd/_auth-to-backend-not-configured_error_log
+T-2/var/log/httpd/_auth-to-backend_access_log
+T-2/var/log/httpd/_auth-to-backend_error_log
 T-2/var/log/httpd/_ciphers_access_log
 T-2/var/log/httpd/_ciphers_error_log
 T-2/var/log/httpd/_custom_domain_access_log
@@ -74,6 +65,8 @@ T-2/var/log/httpd/_prefer-gzip-encoding-to-backend_access_log
 T-2/var/log/httpd/_prefer-gzip-encoding-to-backend_error_log
 T-2/var/log/httpd/_server-alias-duplicated_access_log
 T-2/var/log/httpd/_server-alias-duplicated_error_log
+T-2/var/log/httpd/_server-alias-empty_access_log
+T-2/var/log/httpd/_server-alias-empty_error_log
 T-2/var/log/httpd/_server-alias-wildcard_access_log
 T-2/var/log/httpd/_server-alias-wildcard_error_log
 T-2/var/log/httpd/_server-alias_access_log
@@ -94,6 +87,8 @@ T-2/var/log/httpd/_ssl_ca_crt_only_access_log
 T-2/var/log/httpd/_ssl_ca_crt_only_error_log
 T-2/var/log/httpd/_type-notebook_access_log
 T-2/var/log/httpd/_type-notebook_error_log
+T-2/var/log/httpd/_type-redirect-custom_domain_access_log
+T-2/var/log/httpd/_type-redirect-custom_domain_error_log
 T-2/var/log/httpd/_type-redirect_access_log
 T-2/var/log/httpd/_type-redirect_error_log
 T-2/var/log/httpd/_type-websocket-websocket-path-list-websocket-transparent-false_access_log

software/caddy-frontend/test/test_data/test.TestSlaveGlobalDisableHttp2.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-_monitor-ipv4-test-ipv4-packet-list-test.py
 T-2/etc/plugin/check-_monitor-ipv6-test-ipv6-packet-list-test.py

software/caddy-frontend/test/test_data/test.TestSlaveGlobalDisableHttp2.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestSlaveGlobalDisableHttp2.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibility.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_ssl_from_master_access_log
-T-2/var/log/httpd-cache-direct/_ssl_from_master_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_custom_domain_ssl_crt_ssl_key_access_log
 T-2/var/log/httpd/_custom_domain_ssl_crt_ssl_key_error_log

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibility.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibility.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibility.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityOverrideMaster.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_ssl_from_master_kedifa_overrides_master_certificate_access_log
-T-2/var/log/httpd-cache-direct/_ssl_from_master_kedifa_overrides_master_certificate_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_ssl_from_master_kedifa_overrides_master_certificate_access_log
 T-2/var/log/httpd/_ssl_from_master_kedifa_overrides_master_certificate_error_log

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityOverrideMaster.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityOverrideMaster.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityOverrideMaster.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityUpdate.test_file_list_log-CADDY.txt

@@ -5,10 +5,9 @@ T-1/var/log/expose-csr_id.log
 T-1/var/log/kedifa.log
 T-1/var/log/monitor-httpd-access.log
 T-1/var/log/monitor-httpd-error.log
+T-2/var/log/backend-haproxy.log
 T-2/var/log/frontend-access.log
 T-2/var/log/frontend-error.log
-T-2/var/log/httpd-cache-direct/_ssl_from_master_access_log
-T-2/var/log/httpd-cache-direct/_ssl_from_master_error_log
 T-2/var/log/httpd-csr_id/expose-csr_id.log
 T-2/var/log/httpd/_ssl_from_master_access_log
 T-2/var/log/httpd/_ssl_from_master_error_log

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityUpdate.test_file_list_plugin-CADDY.txt

 T-0/etc/plugin/__init__.py
+T-0/etc/plugin/aibcc-user-caucase-updater.py
 T-0/etc/plugin/aikc-user-caucase-updater.py
 T-0/etc/plugin/buildout-T-0-status.py
+T-0/etc/plugin/caucased-backend-client.py
 T-0/etc/plugin/check-free-disk-space.py
 T-0/etc/plugin/monitor-bootstrap-status.py
 T-0/etc/plugin/monitor-http-frontend.py
@@ -17,13 +19,15 @@ T-1/etc/plugin/monitor-bootstrap-status.py
 T-1/etc/plugin/monitor-http-frontend.py
 T-1/etc/plugin/monitor-httpd-listening-on-tcp.py
 T-2/etc/plugin/__init__.py
+T-2/etc/plugin/backend-client-caucase-updater.py
+T-2/etc/plugin/backend-haproxy-configuration.py
+T-2/etc/plugin/backend_haproxy_http.py
+T-2/etc/plugin/backend_haproxy_https.py
 T-2/etc/plugin/buildout-T-2-status.py
-T-2/etc/plugin/caddy_cached.py
 T-2/etc/plugin/caddy_frontend_ipv4_http.py
 T-2/etc/plugin/caddy_frontend_ipv4_https.py
 T-2/etc/plugin/caddy_frontend_ipv6_http.py
 T-2/etc/plugin/caddy_frontend_ipv6_https.py
-T-2/etc/plugin/caddy_ssl_cached.py
 T-2/etc/plugin/caucase-updater.py
 T-2/etc/plugin/check-free-disk-space.py
 T-2/etc/plugin/expose-csr_id-ip-port-listening.py

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityUpdate.test_file_list_run-CADDY.txt

 T-0/var/run/monitor-httpd.pid
 T-1/var/run/kedifa.pid
 T-1/var/run/monitor-httpd.pid
+T-2/var/run/backend-haproxy-rsyslogd.pid
+T-2/var/run/backend-haproxy.pid
+T-2/var/run/backend_haproxy_configuration_last_state
+T-2/var/run/backend_haproxy_graceful_configuration_state_signature
+T-2/var/run/bhlog.sck
 T-2/var/run/graceful_configuration_state_signature
 T-2/var/run/httpd.pid
 T-2/var/run/monitor-httpd.pid

software/caddy-frontend/test/test_data/test.TestSlaveSlapOSMasterCertificateCompatibilityUpdate.test_supervisor_state-CADDY.txt

+T-0:aibcc-user-caucase-updater-on-watch RUNNING
 T-0:aikc-user-caucase-updater-on-watch RUNNING
 T-0:bootstrap-monitor EXITED
+T-0:caucased-backend-client-{hash-generic}-on-watch RUNNING
 T-0:certificate_authority-{hash-generic}-on-watch RUNNING
 T-0:crond-{hash-generic}-on-watch RUNNING
 T-0:monitor-httpd-{hash-generic}-on-watch RUNNING
@@ -17,8 +19,10 @@ T-1:monitor-httpd-{hash-generic}-on-watch RUNNING
 T-1:monitor-httpd-graceful EXITED
 T-2:6tunnel-11080-{hash-generic}-on-watch RUNNING
 T-2:6tunnel-11443-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26011-{hash-generic}-on-watch RUNNING
-T-2:6tunnel-26012-{hash-generic}-on-watch RUNNING
+T-2:backend-client-login-certificate-caucase-updater-on-watch RUNNING
+T-2:backend-haproxy-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-rsyslogd-{hash-generic}-on-watch RUNNING
+T-2:backend-haproxy-safe-graceful EXITED
 T-2:bootstrap-monitor EXITED
 T-2:certificate_authority-{hash-generic}-on-watch RUNNING
 T-2:crond-{hash-generic}-on-watch RUNNING