Commit dbb00241 authored by Łukasz Nowak's avatar Łukasz Nowak

caddy-frontend: Copy of apache-frontend original

This will make it easier to track changes.
parent 99a88757
apache_frontend
===============
Frontend system using Apache, allowing to rewrite and proxy URLs like
myinstance.myfrontenddomainname.com to real IP/URL of myinstance.
apache_frontend works using the master instance / slave instance design.
It means that a single main instance of Apache will be used to act as frontend
for many slaves.
Software type
=============
Apache frontend is available in 3 software types:
* default : The standard way to use the apache frontend configuring everything with a few given parameters
* custom-personal : This software type allow each slave to edit its apache configuration file
About frontend replication
===========================
Slaves of the root instance are sent as a parameter to requested frontends which will process them. The only difference is that they will then return the « would-be published information » to the root instance instead of publishing it. The root instance will then do a synthesis and publish the information to its slaves. The replicate instance only use 5 type of parameters for itself and will transmit the rest to requested frontends.
These parameters are :
* "-frontend-type" : the type to deploy frontends with. (default to 2)
* "-frontend-quantity" : The quantity of frontends to request (default to "default")
* "-frontend-i-state": The state of frontend i
* "-frontend-config-i-foo": Frontend i will be requested with parameter foo
* "-frontend-software-release-url": Software release to be used for frontends, default to the current software release
* "-sla-i-foo" : where "i" is the number of the concerned frontend (between 1 and "-frontend-quantity") and "foo" a sla parameter.
ex:
<parameter id="-frontend-quantity">3</parameter>
<parameter id="-frontend-type">custom-personal</parameter>
<parameter id="-frontend-2-state">stopped</parameter>
<parameter id="-sla-3-computer_guid">COMP-1234</parameter>
<parameter id="-frontend-software-release-url">http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/tags/slapos-0.183:/software/apache-frontend/software.cfg</parameter>
will request the third frontend on COMP-1234. All frontends will be of software type "custom-personal". The second frontend will be requested with the state stopped
Note: the way slaves are transformed to a parameter avoid modifying more than 3 lines in the frontend logic.
Important NOTE: The way you ask for slave to a replicate frontend is the same as the one you would use for the software given in "-frontend-quantity". Do not forget to use "replicate" for software type. XXXXX So far it is not possible to do a simple request on a replicate frontend if you do not know the software_guid or other sla-parameter of the master instance. In fact we do not know yet the software type of the "requested" frontends. TO BE IMPLEMENTED
XXX Should be moved to specific JSON File
Extra-parameter per frontend with default :
ram-cache-size = 1G
disk-cache-size = 8G
How to deploy a frontend server
===============================
This is to deploy an entire frontend server with a public IPv4.
If you want to use an already deployed frontend to make your service available
via ipv4, switch to the "Example" parts.
First, you will need to request a "master" instance of Apache Frontend with:
* A "domain" parameter where the frontend will be available
* A "public-ipv4" parameter to state which public IPv4 will be used
like::
<?xml version='1.0' encoding='utf-8'?>
<instance>
<parameter id="domain">moulefrite.org</parameter>
<parameter id="public-ipv4">xxx.xxx.xxx.xxx</parameter>
</instance>
Then, it is possible to request many slave instances
(currently only from slapconsole, UI doesn't work yet)
of Apache Frontend, like::
instance = request(
software_release=apache_frontend,
partition_reference='frontend2',
shared=True,
partition_parameter_kw={"url":"https://[1:2:3:4]:1234/someresource"}
)
Those slave instances will be redirected to the "master" instance,
and you will see on the "master" instance the associated RewriteRules of
all slave instances.
Finally, the slave instance will be accessible from:
https://someidentifier.moulefrite.org.
About SSL
=========
Default and custom-personal software type can handle specific ssl for one slave instance.
IMPORTANT: One apache can not serve more than One specific SSL VirtualHost and be compatible with obsolete browser (i.e.: IE8). See http://wiki.apache.org/httpd/NameBasedSSLVHostsWithSNI
#How to have custom configuration in frontend server
#===================================================
#
#In your instance directory, you, as sysadmin, can directly edit two
#configuration files that won't be overwritten by SlapOS to customize your
#instance:
#
# * $PARTITION_PATH/srv/srv/apache-conf.d/apache_frontend.custom.conf
# * $PARTITION_PATH/srv/srv/apache-conf.d/apache_frontend.virtualhost.custom.conf
#
#The first one is included in the end of the main apache configuration file.
#The second one is included in the virtualhost of the main apache configuration file.
#
#SlapOS will jsut create those two files for you, then completely forget them.
#
#Note: make sure that the UNIX user of the instance has read access to those
#files if you edit them.
Instance Parameters
===================
Master Instance Parameters
--------------------------
The parameters for instances are described at `instance-apache-input-schema.json <instance-apache-input-schema.json>`_.
Here some additional informations about the parameters listed, below:
domain
~~~~~~
name of the domain to be used (example: mydomain.com). Subdomains of this
domain will be used for the slave instances (example:
instance12345.mydomain.com). It is then recommended to add a wildcard in DNS
for the subdomains of the chosen domain like::
*.mydomain.com. IN A 123.123.123.123
Using the IP given by the Master Instance.
"domain" is a mandatory Parameter.
public-ipv4
~~~~~~~~~~~
Public ipv4 of the frontend (the one Apache will be indirectly listening to)
port
~~~~
Port used by Apache. Optional parameter, defaults to 4443.
plain_http_port
~~~~~~~~~~~~~~~
Port used by apache to serve plain http (only used to redirect to https).
Optional parameter, defaults to 8080.
Slave Instance Parameters
-------------------------
The parameters for instances are described at `instance-slave-apache-input-schema.json <instance-slave-apache-input-schema.json>`_.
Here some additional informations about the parameters listed, below:
path
~~~~
Only used if type is "zope".
Will append the specified path to the "VirtualHostRoot" of the zope's
VirtualHostMonster.
"path" is an optional parameter, ignored if not specified.
Example of value: "/erp5/web_site_module/hosting/"
apache_custom_https
~~~~~~~~~~~~~~~~~~~
Raw apache configuration in python template format (i.e. write "%%" for one "%") for the slave listening to the https port. Its content will be templatified in order to access functionalities such as cache access, ssl certificates... The list is available above.
NOTE: If you want to use the cache, use the apache option "ProxyPreserveHost On"
apache_custom_http
~~~~~~~~~~~~~~~~~~
Raw apache configuration in python template format (i.e. write "%%" for one "%") for the slave listening to the http port. Its content will be templatified in order to access functionalities such as cache access, ssl certificates... The list is available above
NOTE: If you want to use the cache, use the apache option "ProxyPreserveHost On"
url
~~~
Necesarry to activate cache. url of backend to use.
"url" is an optional parameter.
Example: http://mybackend.com/myresource
domain
~~~~~~
Necesarry to activate cache. The frontend will be accessible from this domain.
"domain" is an optional parameter.
Example: www.mycustomdomain.com
enable_cache
~~~~~~~~~~~~
Necesarry to activate cache.
"enable_cache" is an optional parameter.
ssl_key, ssl_crt, ssl_ca_crt
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SSL certificates of the slave.
They are optional.
Functionalities for apache configuration:
In the slave apache configuration you can use parameters that will be replaced during instanciation. They should be entered as python templates parameters ex:" %(parameter)s"
* cache_access : url of the cache. Should replace backend url in configuration to use the cache
* error_log : path of the slave error log in order to log in a deferenciated file.
* error_log : path of the slave access log in order to log in a deferenciated file.
* ssl_key, ssl_crt, ssl_ca_crt, ssl_crs : path of the certificates given in slave instance parameters
Examples
========
Here are some example of how to make your SlapOS service available through
an already deployed frontend.
Simple Example (default)
------------------------
Request slave frontend instance so that https://[1:2:3:4:5:6:7:8]:1234 will be
redirected and accessible from the proxy::
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
}
)
Zope Example (default)
----------------------
Request slave frontend instance using a Zope backend so that
https://[1:2:3:4:5:6:7:8]:1234 will be redirected and accessible from the
proxy::
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"type":"zope",
}
)
Advanced example
-----------------
Request slave frontend instance using a Zope backend, with Varnish activated,
listening to a custom domain and redirecting to /erp5/ so that
https://[1:2:3:4:5:6:7:8]:1234/erp5/ will be redirected and accessible from
the proxy::
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"enable_cache":"true",
"type":"zope",
"path":"/erp5",
"domain":"mycustomdomain.com",
}
)
Simple Example
---------------
Request slave frontend instance so that https://[1:2:3:4:5:6:7:8]:1234 will be
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
software_type="custom-personal",
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"apache_custom_https":'
ServerName www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLEngine on
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
RewriteRule ^/(.*) https://[1:2:3:4:5:6:7:8]:1234/$1 [L,P]',
"apache_custom_http":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteRule ^/(.*) https://[1:2:3:4:5:6:7:8]:1234/$1 [L,P],
}
)
Simple Cache Example
--------------------
Request slave frontend instance so that https://[1:2:3:4:5:6:7:8]:1234 will be
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
software_type="custom-personal",
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"domain": "www.example.org",
"enable_cache": "True",
"apache_custom_https":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLEngine on
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
RewriteRule ^/(.*) %(cache_access)s/$1 [L,P]',
"apache_custom_http":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteRule ^/(.*) %(cache_access)s/$1 [L,P],
}
)
Advanced example
-----------------
Request slave frontend instance using custom apache configuration, willing to use cache and ssl certificates.
listening to a custom domain and redirecting to /erp5/ so that
https://[1:2:3:4:5:6:7:8]:1234/erp5/ will be redirected and accessible from
the proxy::
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
software_type="custom-personal",
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"enable_cache":"true",
"type":"zope",
"path":"/erp5",
"domain":"example.org",
"apache_custom_https":'
ServerName www.example.org
ServerAlias www.example.org
ServerAdmin example.org
SSLEngine on
SSLProxyEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
# Use personal ssl certificates
SSLCertificateFile %(ssl_crt)s
SSLCertificateKeyFile %(ssl_key)s
SSLCACertificateFile %(ssl_ca_crt)s
SSLCertificateChainFile %(ssl_ca_crt)s
# Configure personal logs
ErrorLog "%(error_log)s"
LogLevel info
LogFormat "%%h %%l %%{REMOTE_USER}i %%t \"%%r\" %%>s %%b \"%%{Referer}i\" \"%%{User-Agent}i\" %%D" combined
CustomLog "%(access_log)s" combined
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Redirect / to /index.html
RewriteRule ^/$ /index.html [R=302,L]
# Use cache
RewriteRule ^/(.*) %(cache_access)s/VirtualHostBase/https/www.example.org:443/erp5/VirtualHostRoot/$1 [L,P]',
"apache_custom_http":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Configure personal logs
ErrorLog "%(error_log)s"
LogLevel info
LogFormat "%%h %%l %%{REMOTE_USER}i %%t \"%%r\" %%>s %%b \"%%{Referer}i\" \"%%{User-Agent}i\" %%D" combined
CustomLog "%(access_log)s" combined
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteRule ^/(.*)$ https://%%{SERVER_NAME}%%{REQUEST_URI}',
"ssl_key":"-----BEGIN RSA PRIVATE KEY-----
XXXXXXX..........XXXXXXXXXXXXXXX
-----END RSA PRIVATE KEY-----",
"ssl_crt":'-----BEGIN CERTIFICATE-----
XXXXXXXXXXX.............XXXXXXXXXXXXXXXXXXX
-----END CERTIFICATE-----',
"ssl_ca_crt":'-----BEGIN CERTIFICATE-----
XXXXXXXXX...........XXXXXXXXXXXXXXXXX
-----END CERTIFICATE-----',
"ssl_csr":'-----BEGIN CERTIFICATE REQUEST-----
XXXXXXXXXXXXXXX.............XXXXXXXXXXXXXXXXXX
-----END CERTIFICATE REQUEST-----',
}
)
Notes
=====
It is not possible with slapos to listen to port <= 1024, because process are
not run as root.
Solution 1 (IPv4 only)
----------------------
It is a good idea then to go on the node where the instance is
and set some iptables rules like (if using default ports)::
iptables -t nat -A PREROUTING -p tcp -d {public_ipv4} --dport 443 -j DNAT --to-destination {listening_ipv4}:4443
iptables -t nat -A PREROUTING -p tcp -d {public_ipv4} --dport 80 -j DNAT --to-destination {listening_ipv4}:8080
Where {public ip} is the public IP of your server, or at least the LAN IP to where your NAT will forward to.
{listening ip} is the private ipv4 (like 10.0.34.123) that the instance is using and sending as connection parameter.
Solution 2 (IPv6 only)
----------------------
It is also possible to directly allow the service to listen on 80 and 443 ports using the following command:
setcap 'cap_net_bind_service=+ep' /opt/slapgrid/$APACHE_FRONTEND_SOFTWARE_RELEASE_MD5/parts/apache/bin/httpd
Then specify in the instance parameters "port" and "plain_http_port" to be 443 and 80, respectively.
# THIS IS NOT A BUILDOUT FILE, despite purposedly using a compatible syntax.
# The only allowed lines here are (regexes):
# - "^#" comments, copied verbatim
# - "^[" section beginings, copied verbatim
# - lines containing an "=" sign which must fit in the following categorie.
# - "^\s*filename\s*=\s*path\s*$" where "path" is relative to this file
# But avoid directories, they are not portable.
# Copied verbatim.
# - "^\s*hashtype\s*=.*" where "hashtype" is one of the values supported
# by the re-generation script.
# Re-generated.
# - other lines are copied verbatim
# Substitution (${...:...}), extension ([buildout] extends = ...) and
# section inheritance (< = ...) are NOT supported (but you should really
# not need these here).
[template]
filename = instance.cfg
md5sum = f686f765e55d1dce2e55a400f0714b3e
[template-apache-frontend]
filename = instance-apache-frontend.cfg
md5sum = b6a2c860ea1cd4bc9d185c7108c52d0a
[template-apache-replicate]
filename = instance-apache-replicate.cfg.in
md5sum = 9e76028df7e93d3e32982884d5dc0913
[template-slave-list]
filename = templates/apache-custom-slave-list.cfg.in
md5sum = 24e514ad6f15859229db46f24a8cd280
[template-slave-configuration]
filename = templates/custom-virtualhost.conf.in
md5sum = d103143e5d50682bd5ad43117d82e2fa
[template-replicate-publish-slave-information]
filename = templates/replicate-publish-slave-information.cfg.in
md5sum = 665e83d660c9b779249b2179d7ce4b4e
[template-apache-frontend-configuration]
filename = templates/apache.conf.in
md5sum = a56045e7b53ff00ab34d2a8f911fc1a1
[template-custom-slave-list]
filename = templates/apache-custom-slave-list.cfg.in
md5sum = 24e514ad6f15859229db46f24a8cd280
[template-not-found-html]
filename = templates/notfound.html
md5sum = f20d6c3d2d94fb685f8d26dfca1e822b
[template-default-virtualhost]
filename = templates/000.conf.in
md5sum = d98a01182f38868612948c87d5231428
[template-default-slave-virtualhost]
filename = templates/default-virtualhost.conf.in
md5sum = 7f38084af107034bedefba971abe165c
[template-cached-slave-virtualhost]
filename = templates/cached-virtualhost.conf.in
md5sum = 1a1a53d9ac4a1591c017d86850a94796
[template-log-access]
filename = templates/template-log-access.conf.in
md5sum = f85005b430978f3bd24ee7ce11b0e304
[template-empty]
filename = templates/empty.in
md5sum = c2314c3a9c3412a38d14b312d3df83c1
[template-wrapper]
filename = templates/wrapper.in
md5sum = 8cde04bfd0c0e9bd56744b988275cfd8
[template-trafficserver-records-config]
filename = templates/trafficserver/records.config.jinja2
md5sum = 84baef0a49c9a65e8f2d2ffdb8c1d39c
[template-trafficserver-storage-config]
filename = templates/trafficserver/storage.config.jinja2
md5sum = 117238225b3fc3c5b5be381815f44c67
[template-nginx-configuration]
filename = templates/nginx.cfg.in
md5sum = 18633ce55e53340efa1ba7693aac4152
[template-nginx-eventsource-slave-virtualhost]
filename = templates/nginx-eventsource-slave.conf.in
md5sum = a5186f666acb2f040ede04c91e60408f
[template-nginx-notebook-slave-virtualhost]
filename = templates/nginx-notebook-slave.conf.in
md5sum = 82d74a7f2aceb2b4a7acc6259291b7f2
[template-apache-lazy-script-call]
filename = templates/apache-lazy-script-call.sh.in
md5sum = ebe5d3d19923eb812a40019cb11276d8
[template-apache-graceful-script]
filename = templates/apache-graceful-script.sh.in
md5sum = 41299cc64200e7b8217fb9dec20bb8b9
[buildout]
extends =
buildout.hash.cfg
../../stack/slapos.cfg
../../component/git/buildout.cfg
../../component/dash/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/apache/buildout.cfg
../../component/gzip/buildout.cfg
../../component/stunnel/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
../../component/pycurl/buildout.cfg
../../component/python-cryptography/buildout.cfg
../../component/rdiff-backup/buildout.cfg
../../component/trafficserver/buildout.cfg
../../component/pycurl/buildout.cfg
../../component/nginx/buildout.cfg
../../stack/nodejs.cfg
# Monitoring stack (keep on bottom)
../../stack/monitor/buildout.cfg
parts +=
template
template-apache-frontend
template-apache-replicate
apache
apache-antiloris
stunnel
dcron
logrotate
rdiff-backup
nginx-push-stream-output
npm-modules
proxy-by-url
http-proxy
# Extent extra-eggs.
[extra-eggs]
eggs +=
websockify
erp5.util
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
output = ${buildout:directory}/template.cfg
mode = 0644
[template-apache-frontend]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-apache-frontend.cfg
output = ${buildout:directory}/template-apache-frontend.cfg
mode = 0644
[template-apache-replicate]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/instance-apache-replicate.cfg.in
mode = 0644
[download-template]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/${:filename}
mode = 640
[template-slave-list]
<=download-template
filename = apache-custom-slave-list.cfg.in
[template-slave-configuration]
<=download-template
filename = custom-virtualhost.conf.in
[template-replicate-publish-slave-information]
<=download-template
filename = replicate-publish-slave-information.cfg.in
[template-apache-frontend-configuration]
<=download-template
filename = apache.conf.in
[template-custom-slave-list]
<=download-template
filename = apache-default-slave-list.cfg.in
[template-not-found-html]
<=download-template
filename = notfound.html
[template-default-virtualhost]
<=download-template
filename = 000.conf.in
[template-default-slave-virtualhost]
<=download-template
filename = default-virtualhost.conf.in
[template-cached-slave-virtualhost]
<=download-template
filename = cached-virtualhost.conf.in
[template-log-access]
<=download-template
filename = template-log-access.conf.in
[template-empty]
<=download-template
filename = empty.in
[template-wrapper]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/templates/wrapper.in
output = ${buildout:directory}/template-wrapper.cfg
mode = 0644
[template-trafficserver-records-config]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/templates/trafficserver/${:filename}
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = records.config.jinja2
download-only = true
mode = 0644
[template-trafficserver-storage-config]
recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/templates/trafficserver/${:filename}
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = storage.config.jinja2
download-only = true
mode = 0644
# NGINX Configuration
[template-nginx-configuration]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/templates/nginx.cfg.in
output = ${buildout:directory}/template-nginx.cfg.in
mode = 0644
[template-apache-lazy-script-call]
<=download-template
filename = apache-lazy-script-call.sh.in
[template-apache-graceful-script]
<=download-template
filename = apache-graceful-script.sh.in
[template-nginx-eventsource-slave-virtualhost]
<=download-template
filename = nginx-eventsource-slave.conf.in
[template-nginx-notebook-slave-virtualhost]
<=download-template
filename = nginx-notebook-slave.conf.in
# Migrated from KVM recipe
[http-proxy]
# https://github.com/nodejitsu/node-http-proxy
recipe = slapos.recipe.build:download-unpacked
# use upstream when merged
url = https://lab.nexedi.com/nexedi/node-http-proxy/repository/archive.zip?ref=a5d3aff428ee8d840068b439f6ce121077f1144f
md5sum = 65602466066444c718215de41f546585
[proxy-by-url]
# https://github.com/dominictarr/proxy-by-url
recipe = slapos.recipe.build:download-unpacked
# use upstream when merged
url = https://lab.nexedi.com/nexedi/proxy-by-url/repository/archive.zip?ref=59fcb11a3e00c45b4b0362e76f29653abb313072
md5sum = c383e0c5ff31b56f7987466e8304c941
[npm-modules]
recipe = plone.recipe.command
destination = ${buildout:parts-directory}/${:_buildout_section_name_}
location = ${buildout:parts-directory}/${:_buildout_section_name_}
command =
export HOME=${:location};
rm -fr ${:destination} &&
mkdir -p ${:destination} &&
cd ${:destination} &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install colors@0.6.0-1 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install socket.io@0.8.7 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install socket.io-client@0.8.7 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install optimist@0.3.1 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install pkginfo@0.2.3
# Development profile of apache-frontend.
# Exactly the same as software.cfg, but fetch the slapos.cookbook
# from git repository instead of fetching stable version,
# allowing to play with bleeding edge environment.
# You'll need to run buildout twice for this profile.
[buildout]
extends =
# Extend in this order, otherwise "parts" will be taken from git profile
common.cfg
parts +=
slapos.toolbox-dev
[slapos.toolbox-dev]
recipe = zc.recipe.egg:develop
egg = slapos.toolbox
setup = ${slapos.toolbox-repository:location}
[buildout]
parts =
directory
configtest
logrotate
cron
cron-entry-logrotate
ca-frontend
certificate-authority
logrotate-entry-apache
logrotate-entry-nginx
apache-frontend
switch-apache-softwaretype
frontend-apache-graceful
frontend-nginx-graceful
dynamic-template-default-vh
not-found-html
promise-frontend-apache-configuration
promise-apache-frontend-v4-https
promise-apache-frontend-v4-http
promise-apache-frontend-v6-https
promise-apache-frontend-v6-http
promise-apache-frontend-cached
promise-apache-frontend-ssl-cached
promise-apache-is-process-older-than-dependency-set
promise-apache-mpm-watchdog
promise-nginx-frontend-v4-https
promise-nginx-frontend-v4-http
promise-nginx-frontend-v6-https
promise-nginx-frontend-v6-http
promise-nginx-configuration
promise-nginx-is-process-older-than-dependency-set
trafficserver-launcher
trafficserver-reload
trafficserver-configuration-directory
trafficserver-records-config
trafficserver-remap-config
trafficserver-plugin-config
trafficserver-storage-config
trafficserver-promise-listen-port
trafficserver-promise-cache-availability
## Nginx
nginx-frontend
## Monitor for apache
monitor-base
monitor-ats-cache-stats-wrapper
monitor-traffic-summary-last-stats-wrapper
monitor-apache-server-status-wrapper
monitor-verify-re6st-connectivity
extends = ${monitor-template:output}
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
# Create all needed directories
[directory]
recipe = slapos.cookbook:mkdirectory
bin = $${buildout:directory}/bin/
etc = $${buildout:directory}/etc/
srv = $${buildout:directory}/srv/
var = $${buildout:directory}/var/
template = $${buildout:directory}/template/
backup = $${:srv}/backup
log = $${:var}/log
run = $${:var}/run
service = $${:etc}/service
etc-run = $${:etc}/run
promise = $${:etc}/promise
logrotate-backup = $${:backup}/logrotate
logrotate-entries = $${:etc}/logrotate.d
cron-entries = $${:etc}/cron.d
crontabs = $${:etc}/crontabs
cronstamps = $${:etc}/cronstamps
ca-dir = $${:srv}/ssl
varnginx = $${:var}/nginx
[switch-apache-softwaretype]
recipe = slapos.cookbook:softwaretype
single-default = $${dynamic-custom-personal-template-slave-list:rendered}
single-custom-personal = $${dynamic-custom-personal-template-slave-list:rendered}
[instance-parameter]
# Fetches parameters defined in SlapOS Master for this instance.
# Always the same.
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
# Define default parameter(s) that will be used later, in case user didn't
# specify it
# All parameters are available through the configuration.XX syntax.
# All possible parameters should have a default.
configuration.domain = example.org
configuration.public-ipv4 =
configuration.port = 4443
configuration.plain_http_port = 8080
configuration.plain_nginx_port = 8081
configuration.nginx_port = 9443
configuration.server-admin = admin@example.com
configuration.apache_custom_https = ""
configuration.apache_custom_http = ""
configuration.apache-key =
configuration.apache-certificate =
configuration.apache-ca-certificate =
configuration.open-port = 80 443
configuration.extra_slave_instance_list =
configuration.disk-cache-size = 8G
configuration.ram-cache-size = 1G
configuration.trafficserver-autoconf-port = 8083
configuration.trafficserver-mgmt-port = 8084
configuration.re6st-verification-url = http://[2001:67c:1254:4::1]/index.html
[frontend-configuration]
template-log-access = ${template-log-access:target}
log-access-configuration = $${directory:etc}/apache-log-access.conf
apache-directory = ${apache:location}
apache-ipv6 = $${instance-parameter:ipv6-random}
apache-https-port = $${instance-parameter:configuration.port}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type instance-parameter:slap-software-type
key slapparameter_dict instance-parameter:configuration
section directory directory
$${:extra-context}
[dynamic-template-default-vh]
< = jinja2-template-base
template = ${template-default-virtualhost:target}
rendered = $${apache-directory:slave-configuration}/000.conf
extensions = jinja2.ext.do
extra-context =
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
[software-release-path]
template-empty = ${template-empty:target}
template-slave-configuration = ${template-slave-configuration:target}
template-default-slave-virtualhost = ${template-default-slave-virtualhost:target}
template-cached-slave-virtualhost = ${template-cached-slave-virtualhost:target}
template-nginx-eventsource-slave-virtualhost = ${template-nginx-eventsource-slave-virtualhost:target}
template-nginx-notebook-slave-virtualhost = ${template-nginx-notebook-slave-virtualhost:target}
apache-location = ${apache:location}
[dynamic-custom-personal-template-slave-list]
< = jinja2-template-base
template = ${template-slave-list:target}
filename = custom-personal-instance-slave-list.cfg
extensions = jinja2.ext.do
extra-context =
key apache_configuration_directory apache-directory:slave-configuration
key nginx_configuration_directory apache-directory:nginx-slave-configuration
key apache_cached_configuration_directory apache-directory:slave-with-cache-configuration
key slave_with_cache_configuration_directory apache-directory:slave-with-cache-configuration
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key nginx_http_port instance-parameter:configuration.plain_nginx_port
key nginx_https_port instance-parameter:configuration.nginx_port
key public_ipv4 instance-parameter:configuration.public-ipv4
key slave_instance_list instance-parameter:slave-instance-list
key extra_slave_instance_list instance-parameter:configuration.extra_slave_instance_list
key custom_ssl_directory apache-directory:vh-ssl
key apache_log_directory apache-directory:slave-log
key local_ipv4 instance-parameter:ipv4-random
key global_ipv6 slap-network-information:global-ipv6
key varnginx directory:varnginx
key empty_template software-release-path:template-empty
key template_custom_slave_configuration software-release-path:template-slave-configuration
key template_default_slave_configuration software-release-path:template-default-slave-virtualhost
key template_cached_slave_configuration software-release-path:template-cached-slave-virtualhost
key template_eventsource_slave_configuration software-release-path:template-nginx-eventsource-slave-virtualhost
key template_notebook_slave_configuration software-release-path:template-nginx-notebook-slave-virtualhost
raw software_type single-custom-personal
key frontend_lazy_graceful_reload frontend-apache-lazy-graceful:rendered
section logrotate_dict logrotate
section frontend_configuration frontend-configuration
section apache_configuration apache-configuration
section nginx_configuration nginx-configuration
key monitor_base_url monitor-instance-parameter:monitor-base-url
key promise_directory monitor-directory:promises
key report_directory monitor-directory:reports
raw bin_directory ${buildout:bin-directory}
[dynamic-virtualhost-template-slave]
<= jinja2-template-base
template = ${template-slave-configuration:target}
rendered = $${directory:template}/slave-virtualhost.conf.in
extensions = jinja2.ext.do
extra-context =
key https_port instance-parameter:configuration.port
key http_port instance-parameter:configuration.plain_http_port
key apache_custom_https instance-parameter:configuration.apache_custom_https
key apache_custom_http instance-parameter:configuration.apache_custom_http
# Deploy Apache Frontend (new way, no recipe, jinja power)
[dynamic-apache-frontend-template]
< = jinja2-template-base
template = ${template-apache-frontend-configuration:target}
rendered = $${apache-configuration:frontend-configuration}
extra-context =
key httpd_home software-release-path:apache-location
key httpd_mod_ssl_cache_directory apache-directory:mod-ssl
key domain instance-parameter:configuration.domain
key document_root apache-directory:document-root
key instance_home buildout:directory
key ipv4_addr instance-parameter:ipv4-random
key ipv6_addr instance-parameter:ipv6-random
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key server_admin instance-parameter:configuration.server-admin
key protected_path apache-configuration:protected-path
key access_control_string apache-configuration:access-control-string
key login_certificate ca-frontend:cert-file
key login_key ca-frontend:key-file
key login_ca_crt ca-custom-frontend:rendered
key ca_dir certificate-authority:ca-dir
key ca_crl certificate-authority:ca-crl
key access_log apache-configuration:access-log
key error_log apache-configuration:error-log
key pid_file apache-configuration:pid-file
key slave_configuration_directory apache-directory:slave-configuration
key cached_port apache-configuration:cache-through-port
key ssl_cached_port apache-configuration:ssl-cache-through-port
key slave_with_cache_configuration_directory apache-directory:slave-with-cache-configuration
section frontend_configuration frontend-configuration
[apache-frontend]
recipe = slapos.cookbook:wrapper
command-line = ${apache:location}/bin/httpd -f $${dynamic-apache-frontend-template:rendered} -DFOREGROUND
wrapper-path = $${directory:service}/frontend_apache
wait-for-files =
$${ca-frontend:cert-file}
$${ca-frontend:key-file}
[not-found-html]
recipe = slapos.cookbook:symbolic.link
target-directory = $${apache-directory:document-root}
link-binary =
${template-not-found-html:target}
[apache-directory]
recipe = slapos.cookbook:mkdirectory
document-root = $${directory:srv}/htdocs
slave-configuration = $${directory:etc}/apache-slave-conf.d/
slave-with-cache-configuration = $${directory:etc}/apache-slave-with-cache-conf.d/
cache = $${directory:var}/cache
mod-ssl = $${:cache}/httpd_mod_ssl
vh-ssl = $${:slave-configuration}/ssl
slave-log = $${directory:log}/httpd
nginx-slave-configuration = $${directory:etc}/nginx-slave-conf.d/
[apache-configuration]
frontend-configuration = $${directory:etc}/apache_frontend.conf
access-log = $${directory:log}/frontend-apache-access.log
error-log = $${directory:log}/frontend-apache-error.log
pid-file = $${directory:run}/httpd.pid
protected-path = /
access-control-string = none
frontend-configuration-verification = ${apache:location}/bin/httpd -Sf $${:frontend-configuration} > /dev/null
frontend-graceful-command = $${:frontend-configuration-verification}; if [ $? -eq 0 ]; then kill -USR1 $(cat $${:pid-file}); fi
# Comunication with ats
cache-port = $${trafficserver-variable:input-port}
cache-through-port = 26011
ssl-cache-through-port = 26012
# Create wrapper for "apachectl conftest" in bin
[configtest]
recipe = slapos.cookbook:wrapper
command-line = ${apache:location}/bin/httpd -f $${directory:etc}/apache_frontend.conf -t
wrapper-path = $${directory:bin}/apache-configtest
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = ${openssl:location}/bin/openssl
ca-dir = $${directory:ca-dir}
requests-directory = $${cadirectory:requests}
wrapper = $${directory:service}/certificate_authority
ca-private = $${cadirectory:private}
ca-certs = $${cadirectory:certs}
ca-newcerts = $${cadirectory:newcerts}
ca-crl = $${cadirectory:crl}
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:ca-dir}/requests/
private = $${directory:ca-dir}/private/
certs = $${directory:ca-dir}/certs/
newcerts = $${directory:ca-dir}/newcerts/
crl = $${directory:ca-dir}/crl/
[ca-frontend]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${cadirectory:certs}/apache_frontend.key
cert-file = $${cadirectory:certs}/apache_frontend.crt
executable = $${directory:service}/frontend_apache
wrapper = $${directory:service}/frontend_apache
key-content = $${instance-parameter:configuration.apache-key}
cert-content = $${instance-parameter:configuration.apache-certificate}
# Put domain name
name = $${instance-parameter:configuration.domain}
[ca-custom-frontend]
< = jinja2-template-base
template = ${template-empty:target}
rendered = $${cadirectory:certs}/apache_frontend.ca.crt
extra-context =
key content instance-parameter:configuration.apache-ca-certificate
[cron]
recipe = slapos.cookbook:cron
dcrond-binary = ${dcron:location}/sbin/crond
cron-entries = $${directory:cron-entries}
crontabs = $${directory:crontabs}
cronstamps = $${directory:cronstamps}
catcher = $${cron-simplelogger:wrapper}
binary = $${directory:service}/crond
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
wrapper = $${directory:bin}/cron_simplelogger
log = $${directory:log}/cron.log
[cron-entry-logrotate]
<= cron
recipe = slapos.cookbook:cron.d
name = logrotate
frequency = 0 0 * * *
command = $${logrotate:wrapper}
# Deploy Logrotate
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = ${logrotate:location}/sbin/logrotate
gzip-binary = ${gzip:location}/bin/gzip
gunzip-binary = ${gzip:location}/bin/gunzip
# Directories
wrapper = $${directory:bin}/logrotate
conf = $${directory:etc}/logrotate.conf
logrotate-entries = $${directory:logrotate-entries}
backup = $${directory:logrotate-backup}
state-file = $${directory:srv}/logrotate.status
[logrotate-entry-apache]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = apache
log = $${apache-configuration:error-log} $${apache-configuration:access-log}
frequency = daily
rotatep-num = 30
post = $${frontend-apache-lazy-graceful:rendered} &
sharedscripts = true
notifempty = true
create = true
[logrotate-entry-nginx]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = apache-nginx
log = $${nginx-configuration:error_log} $${nginx-configuration:access_log}
frequency = daily
rotatep-num = 30
post = $${nginx-configuration:nginx-graceful-command}
sharedscripts = true
notifempty = true
create = true
#################
# Trafficserver
#################
[trafficserver-directory]
recipe = slapos.cookbook:mkdirectory
configuration = $${directory:etc}/trafficserver
local-state = $${directory:var}/trafficserver
bin_path = ${trafficserver:location}/bin
log = $${directory:log}/trafficserver
cache-path = $${directory:srv}/ats_cache
[trafficserver-variable]
wrapper-path = $${directory:service}/trafficserver
reload-path = $${directory:etc-run}/trafficserver-reload
local-ip = $${instance-parameter:ipv4-random}
input-port = 23432
hostname = $${instance-parameter:configuration.frontend-name}
remap = map /HTTPS/ http://$${instance-parameter:ipv4-random}:$${apache-configuration:ssl-cache-through-port}
map / http://$${instance-parameter:ipv4-random}:$${apache-configuration:cache-through-port}
plugin-config = ${trafficserver:location}/libexec/trafficserver/rfc5861.so
cache-path = $${trafficserver-directory:cache-path}
disk-cache-size = $${instance-parameter:configuration.disk-cache-size}
autoconf-port = $${instance-parameter:configuration.trafficserver-autoconf-port}
mgmt-port = $${instance-parameter:configuration.trafficserver-mgmt-port}
ram-cache-size = $${instance-parameter:configuration.ram-cache-size}
[trafficserver-configuration-directory]
recipe = plone.recipe.command
command = cp -rn ${trafficserver:location}/etc/trafficserver/* $${:target}
target = $${trafficserver-directory:configuration}
[trafficserver-launcher]
recipe = slapos.cookbook:wrapper
command-line = ${trafficserver:location}/bin/traffic_cop
wrapper-path = $${trafficserver-variable:wrapper-path}
environment = TS_ROOT=$${buildout:directory}
[trafficserver-reload]
recipe = slapos.cookbook:wrapper
command-line = ${trafficserver:location}/bin/traffic_line -x
wrapper-path = $${trafficserver-variable:reload-path}
environment = TS_ROOT=$${buildout:directory}
# XXX Dedicated Jinja Section without slapparameter
[trafficserver-jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${trafficserver-directory:configuration}/$${:filename}
extra-context =
mode = 600
context =
section ats_directory trafficserver-directory
section ats_configuration trafficserver-variable
$${:extra-context}
[trafficserver-records-config]
< = trafficserver-jinja2-template-base
template = ${template-trafficserver-records-config:location}/${template-trafficserver-records-config:filename}
filename = records.config
extra-context =
import os_module os
[trafficserver-storage-config]
< = trafficserver-jinja2-template-base
template = ${template-trafficserver-storage-config:location}/${template-trafficserver-storage-config:filename}
filename = storage.config
[trafficserver-remap-config]
< = trafficserver-jinja2-template-base
template = ${template-empty:target}
filename = remap.config
context =
key content trafficserver-variable:remap
[trafficserver-plugin-config]
< = trafficserver-jinja2-template-base
template = ${template-empty:target}
filename = plugin.config
context =
key content trafficserver-variable:plugin-config
[trafficserver-promise-listen-port]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/trafficserver-port-listening
hostname = $${trafficserver-variable:local-ip}
port = $${trafficserver-variable:input-port}
[trafficserver-line]
recipe = slapos.cookbook:wrapper
command-line = ${trafficserver:location}/bin/traffic_line
wrapper-path = $${directory:bin}/traffic_line
environment = TS_ROOT=$${buildout:directory}
[trafficserver-promise-cache-availability]
recipe = collective.recipe.template
input =
inline:#!${buildout:executable}
import subprocess
import sys
traffic_line = "$${trafficserver-line:wrapper-path}"
result = float(subprocess.check_output([traffic_line, '-r', 'proxy.node.cache.percent_free' ]))
if result != 0: sys.exit(0)
sys.stderr.write("Cache not available, availability: %s" % result)
sys.exit(127)
output = $${directory:promise}/trafficserver-cache-availability
mode = 700
### End of ATS sections
### Apaches Graceful and promises
[frontend-apache-graceful-bin]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:bin}/frontend-apache-safe-graceful
mode = 0700
extra-context =
key content apache-configuration:frontend-graceful-command
[frontend-apache-graceful]
< = jinja2-template-base
template = ${template-apache-graceful-script:target}
rendered = $${directory:etc-run}/frontend-apache-safe-graceful
mode = 0700
extra-context =
key directory_run directory:run
key directory_etc directory:etc
key apache_graceful_reload_command apache-configuration:frontend-graceful-command
[frontend-apache-lazy-graceful]
< = jinja2-template-base
template = ${template-apache-lazy-script-call:target}
rendered = $${directory:bin}/frontend-apache-lazy-graceful
mode = 0700
pid-file = $${directory:run}/lazy-graceful.pid
extra-context =
key pid_file :pid-file
raw wait_time 60
key lazy_command apache-configuration:frontend-graceful-command
# Promises checking configuration:
[promise-frontend-apache-configuration]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:promise}/frontend-apache-configuration-promise
mode = 0700
extra-context =
key content apache-configuration:frontend-configuration-verification
[promise-apache-frontend-v4-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv4_https
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.port}
[promise-apache-frontend-v4-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv4_http
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.plain_http_port}
[promise-apache-frontend-v6-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv6_https
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.port}
[promise-apache-frontend-v6-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv6_http
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.plain_http_port}
[promise-apache-frontend-cached]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_cached
hostname = $${instance-parameter:ipv4-random}
port = $${apache-configuration:cache-through-port}
[promise-apache-frontend-ssl-cached]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_ssl_cached
hostname = $${instance-parameter:ipv4-random}
port = $${apache-configuration:ssl-cache-through-port}
[promise-apache-is-process-older-than-dependency-set]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/is-process-older-than-dependency-set $${apache-configuration:pid-file}
wrapper-path = $${directory:promise}/apache-frontend-is-running-actual-software-release
[promise-apache-mpm-watchdog]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/apache-mpm-watchdog --url http://$${instance-parameter:ipv4-random}:$${instance-parameter:configuration.plain_http_port}/server-status -u admin -p $${monitor-htpasswd:passwd} -d $${directory:run}/apache-mpm-watchdog.json
wrapper-path = $${directory:promise}/apache-mpm-watchdog
[slap_connection]
# Kept for backward compatiblity
computer_id = $${slap-connection:computer-id}
partition_id = $${slap-connection:partition-id}
server_url = $${slap-connection:server-url}
software_release_url = $${slap-connection:software-release-url}
key_file = $${slap-connection:key-file}
cert_file = $${slap-connection:cert-file}
[slap-parameter]
# Define default parameter(s) that will be used later, in case user didn't
# specify it
# All parameters are available through the configuration.XX syntax.
# All possible parameters should have a default.
domain = example.org
public-ipv4 =
port = 4443
plain_http_port = 8080
server-admin = admin@example.com
apache_custom_https = ""
apache_custom_http = ""
apache-key =
apache-certificate =
open-port = 80 443
extra_slave_instance_list =
frontend-name =
monitor-cors-domains =
monitor-httpd-port = 8072
monitor-username = $${monitor-instance-parameter:username}
monitor-password = $${monitor-htpasswd:passwd}
#######
# Monitoring sections
#
[monitor-instance-parameter]
monitor-httpd-port = $${slap-parameter:monitor-httpd-port}
cors-domains = $${slap-parameter:monitor-cors-domains}
username = $${slap-parameter:monitor-username}
password = $${slap-parameter:monitor-password}
[monitor-conf-parameters]
private-path-list +=
$${directory:logrotate-backup}
[monitor-traffic-summary-last-stats-wrapper]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${monitor-directory:reports}/traffic-summary-last-stats_every_1_hour
mode = 0700
command = export TS_ROOT=$${buildout:directory} && echo "<pre>$(${trafficserver:location}/bin/traffic_logstats -f $${trafficserver-directory:log}/squid.blog)</pre>"
extra-context =
key content monitor-traffic-summary-last-stats-wrapper:command
# Produce ATS Cache stats
[monitor-ats-cache-stats-wrapper]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${monitor-directory:reports}/ats-cache-stats_every_1_hour
mode = 0700
command = export TS_ROOT=$${buildout:directory} && echo "<pre>$(${trafficserver:location}/bin/traffic_shell $${monitor-ats-cache-stats-config:rendered})</pre>"
extra-context =
key content monitor-ats-cache-stats-wrapper:command
[monitor-apache-server-status-wrapper]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${monitor-directory:reports}/monitor-apache-server-status-wrapper
mode = 0700
command = ${curl:location}/bin/curl -s http://$${instance-parameter:ipv4-random}:$${instance-parameter:configuration.plain_http_port}/server-status -u $${monitor-instance-parameter:username}:$${monitor-htpasswd:passwd} 2>&1
extra-context =
key content monitor-apache-server-status-wrapper:command
[monitor-ats-cache-stats-config]
< = jinja2-template-base
template = ${template-empty:target}
rendered = $${trafficserver-configuration-directory:target}/cache-config.stats
mode = 644
context =
raw content show:cache-stats
[monitor-verify-re6st-connectivity]
recipe = slapos.cookbook:check_url_available
path = $${directory:promise}/re6st-connectivity
url = $${instance-parameter:configuration.re6st-verification-url}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
#######################
# Nginx
#
[nginx-frontend]
recipe = slapos.cookbook:wrapper
command-line = ${nginx-push-stream:location}/sbin/nginx -c $${nginx-configuration:output}
wrapper-path = $${directory:service}/frontend_nginx
[nginx-configuration]
recipe = slapos.recipe.template
url = ${template-nginx-configuration:output}
output = $${directory:etc}/nginx.cfg
mode = 0600
access_log = $${directory:log}/nginx-access.log
error_log = $${directory:log}/nginx-error.log
ip = $${slap-network-information:global-ipv6}
local_ip = $${slap-network-information:local-ipv4}
port = $${instance-parameter:configuration.nginx_port}
plain_port = $${instance-parameter:configuration.plain_nginx_port}
worker_processes = 4
worker_connections = 1024
slave-configuration-directory = $${apache-directory:nginx-slave-configuration}
pid-file = $${directory:run}/nginx.pid
nginx-graceful-command = $${:nginx-configuration-verification}; if [ $? -eq 0 ]; then kill -HUP $(cat $${:pid-file}); fi
nginx-configuration-verification = ${nginx-push-stream:location}/sbin/nginx -t -c $${nginx-configuration:output}
[frontend-nginx-graceful]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:etc-run}/frontend-nginx-safe-graceful
mode = 0700
extra-context =
key content nginx-configuration:nginx-graceful-command
[promise-nginx-configuration]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:promise}/nginx-configuration-promise
mode = 0700
extra-context =
key content nginx-configuration:nginx-configuration-verification
[promise-nginx-frontend-v4-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv4_https
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.nginx_port}
[promise-nginx-frontend-v4-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv4_http
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.plain_nginx_port}
[promise-nginx-frontend-v6-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv6_https
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.nginx_port}
[promise-nginx-frontend-v6-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv6_http
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.plain_nginx_port}
[promise-nginx-is-process-older-than-dependency-set]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/is-process-older-than-dependency-set $${nginx-configuration:pid-file}
wrapper-path = $${directory:promise}/promise-nginx-is-process-older-than-dependency-set
{% if slap_software_type in software_type -%}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = ${buildout:directory}/${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-parameter:slap_software_type
key slave_instance_list slap-parameter:slave_instance_list
${:extra-context}
{% set part_list = [] -%}
{% set single_type_key = 'single-' %}
{% if slap_software_type == "replicate" %}
{% set frontend_type = slapparameter_dict.pop('-frontend-type', 'single-default') -%}
{% elif slap_software_type in ['default', 'RootSoftwareInstance'] -%}
{% set frontend_type = "%s%s" % (single_type_key, 'custom-personal') -%}
{% else -%}
{% set frontend_type = "%s%s" % (single_type_key, slap_software_type) -%}
{% endif -%}
{% set frontend_quantity = slapparameter_dict.pop('-frontend-quantity', '1') | int -%}
{% set slave_list_name = 'extra_slave_instance_list' -%}
{% set frontend_list = [] %}
{% set frontend_section_list = [] %}
{% set request_dict = {} %}
{% set namebase = 'apache-frontend' -%}
# XXX Dirty hack, not possible to define default value before
{% set sla_computer_apache_1_key = '-sla-1-computer_guid' -%}
{% if not sla_computer_apache_1_key in slapparameter_dict -%}
{% do slapparameter_dict.__setitem__(sla_computer_apache_1_key, '${slap-connection:computer-id}') -%}
{% endif -%}
# Here we request individualy each frontend.
# The presence of sla parameters is checked and added if found
{% for i in range(1, frontend_quantity + 1) -%}
{% set frontend_name = "%s-%s" % (namebase, i) -%}
{% set request_section_title = 'request-%s' % frontend_name -%}
{% set sla_key = "-sla-%s-" % i -%}
{% set sla_key_length = sla_key | length %}
{% set sla_dict = {} %}
{% set config_key = "-frontend-config-%s-" % i %}
{% set config_key_length = config_key | length %}
{% set config_dict = {} %}
{% for key in slapparameter_dict.keys() %}
{% if key.startswith(sla_key) %}
{% do sla_dict.__setitem__(key[sla_key_length:], slapparameter_dict.pop(key)) %}
# We check for specific configuration regarding the frontend
{% elif key.startswith(config_key) -%}
{% do config_dict.__setitem__(key[config_key_length:], slapparameter_dict.pop(key)) %}
{% endif -%}
{% endfor -%}
{% do frontend_list.append(frontend_name) -%}
{% do frontend_section_list.append(request_section_title) -%}
{% do part_list.append(request_section_title) -%}
# Filling request dict for slave
{% set state_key = "-frontend-%s-state" % i %}
{% do request_dict.__setitem__(request_section_title,
{
'config': config_dict,
'name': frontend_name,
'sla': sla_dict,
'state': slapparameter_dict.pop(state_key, None)
}) %}
{% endfor -%}
{% set authorized_slave_string = slapparameter_dict.pop('-frontend-authorized-slave-string', '') -%}
{% set authorized_slave_list = [] %}
{% set rejected_slave_list = [] %}
{% for slave in slave_instance_list %}
{% if not (slave.has_key('apache_custom_http') and not slave.get('slave_reference') in authorized_slave_string) %}
{% do authorized_slave_list.append(slave) %}
{% else %}
{% do rejected_slave_list.append(slave.get('slave_reference')) %}
{% endif %}
{% endfor -%}
[replicate]
<= slap-connection
recipe = slapos.cookbook:requestoptional
config-monitor-cors-domains = {{ slapparameter_dict.get('monitor-cors-domains', 'monitor.app.officejs.com') }}
config-monitor-username = ${monitor-instance-parameter:username}
config-monitor-password = ${monitor-htpasswd:passwd}
{% set frontend_software_url_key = "-frontend-software-release-url" %}
{% if slapparameter_dict.has_key(frontend_software_url_key) %}
software-url = {{ slapparameter_dict.pop(frontend_software_url_key) }}
{% else %}
software-url = ${slap-connection:software-release-url}
{% endif %}
software-type = {{frontend_type}}
return = private-ipv4 public-ipv4 slave-instance-information-list monitor-base-url
{% for section, frontend_request in request_dict.iteritems() %}
[{{section}}]
<= replicate
name = {{ frontend_request.get('name') }}
{% if frontend_request.get('state') %}
state = {{ frontend_request.get('state') }}
{% endif%}
{% set slave_configuration_dict = frontend_request.get('config') %}
{% do slave_configuration_dict.update(**slapparameter_dict) %}
{% do slave_configuration_dict.__setitem__(slave_list_name, json_module.dumps(authorized_slave_list)) %}
{% do slave_configuration_dict.__setitem__("frontend-name", frontend_request.get('name')) %}
config-_ = {{ json_module.dumps(slave_configuration_dict) }}
{% if frontend_request.get('sla') %}
{% for parameter, value in frontend_request.get('sla').iteritems() -%}
sla-{{ parameter }} = {{ value }}
{% endfor -%}
{% endif -%}
{% endfor -%}
[publish-information]
<= monitor-publish
recipe = slapos.cookbook:publish
domain = {{ slapparameter_dict.get('domain') }}
slave-amount = {{ slave_instance_list | length }}
accepted-slave-amount = {{ authorized_slave_list | length }}
rejected-slave-amount = {{ rejected_slave_list | length }}
rejected-slave-list = {{ json_module.dumps(rejected_slave_list) }}
#----------------------------
#--
#-- Publish slave information
[publish-slave-information]
recipe = slapos.cookbook:softwaretype
default = ${dynamic-publish-slave-information:rendered}
RootSoftwareInstance = ${dynamic-publish-slave-information:rendered}
replicate = ${dynamic-publish-slave-information:rendered}
custom-personal = ${dynamic-publish-slave-information:rendered}
custom-group = ${dynamic-publish-slave-information:rendered}
[slave-information]
{% for frontend_section in frontend_section_list -%}
{{ frontend_section }} = {{ "${%s:connection-slave-instance-information-list}" % frontend_section }}
{% endfor -%}
[dynamic-publish-slave-information]
< = jinja2-template-base
template = {{ template_publish_slave_information }}
filename = dynamic-publish-slave-information.cfg
extensions = jinja2.ext.do
extra-context =
section slave_information slave-information
[monitor-conf-parameters]
monitor-url-list +=
{% for frontend in frontend_section_list %}
{{ ' ${' + frontend + ':connection-monitor-base-url}' }}
{% endfor -%}
[buildout]
extends = {{ template_monitor }}
parts =
monitor-base
publish-slave-information
publish-information
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
# publish-information
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[slap_connection]
# Kept for backward compatiblity
computer_id = ${slap-connection:computer-id}
partition_id = ${slap-connection:partition-id}
server_url = ${slap-connection:server-url}
software_release_url = ${slap-connection:software-release-url}
key_file = ${slap-connection:key-file}
cert_file = ${slap-connection:cert-file}
[slap-parameter]
slave_instance_list =
-frontend-quantity = 1
-frontend-type = single-default
{%- endif %}
[buildout]
parts =
dynamic-template-apache-replicate
switch-softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[slap-parameters]
recipe = slapos.cookbook:slapconfiguration
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-parameters:slap-software-type
key slapparameter_dict slap-parameters:configuration
key slave_instance_list slap-parameters:slave-instance-list
$${:extra-context}
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = $${dynamic-template-apache-replicate:rendered}
RootSoftwareInstance = $${dynamic-template-apache-replicate:rendered}
custom-personal = $${dynamic-template-apache-replicate:rendered}
single-default = ${template-apache-frontend:output}
single-custom-personal = ${template-apache-frontend:output}
replicate = $${dynamic-template-apache-replicate:rendered}
[dynamic-template-apache-replicate]
< = jinja2-template-base
template = ${template-apache-replicate:target}
filename = instance-apache-replicate.cfg
extensions = jinja2.ext.do
extra-context =
raw template_publish_slave_information ${template-replicate-publish-slave-information:target}
# Must match the key id in [switch-softwaretype] which uses this section.
raw software_type RootSoftwareInstance-default-custom-personal-replicate
raw template_monitor ${monitor2-template:rendered}
[buildout]
extends = common.cfg
[versions]
PyRSS2Gen = 1.1
apache-libcloud = 0.19.0
cns.recipe.symlink = 0.2.3
ecdsa = 0.13
gitdb = 0.6.4
plone.recipe.command = 1.1
pycrypto = 2.6.1
rdiff-backup = 1.0.5+SlapOSPatched001
slapos.recipe.template = 4.3
slapos.toolbox = 0.76
smmap = 0.9.0
numpy = 1.11.2
pyasn1 = 0.2.3
websockify = 0.8.0
# Required by:
# slapos.toolbox==0.71
dnspython = 1.15.0
# Required by:
# slapos.toolbox==0.71
erp5.util = 0.4.49
# Required by:
# slapos.toolbox==0.71
passlib = 1.6.5
# Required by:
# slapos.toolbox==0.71
GitPython = 2.0.8
# Required by:
# slapos.toolbox==0.71
atomize = 0.2.0
# Required by:
# slapos.toolbox==0.71
feedparser = 5.2.1
# Required by:
# slapos.toolbox==0.71
lockfile = 0.12.2
# Required by:
# slapos.toolbox==0.71
paramiko = 2.0.1
# Required by:
# slapos.toolbox==0.71
pycurl = 7.43.0
# Required by:
# slapos.toolbox==0.71
rpdb = 0.1.5
<VirtualHost *:{{ https_port }}>
ServerName www.example.org
SSLEngine on
SSLProxyEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
# Rewrite part
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
ErrorDocument 404 /notfound.html
</VirtualHost>
<VirtualHost *:{{ http_port }}>
ServerName www.example.org
ErrorDocument 404 /notfound.html
</VirtualHost>
\ No newline at end of file
{% if software_type == slap_software_type -%}
{% set cached_server_dict = {} -%}
{% set part_list = [] -%}
{% set cache_port = apache_configuration.get('cache-port') %}
{% set cached_port = apache_configuration.get('cache-through-port') %}
{% set ssl_cached_port = apache_configuration.get('ssl-cache-through-port') %}
{% set cache_access = "http://%s:%s" % (local_ipv4, cache_port) -%}
{% set ssl_cache_access = "http://%s:%s/HTTPS" % (local_ipv4, cache_port) -%}
{% set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
{% set NGINX_TYPE_LIST = ['eventsource', 'notebook'] -%}
{% set generic_instance_parameter_dict = {'cache_access': cache_access,} -%}
{% set slave_log_dict = {} -%}
{% if extra_slave_instance_list -%}
{% set slave_instance_information_list = [] -%}
{% set slave_instance_list = slave_instance_list + json_module.loads(extra_slave_instance_list) -%}
{% endif -%}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
extensions = jinja2.ext.do
extra-context =
context =
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
${:extra-context}
{% do logrotate_dict.pop('recipe') %}
[logrotate]
{% for key, value in logrotate_dict.iteritems() -%}
{{ key }} = {{ value }}
{% endfor %}
post = {{ frontend_lazy_graceful_reload }} &
frequency = daily
rotatep-num = 30
sharedscripts = true
notifempty = true
create = true
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = {{ custom_ssl_directory }}/requests/
private = {{ custom_ssl_directory }}/private/
certs = {{ custom_ssl_directory }}/certs/
newcerts = {{ custom_ssl_directory }}/newcerts/
crl = {{ custom_ssl_directory }}/crl/
{# Loop throught slave list to set up slaves #}
{% for slave_instance in slave_instance_list -%}
{# # Do all set and do upper, so it makes easy to read the file later #}
{% set slave_reference = slave_instance.get('slave_reference') -%}
{% set slave_type = slave_instance.get('type', '') -%}
{% set slave_section_title = 'dynamic-template-slave-instance-%s' % slave_reference -%}
{% set slave_parameter_dict = generic_instance_parameter_dict.copy() -%}
{% set slave_publish_dict = {} -%}
{% set slave_configuration_section_name = 'slave-instance-%s-configuration' % slave_reference %}
{% set slave_logrotate_section = slave_reference + "-logs" -%}
{% set slave_password_section = slave_reference + "-password" -%}
{% set slave_ln_section = slave_reference + "-ln" -%}
{% set slave_htaccess_section = slave_reference + '-htaccess' %}
{# extend parts #}
{% do part_list.extend([slave_htaccess_section, slave_ln_section]) -%}
{% do part_list.extend([slave_logrotate_section, slave_section_title]) -%}
{% set slave_log_folder = logrotate_dict.get('backup') + '/' + slave_reference + "-logs" -%}
{# Set Up log files #}
{% do slave_parameter_dict.__setitem__('access_log', '/'.join([apache_log_directory, '%s_access_log' % slave_reference])) -%}
{% do slave_parameter_dict.__setitem__('error_log', '/'.join([apache_log_directory, '%s_error_log' % slave_reference])) -%}
{% do slave_instance.__setitem__('access_log', slave_parameter_dict.get('access_log')) -%}
{% do slave_instance.__setitem__('error_log', slave_parameter_dict.get('error_log')) -%}
{# Add slave log directory to the slave log access dict #}
{% do slave_log_dict.__setitem__(slave_reference, slave_log_folder) %}
{% set slave_log_access_url = 'https://' + slave_reference + ':${'+ slave_password_section +':passwd}@[' + frontend_configuration.get('apache-ipv6') + ']:' + frontend_configuration.get('apache-https-port') + '/' + slave_reference.lower() + '/' %}
{% do slave_publish_dict.__setitem__('log-access', slave_log_access_url) %}
{% do slave_publish_dict.__setitem__('slave-reference', slave_reference) %}
{% do slave_publish_dict.__setitem__('public-ipv4', public_ipv4) %}
{# Set slave domain if none was defined #}
{% if slave_instance.get('custom_domain', None) == None -%}
{% set domain_prefix = slave_instance.get('slave_reference').replace("-", "").replace("_", "").lower() -%}
{% if slave_type in NGINX_TYPE_LIST -%}
{% do slave_instance.__setitem__('custom_domain', "%s.%s" % (domain_prefix, slapparameter_dict.get('nginx-domain', slapparameter_dict.get('domain')))) -%}
{% else -%}
{% do slave_instance.__setitem__('custom_domain', "%s.%s" % (domain_prefix, slapparameter_dict.get('domain'))) -%}
{% endif -%}
{% endif -%}
{% set enable_cache = (('' ~ slave_instance.get('enable_cache', '')).lower() in TRUE_VALUES and slave_type != 'redirect') -%}
{% if enable_cache and 'url' in slave_instance -%}
{% if 'domain' in slave_instance -%}
{% do slave_instance.__setitem__('custom_domain', slave_instance.get('domain')) -%}
{% endif -%}
{% do slave_instance.__setitem__('backend_url', slave_instance.get('url')) -%}
{% do slave_instance.__setitem__('https_backend_url', slave_instance.get('https-url', slave_instance.get('url'))) -%}
{% do slave_instance.__setitem__('url', cache_access) -%}
{% do slave_instance.__setitem__('https-url', ssl_cache_access) -%}
{% do cached_server_dict.__setitem__(slave_reference, slave_configuration_section_name) -%}
{% endif -%}
{% if not slave_instance.has_key('apache_custom_http') %}
{% do slave_publish_dict.__setitem__('domain', slave_instance.get('custom_domain')) -%}
{% do slave_publish_dict.__setitem__('url', "http://%s" % slave_instance.get('custom_domain')) -%}
{% do slave_publish_dict.__setitem__('site_url', "http://%s" % slave_instance.get('custom_domain')) -%}
{% do slave_publish_dict.__setitem__('secure_access', 'https://%s' % slave_instance.get('custom_domain')) -%}
{% endif -%}
[slave-log-directories]
{{slave_reference}} = {{ slave_log_folder }}
{# Set slave logrotate entry #}
[{{slave_logrotate_section}}]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = ${:_buildout_section_name_}
log = {{slave_parameter_dict.get('access_log')}} {{slave_parameter_dict.get('error_log')}}
backup = {{ slave_log_folder }}
{# integrate current logs inside #}
[{{slave_ln_section}}]
recipe = plone.recipe.command
stop-on-error = false
command = ln -s {{slave_parameter_dict.get('error_log')}} {{ slave_log_folder }}/apache-error.log && ln -s {{slave_parameter_dict.get('access_log')}} {{ slave_log_folder }}/apache-access.log
{# Set password for slave #}
[{{slave_password_section}}]
recipe = slapos.cookbook:generate.password
storage-path = {{apache_configuration_directory}}/.{{slave_reference}}.passwd
bytes = 8
{# Set up htaccess file for slave #}
[{{slave_htaccess_section}}]
recipe = plone.recipe.command
stop-on-error = true
htaccess-path = {{apache_configuration_directory}}/.{{slave_reference}}.htaccess
command = {{frontend_configuration.get('apache-directory')}}/bin/htpasswd -cb ${:htaccess-path} {{ slave_reference }} {{ '${' + slave_password_section + ':passwd}' }}
{# ################################################## #}
{# Set Slave Certificates if needed #}
{# Set ssl certificates for each slave #}
{% for cert_name in ('ssl_ca_crt', 'ssl_csr', 'ssl_proxy_ca_crt')-%}
{% if cert_name in slave_instance -%}
{% set cert_title = '%s-%s' % (slave_reference, cert_name.replace('ssl_', '')) -%}
{% set cert_file = '/'.join([custom_ssl_directory, cert_title.replace('-','.')]) -%}
{% do part_list.append(cert_title) -%}
{% do slave_parameter_dict.__setitem__(cert_name, cert_file) -%}
{% do slave_instance.__setitem__('path_to_' + cert_name, cert_file) -%}
{# Store certificates on fs #}
[{{ cert_title }}]
< = jinja2-template-base
template = {{ empty_template }}
rendered = {{ cert_file }}
extra-context =
key content {{ cert_title + '-config:value' }}
# Store certificate in config
[{{ cert_title + '-config' }}]
value = {{ dumps(slave_instance.get(cert_name)) }}
{% endif -%}
{% endfor -%}
{% if 'ssl_key' in slave_instance and 'ssl_crt' in slave_instance -%}
{% set cert_title = '%s-crt' % (slave_reference) -%}
{% set key_title = '%s-key' % (slave_reference) -%}
{% set cert_file = '/'.join([custom_ssl_directory, cert_title.replace('-','.')]) -%}
{% set key_file = '/'.join([custom_ssl_directory, key_title.replace('-','.')]) -%}
{% do part_list.append(cert_title) -%}
{% do slave_parameter_dict.__setitem__("ssl_crt", cert_file) -%}
{% do slave_parameter_dict.__setitem__("ssl_key", key_file) -%}
{% do slave_instance.__setitem__('path_to_ssl_crt', cert_file) -%}
{% do slave_instance.__setitem__('path_to_ssl_key', key_file) -%}
[{{cert_title}}]
recipe = slapos.cookbook:certificate_authority.request
#openssl-binary = ${openssl:location}/bin/openssl
requests-directory = ${cadirectory:requests}
ca-private = ${cadirectory:private}
ca-certs = ${cadirectory:certs}
ca-newcerts = ${cadirectory:newcerts}
ca-crl = ${cadirectory:crl}
key-file = {{ key_file }}
cert-file = {{ cert_file }}
key-content = {{ dumps(slave_instance.get('ssl_key')) }}
cert-content = {{ dumps(slave_instance.get('ssl_crt')) }}
{% endif -%}
{# ########################################## #}
{# Set Slave Configuration #}
[{{ slave_configuration_section_name }}]
{% set apache_custom_http = ((slave_instance.pop('apache_custom_http', '')) % slave_parameter_dict) -%}
{% set apache_custom_https = ((slave_instance.pop('apache_custom_https', '')) % slave_parameter_dict) -%}
apache_custom_http = {{ dumps(apache_custom_http) }}
apache_custom_https = {{ dumps(apache_custom_https) }}
{{ '\n' }}
{% for key, value in slave_instance.iteritems() -%}
{{ key }} = {{ dumps(value) }}
{% endfor %}
[{{ slave_section_title }}]
< = jinja2-template-base
{% if slave_type in NGINX_TYPE_LIST %}
rendered = {{ nginx_configuration_directory }}/${:filename}
{% else %}
rendered = {{ apache_configuration_directory }}/${:filename}
{% endif %}
{% if apache_custom_http %}
template = {{ template_custom_slave_configuration }}
{% elif slave_type == 'eventsource' %}
template = {{ template_eventsource_slave_configuration }}
{% elif slave_type == 'notebook' %}
template = {{ template_notebook_slave_configuration }}
{% else %}
template = {{ template_default_slave_configuration }}
{% endif %}
filename = {{ '%s.conf' % slave_reference }}
extra-context =
raw https_port {{ https_port }}
raw http_port {{ http_port }}
raw global_ipv6 {{ global_ipv6 }}
raw local_ipv4 {{ local_ipv4 }}
raw nginx_http_port {{ nginx_http_port }}
raw nginx_https_port {{ nginx_https_port }}
section slave_parameter {{ slave_configuration_section_name }}
{{ '\n' }}
{% set check_error_log_section_title = 'check-%s-error-log-last-hour' % slave_instance.get('slave_reference') -%}
{% do part_list.append(check_error_log_section_title) -%}
[{{ check_error_log_section_title }}]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/check-error-on-apache-log -l {{ slave_instance.get('error_log') }} -d 3600
filename = {{ check_error_log_section_title }}
wrapper-path = {{ promise_directory }}/${:filename}
{% set check_error_log_section_title = 'check-%s-error-log-last-day' % slave_instance.get('slave_reference') -%}
{% do part_list.append(check_error_log_section_title) -%}
[{{ check_error_log_section_title }}]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/check-error-on-apache-log -l {{ slave_instance.get('error_log') }} -d 86400
filename = {{ check_error_log_section_title }}
wrapper-path = {{ promise_directory }}/${:filename}
{% set monitor_ipv6_test = slave_instance.get('monitor-ipv6-test', '') %}
{% if monitor_ipv6_test %}
{% set monitor_ipv6_section_title = 'check-%s-ipv6-packet-list-test' % slave_instance.get('slave_reference') %}
{% do part_list.append(monitor_ipv6_section_title) -%}
[{{ monitor_ipv6_section_title }}]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/is-icmp-packet-lost -a {{monitor_ipv6_test}}
filename = {{ monitor_ipv6_section_title }}
wrapper-path = {{ promise_directory }}/${:filename}
{% endif %}
{% set monitor_ipv4_test = slave_instance.get('monitor-ipv4-test', '') %}
{% if monitor_ipv4_test %}
{% set monitor_ipv4_section_title = 'check-%s-ipv4-packet-list-test' % slave_instance.get('slave_reference') %}
{% do part_list.append(monitor_ipv4_section_title) -%}
[{{ monitor_ipv4_section_title }}]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/is-icmp-packet-lost -4 -a {{monitor_ipv4_test}}
filename = {{ monitor_ipv4_section_title }}
wrapper-path = {{ promise_directory }}/${:filename}
{% endif %}
{% set re6st_optimal_test = slave_instance.get('re6st-optimal-test', '') %}
{% if re6st_optimal_test %}
{% set re6st_ipv6, re6st_ipv4 = re6st_optimal_test.split(",") %}
{% set re6st_optimal_test_section_title = 'check-%s-re6st-optimal-test' % slave_instance.get('slave_reference') %}
{% do part_list.append(re6st_optimal_test_section_title) -%}
[{{ re6st_optimal_test_section_title }}]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/check-re6st-optimal-status -4 {{re6st_ipv4}} -6 {{re6st_ipv6}}
filename = {{ re6st_optimal_test_section_title }}
wrapper-path = {{ promise_directory }}/${:filename}
{% endif %}
{# ############################### #}
{# Publish Slave Information #}
{% if not extra_slave_instance_list -%}
{% set publish_section_title = 'publish-%s-connection-information' % slave_instance.get('slave_reference') -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
{% for key, value in slave_publish_dict.iteritems() %}
{{ key }} = {{ value }}
{% endfor %}
{% else -%}
{% do slave_instance_information_list.append(slave_publish_dict) -%}
{% endif -%}
{# End of the main for loop#}
{% endfor -%}
###############################################
### Prepare virtualhost for slaves using cache
{% for slave_reference, slave_configuration_section_name in cached_server_dict.iteritems() %}
{% set cached_slave_configuration_section_title = '%s-cached-virtualhost' % slave_reference %}
{% do part_list.append(cached_slave_configuration_section_title) -%}
[{{ cached_slave_configuration_section_title }}]
< = jinja2-template-base
template = {{ template_cached_slave_configuration }}
filename = {{ '%s.conf' % slave_reference }}
rendered = {{ apache_cached_configuration_directory }}/${:filename}
extensions = jinja2.ext.do
extra-context =
section slave_parameter {{ slave_configuration_section_name }}
raw cached_port {{ cached_port }}
raw ssl_cached_port {{ ssl_cached_port }}
{{ '\n' }}
{% endfor %}
[slave-log-directories]
recipe = slapos.cookbook:mkdirectory
{# Define log access #}
[apache-log-access]
< = jinja2-template-base
template = {{frontend_configuration.get('template-log-access')}}
rendered = {{frontend_configuration.get('log-access-configuration')}}
extra-context =
section slave_log_directory slave-log-directories
raw apache_log_directory {{apache_log_directory}}
raw apache_configuration_directory {{apache_configuration_directory}}
{# Publish information for the instance #}
[publish-apache-information]
recipe = slapos.cookbook:publish
public-ipv4 = {{ public_ipv4 }}
private-ipv4 = {{ local_ipv4 }}
{% if extra_slave_instance_list -%}
slave-instance-information-list = {{ json_module.dumps(slave_instance_information_list) }}
{% endif -%}
monitor-base-url = {{ monitor_base_url }}
[buildout]
parts +=
slave-log-directories
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor %}
publish-apache-information
apache-log-access
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
cache-access = {{ cache_access }}
{% endif -%}
#!/bin/sh
RUN_DIR={{ directory_run }}
ETC_DIR={{ directory_etc }}
APACHE_SIGNATURE_FILE=$RUN_DIR/apache_configuration.signature
NAPACHE_SIGNATURE_FILE=$RUN_DIR/napache_configuration.signature
touch $APACHE_SIGNATURE_FILE
sha256sum $ETC_DIR/apache*.conf $ETC_DIR/apache-*.d/*.conf $ETC_DIR/apache-*.d/ssl/*.*key $ETC_DIR/apache-*.d/ssl/*.*crt* | sort -k 66 > $NAPACHE_SIGNATURE_FILE
# If no diff, no restart for now
if diff "$APACHE_SIGNATURE_FILE" "$NAPACHE_SIGNATURE_FILE"; then
echo "Nothing Changed, so nothing to reload"
exit 0
fi
echo "Reloading apache.."
{{ apache_graceful_reload_command }}
mv "$NAPACHE_SIGNATURE_FILE" "$APACHE_SIGNATURE_FILE"
#!/bin/bash
PID=$$
PIDFILE={{ pid_file }}
sleep $((RANDOM % 10))
if [ ! -f $PIDFILE ]; then
echo $PID > $PIDFILE
sleep {{ wait_time }}
{{ lazy_command }}
else
ps --pid `cat $PIDFILE` &>/dev/null
if [ $? -eq 0 ]; then
echo "Skipped"
else
echo $PID > $PIDFILE
sleep {{ wait_time }}
{{ lazy_command }}
fi
fi
# Apache configuration file for Zope
# Automatically generated
# Basic server configuration
PidFile "{{ pid_file }}"
ServerName {{ domain }}
DocumentRoot {{ document_root }}
ServerRoot {{ instance_home }}
{{ "Listen %s:%s" % (ipv4_addr, cached_port) }}
{{ "Listen %s:%s" % (ipv4_addr, ssl_cached_port) }}
{% for ip in (ipv4_addr, "[%s]" % ipv6_addr) -%}
{% for port in (http_port, https_port) -%}
{{ "Listen %s:%s" % (ip, port) }}
{% endfor -%}
{% endfor -%}
ServerAdmin {{ server_admin }}
TypesConfig {{ httpd_home }}/conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
# As backend is trusting REMOTE_USER header unset it always
RequestHeader unset REMOTE_USER
ServerTokens Prod
# Disable TRACE Method
TraceEnable off
# Log configuration
ErrorLog "{{ error_log }}"
LogLevel notice
# LogFormat "%h %{REMOTE_USER}i %{Host}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
# LogFormat "%h %{REMOTE_USER}i %{Host}i %l %u %t \"%r\" %>s %b" common
# CustomLog "{{ access_log }}" common
LogFormat "%h %l %{REMOTE_USER}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog "{{ access_log }}" combined
<Directory {{ protected_path }}>
Order Deny,Allow
Allow from {{ access_control_string }}
</Directory>
<Directory {{ document_root }}>
Require all granted
Options -Indexes
ErrorDocument 404 /notfound.html
RewriteEngine on
RewriteRule ^/?$ notfound.html [R=404,L]
</Directory>
# List of modules
LoadModule unixd_module {{ httpd_home }}/modules/mod_unixd.so
LoadModule access_compat_module {{ httpd_home }}/modules/mod_access_compat.so
LoadModule authz_core_module {{ httpd_home }}/modules/mod_authz_core.so
LoadModule authz_host_module {{ httpd_home }}/modules/mod_authz_host.so
LoadModule authn_core_module {{ httpd_home }}/modules/mod_authn_core.so
LoadModule log_config_module {{ httpd_home }}/modules/mod_log_config.so
LoadModule deflate_module {{ httpd_home }}/modules/mod_deflate.so
LoadModule setenvif_module {{ httpd_home }}/modules/mod_setenvif.so
LoadModule version_module {{ httpd_home }}/modules/mod_version.so
LoadModule proxy_module {{ httpd_home }}/modules/mod_proxy.so
LoadModule proxy_http_module {{ httpd_home }}/modules/mod_proxy_http.so
LoadModule ssl_module {{ httpd_home }}/modules/mod_ssl.so
LoadModule mime_module {{ httpd_home }}/modules/mod_mime.so
LoadModule dav_module {{ httpd_home }}/modules/mod_dav.so
LoadModule dav_fs_module {{ httpd_home }}/modules/mod_dav_fs.so
LoadModule negotiation_module {{ httpd_home }}/modules/mod_negotiation.so
LoadModule rewrite_module {{ httpd_home }}/modules/mod_rewrite.so
LoadModule headers_module {{ httpd_home }}/modules/mod_headers.so
LoadModule cache_module {{ httpd_home }}/modules/mod_cache.so
LoadModule cache_socache_module {{ httpd_home }}/modules/mod_cache_socache.so
LoadModule socache_shmcb_module {{ httpd_home }}/modules/mod_socache_shmcb.so
LoadModule antiloris_module {{ httpd_home }}/modules/mod_antiloris.so
LoadModule alias_module {{ httpd_home }}/modules/mod_alias.so
LoadModule autoindex_module {{ httpd_home }}/modules/mod_autoindex.so
LoadModule auth_basic_module {{ httpd_home }}/modules/mod_auth_basic.so
LoadModule authz_user_module {{ httpd_home }}/modules/mod_authz_user.so
LoadModule authn_file_module {{ httpd_home }}/modules/mod_authn_file.so
LoadModule filter_module {{ httpd_home }}/modules/mod_filter.so
LoadModule http2_module {{ httpd_home }}/modules/mod_http2.so
LoadModule info_module {{ httpd_home }}/modules/mod_info.so
LoadModule status_module {{ httpd_home }}/modules/mod_status.so
LoadModule reqtimeout_module {{ httpd_home }}/modules/mod_reqtimeout.so
# The following directives modify normal HTTP response behavior to
# handle known problems with browser implementations.
BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4\.0" force-response-1.0
BrowserMatch "Java/1\.0" force-response-1.0
BrowserMatch "JDK/1\.0" force-response-1.0
# The following directive disables redirects on non-GET requests for
# a directory that does not include the trailing slash. This fixes a
# problem with Microsoft WebFolders which does not appropriately handle
# redirects for folders with DAV methods.
# Same deal with Apple's DAV filesystem and Gnome VFS support for DAV.
BrowserMatch "Microsoft Data Access Internet Publishing Provider" redirect-carefully
BrowserMatch "MS FrontPage" redirect-carefully
BrowserMatch "^WebDrive" redirect-carefully
BrowserMatch "^WebDAVFS/1.[0123]" redirect-carefully
BrowserMatch "^gnome-vfs" redirect-carefully
BrowserMatch "^XML Spy" redirect-carefully
BrowserMatch "^Dreamweaver-WebDAV-SCM1" redirect-carefully
{% if slapparameter_dict.get('enable-http2-by-default', 'true') == 'true' %}
Protocols h2 http/1.1
{% endif %}
# Increase IPReadLimit to 10
<IfModule antiloris_module>
# IPReadLimit - Maximum simultaneous connections in READ state per IP address
IPReadLimit {{ slapparameter_dict.get('ip-read-limit', '10') }}
</IfModule>
ExtendedStatus On
<Location /server-status>
SetHandler server-status
Order Deny,Allow
Deny from all
Allow from All
AuthType basic
AuthName "Apache Server Status"
AuthBasicProvider file
AuthUserFile {{ instance_home }}/etc/monitor-htpasswd
Require valid-user
</Location>
ServerLimit {{ slapparameter_dict.get('mpm-server-limit', '16') }}
MaxClients {{ slapparameter_dict.get('mpm-max-clients', '400') }}
StartServers {{ slapparameter_dict.get('mpm-start-servers', '3') }}
ThreadsPerChild {{ slapparameter_dict.get('mpm-thread-per-child', '25') }}
#MaxRequestsPerChild 0 // Default value is 0
GracefulShutdownTimeout {{ slapparameter_dict.get('mpm-graceful-shutdown-timeout', '5') }}
# Deflate
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript application/x-javascript application/javascript
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
# SSL Configuration
SSLCertificateFile {{ login_certificate }}
SSLCertificateKeyFile {{ login_key }}
{% if slapparameter_dict.get('apache-ca-certificate') %}
SSLCACertificateFile {{ login_ca_crt }}
{% endif %}
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLSessionCache shmcb:/{{ httpd_mod_ssl_cache_directory }}/ssl_scache(512000)
SSLSessionCacheTimeout 300
SSLRandomSeed startup /dev/urandom 256
SSLRandomSeed connect builtin
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
# Accept proxy to sites using self-signed SSL certificates
SSLProxyCheckPeerCN off
SSLProxyCheckPeerExpire off
include {{frontend_configuration.get('log-access-configuration')}}
includeoptional {{ slave_configuration_directory }}/*.conf
includeoptional {{ slave_with_cache_configuration_directory }}/*.conf
ErrorDocument 404 /notfound.html
RewriteRule (.*) /notfound.html [R=404,L]
{% set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
{% set server_alias_list = slave_parameter.get('server-alias', '').split() %}
{% set ssl_proxy_verify = ('' ~ slave_parameter.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES -%}
# Only accept generic (i.e not Zope) backends on http
<VirtualHost *:{{ cached_port }}>
ServerName {{ slave_parameter.get('custom_domain') }}
{%- for server_alias in server_alias_list %}
ServerAlias {{ server_alias }}
{% endfor %}
SSLProxyEngine on
{% if ssl_proxy_verify -%}
{% if 'ssl_proxy_ca_crt' in slave_parameter -%}
SSLProxyCACertificateFile {{ slave_parameter.get('path_to_ssl_proxy_ca_crt', '') }}
{% endif %}
SSLProxyVerify require
#SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
{% endif %}
# Rewrite part
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
RewriteRule ^/(.*)$ {{ slave_parameter.get('backend_url', '') }}/$1 [L,P]
</VirtualHost>
<VirtualHost *:{{ ssl_cached_port }}>
ServerName {{ slave_parameter.get('custom_domain') }}
{%- for server_alias in server_alias_list %}
ServerAlias {{ server_alias }}
{% endfor %}
SSLProxyEngine on
{% if ssl_proxy_verify -%}
{% if 'ssl_proxy_ca_crt' in slave_parameter -%}
SSLProxyCACertificateFile {{ slave_parameter.get('path_to_ssl_proxy_ca_crt', '') }}
{% endif %}
SSLProxyVerify require
#SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
{% endif %}
# Rewrite part
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
RewriteRule ^/(.*)$ {{ slave_parameter.get('https_backend_url', '') }}/$1 [L,P]
</VirtualHost>
<VirtualHost *:{{ https_port }}>
{{ slave_parameter.get('apache_custom_https', '') }}
</VirtualHost>
<VirtualHost *:{{ http_port }}>
{{ slave_parameter.get('apache_custom_https', '') }}
</VirtualHost>
{%- set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
{%- set disable_no_cache_header = ('' ~ slave_parameter.get('disable-no-cache-request', '')).lower() in TRUE_VALUES -%}
{%- set disable_via_header = ('' ~ slave_parameter.get('disable-via-header', '')).lower() in TRUE_VALUES -%}
{%- set prefer_gzip = ('' ~ slave_parameter.get('prefer-gzip-encoding-to-backend', '')).lower() in TRUE_VALUES -%}
{%- set server_alias_list = slave_parameter.get('server-alias', '').split() -%}
{%- set enable_h2 = ('' ~ slave_parameter.get('enable-http2', 'true')).lower() in TRUE_VALUES -%}
{%- set ssl_proxy_verify = ('' ~ slave_parameter.get('ssl-proxy-verify', '')).lower() in TRUE_VALUES -%}
{%- set disabled_cookie_list = slave_parameter.get('disabled-cookie-list', '').split() -%}
{%- set https_only = ('' ~ slave_parameter.get('https-only', '')).lower() in TRUE_VALUES -%}
{%- set slave_type = slave_parameter.get('type', '') -%}
{%- set ssl_configuration_list = [('SSLCertificateFile', 'path_to_ssl_crt'),
('SSLCertificateKeyFile', 'path_to_ssl_key'),
('SSLCACertificateFile', 'path_to_ssl_ca_crt'),
('SSLCertificateChainFile', 'path_to_ssl_ca_crt')] -%}
<VirtualHost *:{{ https_port }}>
ServerName {{ slave_parameter.get('custom_domain') }}
ServerAlias {{ slave_parameter.get('custom_domain') }}
{%- for server_alias in server_alias_list %}
ServerAlias {{ server_alias }}
{% endfor %}
SSLEngine on
SSLProxyEngine on
{% if ssl_proxy_verify -%}
{% if 'ssl_proxy_ca_crt' in slave_parameter -%}
SSLProxyCACertificateFile {{ slave_parameter.get('path_to_ssl_proxy_ca_crt', '') }}
{% endif %}
SSLProxyVerify require
#SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
{% endif %}
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
{% if enable_h2 %}
Protocols h2 http/1.1
{% endif -%}
{% for key, value in ssl_configuration_list -%}
{% if value in slave_parameter -%}
{{ ' %s' % key }} {{ slave_parameter.get(value) }}
{% endif -%}
{% endfor -%}
# One Slave two logs
ErrorLog "{{ slave_parameter.get('error_log') }}"
LogLevel notice
LogFormat "%h %l %{REMOTE_USER}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog "{{ slave_parameter.get('access_log') }}" combined
# Rewrite part
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
{% if disable_via_header %}
Header unset Via
{% endif -%}
{% if disable_no_cache_header %}
RequestHeader unset Cache-Control
RequestHeader unset Pragma
{% endif -%}
{%- for disabled_cookie in disabled_cookie_list %}
{{' RequestHeader edit Cookie "(^%(disabled_cookie)s=[^;]*; |; %(disabled_cookie)s=[^;]*|^%(disabled_cookie)s=[^;]*$)" ""' % dict(disabled_cookie=disabled_cookie) }}
{% endfor -%}
{%- if prefer_gzip %}
RequestHeader edit Accept-Encoding "(^gzip,.*|.*, gzip,.*|.*, gzip$|^gzip$)" "gzip"
{% endif %}
{% if slave_type == 'zope' -%}
{% if 'default-path' in slave_parameter %}
RewriteRule ^/?$ {{ slave_parameter.get('default-path') }} [R=301,L]
{% endif -%}
# First, we check if we have a zope backend server
# If so, let's use Virtual Host Monster rewrite
# We suppose that Apache listens to 443 (even indirectly thanks to things like iptables)
RewriteRule ^/(.*)$ {{ slave_parameter.get('https-url', slave_parameter.get('url', '')) }}/VirtualHostBase/https//%{SERVER_NAME}:{{ slave_parameter.get('virtualhostroot-https-port', '443') }}/{{ slave_parameter.get('path', '') }}/VirtualHostRoot/$1 [L,P]
{% elif slave_type == 'redirect' -%}
RewriteRule (.*) {{ slave_parameter.get('https-url', slave_parameter.get('url', ''))}}$1 [R,L]
{% else -%}
{% if 'default-path' in slave_parameter %}
RewriteRule ^/?$ {{ slave_parameter.get('default-path') }} [R=301,L]
{% endif -%}
RewriteRule ^/(.*)$ {{ slave_parameter.get('https-url', slave_parameter.get('url', '')) }}/$1 [L,P]
{% endif -%}
</VirtualHost>
<VirtualHost *:{{ http_port }}>
ServerName {{ slave_parameter.get('custom_domain') }}
ServerAlias {{ slave_parameter.get('custom_domain') }}
{%- for server_alias in server_alias_list %}
ServerAlias {{ server_alias }}
{% endfor %}
SSLProxyEngine on
{% if ssl_proxy_verify -%}
{% if 'ssl_proxy_ca_crt' in slave_parameter -%}
SSLProxyCACertificateFile {{ slave_parameter.get('path_to_ssl_proxy_ca_crt', '') }}
{% endif %}
SSLProxyVerify require
#SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
{% endif %}
# Rewrite part
ProxyPreserveHost On
ProxyTimeout 600
{% if disable_via_header %}
Header unset Via
{% endif -%}
RewriteEngine On
# One Slave two logs
ErrorLog "{{ slave_parameter.get('error_log') }}"
LogLevel notice
LogFormat "%h %l %{REMOTE_USER}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog "{{ slave_parameter.get('access_log') }}" combined
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
{% if enable_h2 %}
Protocols h2 http/1.1
{% endif -%}
{% if disable_no_cache_header %}
RequestHeader unset Cache-Control
RequestHeader unset Pragma
{% endif -%}
{%- for disabled_cookie in disabled_cookie_list %}
{{' RequestHeader edit Cookie "(^%(disabled_cookie)s=[^;]*; |; %(disabled_cookie)s=[^;]*|^%(disabled_cookie)s=[^;]*$)" ""' % dict(disabled_cookie=disabled_cookie) }}
{% endfor -%}
{%- if prefer_gzip %}
RequestHeader edit Accept-Encoding "(^gzip,.*|.*, gzip,.*|.*, gzip$|^gzip$)" "gzip"
{% endif %}
# Next line is forbidden and people who copy it will be hanged short
{% if https_only -%}
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteCond %{SERVER_PORT} !^{{ https_port }}$
RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [NC,R,L]
{% elif slave_type == 'redirect' -%}
RewriteRule (.*) {{slave_parameter.get('url', '')}}$1 [R,L]
{% elif slave_type == 'zope' -%}
{% if 'default-path' in slave_parameter %}
RewriteRule ^/?$ {{ slave_parameter.get('default-path') }} [R=301,L]
{% endif -%}
# First, we check if we have a zope backend server
# If so, let's use Virtual Host Daemon rewrite
# We suppose that Apache listens to 80 (even indirectly thanks to things like iptables)
RewriteRule ^/(.*)$ {{ slave_parameter.get('url', '') }}/VirtualHostBase/http/%{SERVER_NAME}:{{ slave_parameter.get('virtualhostroot-http-port', '80') }}/{{ slave_parameter.get('path', '') }}/VirtualHostRoot/$1 [L,P]
{% else -%}
{% if 'default-path' in slave_parameter %}
RewriteRule ^/?$ {{ slave_parameter.get('default-path') }} [R=301,L]
{% endif -%}
RewriteRule ^/(.*)$ {{ slave_parameter.get('url', '') }}/$1 [L,P]
{% endif -%}
# If nothing exist : put a nice error
# ErrorDocument 404 /notfound.html
# Dadiboom
</VirtualHost>
{{ content }}
\ No newline at end of file
{% set url = slave_parameter.get('url') -%}
{% set https_url = slave_parameter.get('https-url', url) -%}
{% if url.startswith("http://") or url.startswith("https://") -%}
{% set upstream = url.split("/")[2] -%}
{% set https_upstream = https_url.split("/")[2] -%}
{% set protocol = url.split("/")[0] -%}
{% set https_protocol = https_url.split("/")[0] -%}
{% set proxy_pass = '%s//%s' % (protocol, slave_parameter.get('slave_reference')) -%}
{% set https_proxy_pass = '%s//https_%s' % (protocol, slave_parameter.get('slave_reference')) -%}
{%- set ssl_configuration_list = [('ssl_certificate', 'path_to_ssl_crt'),
('ssl_certificate_key', 'path_to_ssl_key')] -%}
upstream {{ slave_parameter.get('slave_reference') }} {
server {{ upstream }};
}
upstream https_{{ slave_parameter.get('slave_reference') }} {
server {{ https_upstream }};
}
server {
listen [{{ global_ipv6 }}]:{{ nginx_http_port }};
listen {{ local_ipv4 }}:{{ nginx_http_port }};
server_name {{ slave_parameter.get('custom_domain') }};
error_log {{ slave_parameter.get('error_log') }} error;
access_log {{ slave_parameter.get('access_log') }} custom;
location /pub {
push_stream_publisher;
push_stream_channels_path $arg_id;
# store messages in memory
push_stream_store_messages off;
# Message size limit
# client_max_body_size MUST be equal to client_body_buffer_size or
# you will be sorry.
client_max_body_size 16k;
client_body_buffer_size 16k;
}
location ~ /sub/(.*) {
# activate subscriber mode for this location
add_header "Access-Control-Allow-Origin" "*";
add_header 'Access-Control-Allow-Credentials' 'false';
add_header 'Access-Control-Allow-Methods' 'GET, HEAD, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since';
push_stream_subscriber eventsource;
# positional channel path
push_stream_channels_path $1;
# content-type
default_type "text/event-stream; charset=utf-8";
}
}
server {
listen [{{ global_ipv6 }}]:{{ nginx_https_port }} ssl;
listen {{ local_ipv4 }}:{{ nginx_https_port }} ssl;
server_name {{ slave_parameter.get('custom_domain') }};
error_log {{ slave_parameter.get('error_log') }} error;
access_log {{ slave_parameter.get('access_log') }} custom;
ssl on;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5';
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
{% for key, value in ssl_configuration_list -%}
{% if value in slave_parameter -%}
{{ ' %s' % key }} {{ slave_parameter.get(value) }};
{% endif -%}
{% endfor %}
location /pub {
push_stream_publisher;
push_stream_channels_path $arg_id;
# store messages in memory
push_stream_store_messages off;
# Message size limit
# client_max_body_size MUST be equal to client_body_buffer_size or
# you will be sorry.
client_max_body_size 16k;
client_body_buffer_size 16k;
}
location ~ /sub/(.*) {
# activate subscriber mode for this location
add_header "Access-Control-Allow-Origin" "*";
add_header 'Access-Control-Allow-Credentials' 'false';
add_header 'Access-Control-Allow-Methods' 'GET, HEAD, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since';
push_stream_subscriber eventsource;
# positional channel path
push_stream_channels_path $1;
# content-type
default_type "text/event-stream; charset=utf-8";
}
}
{% endif -%}
{% set url = slave_parameter.get('url') -%}
{% set https_url = slave_parameter.get('https-url', url) -%}
{% if url.startswith("http://") or url.startswith("https://") -%}
{% set upstream = url.split("/")[2] -%}
{% set https_upstream = https_url.split("/")[2] -%}
{% set protocol = url.split("/")[0] -%}
{% set https_protocol = https_url.split("/")[0] -%}
{% set proxy_pass = '%s//%s' % (protocol, slave_parameter.get('slave_reference')) -%}
{% set https_proxy_pass = '%s//https_%s' % (protocol, slave_parameter.get('slave_reference')) -%}
{%- set ssl_configuration_list = [('ssl_certificate', 'path_to_ssl_crt'),
('ssl_certificate_key', 'path_to_ssl_key')] -%}
upstream {{ slave_parameter.get('slave_reference') }} {
server {{ upstream }};
}
upstream https_{{ slave_parameter.get('slave_reference') }} {
server {{ https_upstream }};
}
server {
listen [{{ global_ipv6 }}]:{{ nginx_http_port }};
listen {{ local_ipv4 }}:{{ nginx_http_port }};
server_name {{ slave_parameter.get('custom_domain') }};
error_log {{ slave_parameter.get('error_log') }} error;
access_log {{ slave_parameter.get('access_log') }} custom;
location / {
proxy_pass {{ proxy_pass }};
proxy_set_header Host $host;
}
location ~ /api/kernels/ {
proxy_pass {{ proxy_pass }};
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
location ~ /terminals/ {
proxy_pass {{ proxy_pass }};
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
}
server {
listen [{{ global_ipv6 }}]:{{ nginx_https_port }} ssl;
listen {{ local_ipv4 }}:{{ nginx_https_port }} ssl;
server_name {{ slave_parameter.get('custom_domain') }};
error_log {{ slave_parameter.get('error_log') }} error;
access_log {{ slave_parameter.get('access_log') }} custom;
ssl on;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5';
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
{% for key, value in ssl_configuration_list -%}
{% if value in slave_parameter -%}
{{ ' %s' % key }} {{ slave_parameter.get(value) }};
{% endif -%}
{% endfor %}
location / {
proxy_pass {{ https_proxy_pass }};
proxy_set_header Host $host;
}
location ~ /api/kernels/ {
proxy_pass {{ https_proxy_pass }};
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
location ~ /terminals/ {
proxy_pass {{ https_proxy_pass }};
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
}
{% endif -%}
daemon off; # run in the foreground so supervisord can look after it
worker_processes $${nginx-configuration:worker_processes};
pid $${nginx-configuration:pid-file};
events {
worker_connections $${nginx-configuration:worker_connections};
# multi_accept on;
}
error_log $${nginx-configuration:error_log};
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_tokens off;
error_log $${nginx-configuration:error_log};
log_format custom '$remote_addr - $remote_user $time_local $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time';
access_log $${nginx-configuration:access_log} custom;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
default_type application/octet-stream;
ssl_certificate $${ca-frontend:cert-file};
ssl_certificate_key $${ca-frontend:key-file};
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
##
# Push stream Settings
##
push_stream_shared_memory_size 32m;
fastcgi_temp_path $${directory:varnginx} 1 2;
uwsgi_temp_path $${directory:varnginx} 1 2;
scgi_temp_path $${directory:varnginx} 1 2;
client_body_temp_path $${directory:varnginx} 1 2;
proxy_temp_path $${directory:varnginx} 1 2;
include $${nginx-configuration:slave-configuration-directory}/*.conf;
server {
listen [$${nginx-configuration:ip}]:$${nginx-configuration:plain_port};
listen $${nginx-configuration:local_ip}:$${nginx-configuration:plain_port};
## Serve an error 204 (No Content) for favicon.ico
location = /favicon.ico {
return 204;
}
location / {
root $${apache-directory:document-root};
index notfound.html;
}
}
server {
listen [$${nginx-configuration:ip}]:$${nginx-configuration:port} ssl;
listen $${nginx-configuration:local_ip}:$${nginx-configuration:port} ssl;
ssl on;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5';
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
## Serve an error 204 (No Content) for favicon.ico
location = /favicon.ico {
return 204;
}
location / {
root $${apache-directory:document-root};
index notfound.html;
}
}
}
<html>
<head>
<title>Instance not found</title>
</head>
<body>
<h1>This instance has not been found.</h1>
<p>If this error persists, please check your instance URL and status on SlapOS Master.</p>
</body>
</html>
{% set part_list = [] -%}
{% set slave_information_dict = {} -%}
# regroup slave information from all frontends
{%- for frontend, slave_list_raw in slave_information.iteritems() -%}
{% if slave_list_raw %}
{% set slave_list = json_module.loads(slave_list_raw) -%}
{% else %}
{% set slave_list = [] %}
{% endif %}
{% for slave_dict in slave_list -%}
{% set slave_reference = slave_dict.pop('slave-reference') %}
{% set log_access_url = slave_dict.pop('log-access', '') %}
{% set current_slave_dict = slave_information_dict.get(slave_reference, {}) %}
{% do current_slave_dict.update(slave_dict) -%}
{% set log_access_list = current_slave_dict.get('log-access-urls', []) %}
{% do log_access_list.append( frontend + ': ' + log_access_url) %}
{% do current_slave_dict.__setitem__(
'log-access-urls',
log_access_list
) %}
{% do current_slave_dict.__setitem__(
'replication_number',
current_slave_dict.get('replication_number', 0) + 1
) -%}
{% do slave_information_dict.__setitem__(slave_reference, current_slave_dict) -%}
{% endfor -%}
{% endfor %}
# Publish information for each slave
{% for slave_reference, slave_information in slave_information_dict.iteritems() %}
{% set publish_section_title = 'publish-%s' % slave_reference -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
-slave-reference = {{ slave_reference }}
log-access-url = {{ json_module.dumps(slave_information.pop('log-access-urls', 1000)) }}
{% for key, value in slave_information.iteritems() -%}
{{ key }} = {{ value }}
{% endfor -%}
{% endfor %}
[buildout]
parts =
{% for part in part_list %}
{{ ' %s' % part }}
{% endfor %}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
\ No newline at end of file
{% for slave, directory in slave_log_directory.iteritems() %}
Alias /{{slave}}/ {{directory}}/
<Directory {{directory}}>
Order Deny,Allow
Deny from env=AUTHREQUIRED
<Files ".??*">
Order Allow,Deny
Deny from all
</Files>
AuthType Basic
AuthName "Log Access {{slave}}"
AuthUserFile "{{ apache_configuration_directory + '/.' + slave.upper() + '.htaccess'}}"
Require user {{slave.upper()}}
Options Indexes FollowSymLinks
Satisfy all
</Directory>
{% endfor %}
#
#
# Process Records Config File
#
# <RECORD-TYPE> <NAME> <TYPE> <VALUE (till end of line)>
#
# RECORD-TYPE: CONFIG, LOCAL
# NAME: name of variable
# TYPE: INT, STRING, FLOAT
# VALUE: Initial value for record
#
#
# *NOTE*: All options covered in this file should be documented in the
# administration guide or the addendum:
#
#
##############################################################################
#
# System Variables
#
##############################################################################
CONFIG proxy.config.proxy_name STRING {{ ats_configuration['hostname'] }}
CONFIG proxy.config.local_state_dir {{ ats_directory['local-state'] }}
CONFIG proxy.config.config_dir STRING {{ ats_directory['configuration'] }}
CONFIG proxy.config.bin_path STRING {{ ats_directory['bin_path'] }}
CONFIG proxy.config.proxy_binary_opts STRING -M
CONFIG proxy.config.env_prep STRING example_prep.sh
CONFIG proxy.config.alarm_email STRING nobody
CONFIG proxy.config.syslog_facility STRING LOG_DAEMON
CONFIG proxy.config.output.logfile STRING traffic.out
CONFIG proxy.config.snapshot_dir STRING snapshots
CONFIG proxy.config.system.mmap_max INT 2097152
##############################################################################
#
# Main threads configuration (worker threads). Also see configurations for
# SSL threads, disk I/O threads and task threads in their respective areas.
#
##############################################################################
CONFIG proxy.config.exec_thread.autoconfig INT 1
CONFIG proxy.config.exec_thread.autoconfig.scale FLOAT 1.5
CONFIG proxy.config.exec_thread.limit INT 2
CONFIG proxy.config.accept_threads INT 1
##############################################################################
#
# Local Manager
#
##############################################################################
CONFIG proxy.config.admin.admin_user STRING admin
CONFIG proxy.config.admin.number_config_bak INT 3
CONFIG proxy.config.admin.user_id STRING {{ '#%s' % os_module.geteuid() }}
##############################################################################
#
# Process Manager
#
##############################################################################
CONFIG proxy.config.admin.autoconf_port INT {{ ats_configuration.get('autoconf-port', '8083') }}
CONFIG proxy.config.process_manager.mgmt_port INT {{ ats_configuration.get('mgmt-port', '8084') }}
##############################################################################
#
# In order to only bind a specific IP, use the following config, as in
# the example below. Note - this can contain two addresses, one for IPv4
# sockets and one for IPv6 sockets.
#
##############################################################################
LOCAL proxy.local.incoming_ip_to_bind STRING {{ ats_configuration['local-ip'] }}
#LOCAL proxy.local.outgoing_ip_to_bind STRING 10.0.143.32
#LOCAL proxy.local.incoming_ip_to_bind STRING 192.168.101.17 fc07:192:168:101::17
##############################################################################
#
# Alarm Configuration
#
##############################################################################
# execute alarm as "<abs_path>/<bin> "<MSG_STRING_FROM_PROXY>""
CONFIG proxy.config.alarm.bin STRING example_alarm_bin.sh
CONFIG proxy.config.alarm.abs_path STRING NULL
##############################################################################
#
# HTTP Engine
#
##############################################################################
##########
# basics #
##########
# The server ports are listed here. These are separated by spaces or commas.
# Each port is a colon separated list of values, which must include a
# port number. The order is irrelevant. Other options are
# ipv4 - Use IPv4 (default)
# ipv6 - Use IPv6
# tr-in - Transparent inbound.
# tr-out - Transparent outbound.
# tr-full - Fully transparent (inbound and outbound).
# tr-pass - Transparently Pass-through non-HTTP traffic (in conjuction with tr-in).
# ssl - SSL terminated port.
# blind - Blind tunnel port (CONNECT only)
# ip-in=[addr] - Bind inbound IP address (listen for client).
# ip-out=[addr] - Bind outbound IP address (connect to origin server).
# ip-resolve=[style] - Set the IP resolution style.
#
# Note - address types must agree with each other and the ipv4/ipv6
# option if specified. IPv6 addresses must be enclosed in brackets.
# ip-out can be repeated as long as each address is a different
# family. If ip-in is specified as an IPv6 address, the port is
# forced to IPv6. Transparent ports cannot be bound to an IP
# address on the transparent side.
#
# The '=' is optional for any option with a value.
#
# Example 1: Port 8080 IPv6 inbound transparent, and port 80 IPv4
# "8080:ipv6:tr-in 80"
#
# Example 2: Listen on standard http and https ports for IPv4 and IPv6,
# fully transparent on the http ports. Also provide an non-transparent
# port at address 192.168.1.56 on port 8080.
# "80:ipv4:tr-full tr-full:80:ipv6 443:ipv4:ssl 443:ssl:ipv6 ip-in=192.168.1.56:8080"
#
CONFIG proxy.config.http.server_ports STRING {{ ats_configuration['local-ip'] + ':' + ats_configuration['input-port'] }}
# Ports on the origin server to which a blind tunnel may connect.
CONFIG proxy.config.http.connect_ports STRING 443 563
# The via settings have four values
# 0 - Do not modify / set this via header
# 1 - Update the via, with normal verbosity
# 2 - Update the via, with higher verbosity
# 3 - Update the via, with highest verbosity
CONFIG proxy.config.http.insert_request_via_str INT 2
CONFIG proxy.config.http.insert_response_via_str INT 2
# Insert a Server: header, this has three values
# 0 - Don't add or modify the Server: header
# 1 - Add a Server: header
# 2 - Only add a Server: header if one doesn't exist already
CONFIG proxy.config.http.response_server_enabled INT 1
CONFIG proxy.config.http.insert_age_in_response INT 1
CONFIG proxy.config.http.enable_url_expandomatic INT 0
CONFIG proxy.config.http.no_dns_just_forward_to_parent INT 0
CONFIG proxy.config.http.uncacheable_requests_bypass_parent INT 1
CONFIG proxy.config.http.keep_alive_enabled_in INT 1
CONFIG proxy.config.http.keep_alive_enabled_out INT 1
CONFIG proxy.config.http.chunking_enabled INT 1
# send http11 requests:
# 0 - Never
# 1 - Always
# 2 - if the server has returned http1.1 before
# 3 - if the client request is 1.1 & the server has returned 1.1 before
# If use_client_addr is set to 1, options 2 and 3 cause the proxy to use
# the client HTTP version for upstream requests.
CONFIG proxy.config.http.send_http11_requests INT 1
# Share server connections
# 0 - Never
# 1 - Share, with a single global connection pool
# 2 - Share, with a connection pool per worker thread
CONFIG proxy.config.http.share_server_sessions INT 2
##########################
# HTTP referer filtering #
##########################
CONFIG proxy.config.http.referer_filter INT 0
CONFIG proxy.config.http.referer_format_redirect INT 0
CONFIG proxy.config.http.referer_default_redirect STRING http://www.example.com/
##############################
# parent proxy configuration #
##############################
CONFIG proxy.config.http.parent_proxy_routing_enable INT 0
CONFIG proxy.config.http.parent_proxy.retry_time INT 300
# Parent fail threshold is the number of request that must
# fail within the retry window for the parent to be marked
# down
CONFIG proxy.config.http.parent_proxy.fail_threshold INT 10
CONFIG proxy.config.http.parent_proxy.total_connect_attempts INT 4
CONFIG proxy.config.http.parent_proxy.per_parent_connect_attempts INT 2
CONFIG proxy.config.http.parent_proxy.connect_attempts_timeout INT 30
CONFIG proxy.config.http.forward.proxy_auth_to_parent INT 0
###################################
# HTTP connection timeouts (secs) #
###################################
# out: proxy -> origin server connection
# in : ua -> proxy connection
CONFIG proxy.config.http.keep_alive_no_activity_timeout_in INT 115
CONFIG proxy.config.http.keep_alive_no_activity_timeout_out INT 120
CONFIG proxy.config.http.transaction_no_activity_timeout_in INT 300
CONFIG proxy.config.http.transaction_no_activity_timeout_out INT 300
CONFIG proxy.config.http.transaction_active_timeout_in INT 900
CONFIG proxy.config.http.transaction_active_timeout_out INT 0
CONFIG proxy.config.http.accept_no_activity_timeout INT 120
CONFIG proxy.config.http.background_fill_active_timeout INT 60
CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0.5
##################################
# origin server connect attempts #
##################################
CONFIG proxy.config.http.connect_attempts_max_retries INT 1
CONFIG proxy.config.http.connect_attempts_max_retries_dead_server INT 3
CONFIG proxy.config.http.connect_attempts_rr_retries INT 3
CONFIG proxy.config.http.connect_attempts_timeout INT 160
CONFIG proxy.config.http.post_connect_attempts_timeout INT 1800
CONFIG proxy.config.http.down_server.cache_time INT 300
CONFIG proxy.config.http.down_server.abort_threshold INT 10
##################################
# congestion control #
##################################
CONFIG proxy.config.http.congestion_control.enabled INT 0
#############################
# negative response caching #
#############################
CONFIG proxy.config.http.negative_caching_enabled INT 0
CONFIG proxy.config.http.negative_caching_lifetime INT 1800
#########################
# proxy users variables #
#########################
CONFIG proxy.config.http.anonymize_remove_from INT 0
CONFIG proxy.config.http.anonymize_remove_referer INT 0
CONFIG proxy.config.http.anonymize_remove_user_agent INT 0
CONFIG proxy.config.http.anonymize_remove_cookie INT 0
CONFIG proxy.config.http.anonymize_remove_client_ip INT 0
CONFIG proxy.config.http.anonymize_insert_client_ip INT 1
CONFIG proxy.config.http.anonymize_other_header_list STRING NULL
CONFIG proxy.config.http.insert_squid_x_forwarded_for INT 1
############
# security #
############
CONFIG proxy.config.http.push_method_enabled INT 0
# ###################################
# # HTTP Quick filtering (security) #
# ###################################
# this functionality is moved to ip_allow.config
#################
# cache control #
#################
CONFIG proxy.config.http.cache.http INT 1
# Enabling this setting allows the proxy to cache empty documents. This currently
# requires that the response has a Content-Length: header, with a value of "0".
CONFIG proxy.config.http.cache.allow_empty_doc INT 0
CONFIG proxy.config.http.cache.ignore_client_no_cache INT 0
CONFIG proxy.config.http.cache.ims_on_client_no_cache INT 0
CONFIG proxy.config.http.cache.ignore_server_no_cache INT 0
CONFIG proxy.config.http.cache.ignore_client_cc_max_age INT 0
CONFIG proxy.config.http.normalize_ae_gzip INT 0
# cache responses to cookies has 5 options:
# 0 - do not cache any responses to cookies
# 1 - cache for any content-type
# 2 - cache only for image types
# 3 - cache for all but text content-types
# 4 - cache for all but text content-types except OS response
# without "Set-Cookie" or with "Cache-Control: public"
# See also cache-responses-to-cookies in cache.config.
CONFIG proxy.config.http.cache.cache_responses_to_cookies INT 1
CONFIG proxy.config.http.cache.ignore_authentication INT 0
CONFIG proxy.config.http.cache.cache_urls_that_look_dynamic INT 1
CONFIG proxy.config.http.cache.enable_default_vary_headers INT 0
# when_to_revalidate has 5 options:
# 0 - default. use cache directives or heuristic
# 1 - stale if heuristic
# 2 - always stale (always revalidate)
# 3 - never stale
# 4 - always revalidate if request is conditional, else default is used
CONFIG proxy.config.http.cache.when_to_revalidate INT 0
# Some old MSIE browsers don't send no-cache headers to
# reverse proxies or transparent caches, this variable controls
# when to add no-cache headers to MSIE requests:
# -1 - no-cache is never added, stats are not updated
# 0 - default; no-cache not added to MSIE requests
# 1 - no-cache added to IMS MSIE requests
# 2 - no-cache added to all MSIE requests
CONFIG proxy.config.http.cache.when_to_add_no_cache_to_msie_requests INT -1
# required headers: three options:
# 0 - No required headers to make document cachable
# 1 - "Last-Modified:", "Expires:", or "Cache-Control: max-age" required
# 2 - explicit lifetime required, "Expires:" or "Cache-Control: max-age"
CONFIG proxy.config.http.cache.required_headers INT 2
CONFIG proxy.config.http.cache.max_stale_age INT 604800
CONFIG proxy.config.http.cache.range.lookup INT 1
########################
# heuristic expiration #
########################
CONFIG proxy.config.http.cache.heuristic_min_lifetime INT 3600
CONFIG proxy.config.http.cache.heuristic_max_lifetime INT 86400
CONFIG proxy.config.http.cache.heuristic_lm_factor FLOAT 0.10
CONFIG proxy.config.http.cache.fuzz.time INT 240
CONFIG proxy.config.http.cache.fuzz.probability FLOAT 0.005
#########################################
# dynamic content & content negotiation #
#########################################
CONFIG proxy.config.http.cache.vary_default_text STRING NULL
CONFIG proxy.config.http.cache.vary_default_images STRING NULL
CONFIG proxy.config.http.cache.vary_default_other STRING NULL
##############################################################
# The HTTP stats are expensive, turn off you don't need them #
##############################################################
CONFIG proxy.config.http.enable_http_stats INT 1
##############################################################################
#
# Customizable User Response Pages
#
##############################################################################
# 1 - enable customizable user response pages in only the "default" directory
# 2 - enable language-targeted user response pages
CONFIG proxy.config.body_factory.enable_customizations INT 1
CONFIG proxy.config.body_factory.enable_logging INT 0
# 0 - never suppress generated responses
# 1 - always suppress generated responses
# 2 - suppress responses for intercepted traffic
CONFIG proxy.config.body_factory.response_suppression_mode INT 0
##############################################################################
#
# Net Subsystem
#
##############################################################################
CONFIG proxy.config.net.connections_throttle INT 30000
# Enable defer accept / accept filtering. On Linux, this is a timeout, sec.
CONFIG proxy.config.net.defer_accept INT 45
##############################################################################
#
# Cluster Subsystem
#
##############################################################################
# cluster type requires restart to change
# 1 is full clustering, 2 is mgmt only, 3 is no clustering
LOCAL proxy.local.cluster.type INT 3
CONFIG proxy.config.cluster.cluster_port INT 8086
CONFIG proxy.config.cluster.rsport INT 8088
CONFIG proxy.config.cluster.mcport INT 8089
CONFIG proxy.config.cluster.mc_group_addr STRING 224.0.1.37
CONFIG proxy.config.cluster.mc_ttl INT 1
CONFIG proxy.config.cluster.log_bogus_mc_msgs INT 1
CONFIG proxy.config.cluster.ethernet_interface STRING lo
##############################################################################
#
# Cache
#
##############################################################################
CONFIG proxy.config.cache.permit.pinning INT 0
# default the ram cache size to AUTO_SIZE (-1) based on cache size
# (approximately 10 MB of RAM cache per GB of disk cache)
# alternatively, set to a fixed value such as 21474836480 (20GB)
CONFIG proxy.config.cache.ram_cache.size INT {{ ats_configuration.get('ram-cache-size', '1G') }}
# Replacement algorithm
# 0 : Clocked Least Frequently Used by Size (CLFUS) w/optional compression
# 1 : LRU w/o optional compression - trivially simple
CONFIG proxy.config.cache.ram_cache.algorithm INT 0
# Filter inserts into the RAM cache to ensure that they have been seen at
# least once. For LRU, this provides scan resistance. Note that CLFUS
# already requires that a document have history before it is inserted, so
# for CLFUS, setting this option means that a document must be seen three
# times before it is added to the RAM cache.
CONFIG proxy.config.cache.ram_cache.use_seen_filter INT 0
# Compress the content of the ram cache:
# 0 : no compression
# 1 : fastlz (extremely fast, relatively low compression)
# 2 : libz (moderate speed, reasonable compression)
# 3 : liblzma (very slow, high compression)
# NOTE: compression runs on task threads. To use more cores for
# compression, increase proxy.config.task_threads.
CONFIG proxy.config.cache.ram_cache.compress INT 0
# The maximum number of alternates that are allowed for any given URL.
# It is not possible to strictly enforce this if the variable
# 'proxy.config.cache.vary_on_user_agent' is set to 1.
# The default value for 'proxy.config.cache.vary_on_user_agent' is 0.
# (0 disables the maximum number of alts check)
CONFIG proxy.config.cache.limits.http.max_alts INT 5
# The target size of a contiguous fragment on disk.
# Acceptable values are powers of 2, e.g. 65536, 131072, 262144, 524288, 1048576, 2097152.
# Larger could waste memory on slow connections, smaller could waste seeks.
CONFIG proxy.config.cache.target_fragment_size INT 1048576
# The maximum size of a document that will be stored in the cache.
# (0 disables the maximum document size check)
CONFIG proxy.config.cache.max_doc_size INT 0
# enable the cache to read from an object while it is being added to the cache
CONFIG proxy.config.cache.enable_read_while_writer INT 0
# How many I/O threads to allocate per disk (spindle). Be aware that RAID
# disks would show up to TS as a single spindle.
CONFIG proxy.config.cache.threads_per_disk INT 8
# Time (in ms) to delay until retrying to acquire a cache lock. Setting
# this low can reduce latencies in some cases, but can consume more CPU.
# If you experience CPU spinning, try increasing this setting.
CONFIG proxy.config.cache.mutex_retry_delay INT 2
# The interim storage disks. Must use raw disks.
# Only support at most 8 interim disks now. e.g.
# proxy.config.cache.interim.storage STRING /dev/sda /dev/sdb/
LOCAL proxy.config.cache.interim.storage STRING NULL
##############################################################################
#
# DNS
#
##############################################################################
CONFIG proxy.config.dns.search_default_domains INT 0
CONFIG proxy.config.dns.splitDNS.enabled INT 0
CONFIG proxy.config.dns.max_dns_in_flight INT 2048
# Additional URL expansions for http DNS lookup
CONFIG proxy.config.dns.url_expansions STRING NULL
CONFIG proxy.config.dns.round_robin_nameservers INT 0
CONFIG proxy.config.dns.nameservers STRING NULL
CONFIG proxy.config.dns.resolv_conf STRING /etc/resolv.conf
# This provides additional resilience against DNS forgery, particularly in
# forward or transparent proxies, but requires that the resolver populates
# the queries section of the response properly.
CONFIG proxy.config.dns.validate_query_name INT 0
##############################################################################
#
# HostDB
#
##############################################################################
# in entries, may not be changed while running
# note that in order to increase hostdb.size, hostdb.storage_size should
# also be increase. These are best guesses, you will have to monitor this.
CONFIG proxy.config.hostdb.size INT 120000
CONFIG proxy.config.hostdb.storage_size INT 32M
# ttl modes:
# 0 = obey
# 1 = ignore
# 2 = min(X,ttl)
# 3 = max(X,ttl)
CONFIG proxy.config.hostdb.ttl_mode INT 0
# in minutes...
CONFIG proxy.config.hostdb.timeout INT 1440
# round-robin addresses for single clients
# (can cause authentication problems)
CONFIG proxy.config.hostdb.strict_round_robin INT 0
##############################################################################
#
# Logging Config
#
##############################################################################
# possible values for logging_enabled:
# 0: no logging at all
# 1: log errors only
# 2: log transactions only
# 3: full logging (errors + transactions)
CONFIG proxy.config.log.logging_enabled INT 3
CONFIG proxy.config.log.max_secs_per_buffer INT 5
CONFIG proxy.config.log.max_space_mb_for_logs INT 25000
CONFIG proxy.config.log.max_space_mb_for_orphan_logs INT 25
CONFIG proxy.config.log.max_space_mb_headroom INT 1000
CONFIG proxy.config.log.hostname STRING localhost
CONFIG proxy.config.log.logfile_dir STRING {{ ats_directory['log'] }}
CONFIG proxy.config.log.logfile_perm STRING rw-r--r--
CONFIG proxy.config.log.custom_logs_enabled INT 0
CONFIG proxy.config.log.squid_log_enabled INT 1
CONFIG proxy.config.log.squid_log_is_ascii INT 0
CONFIG proxy.config.log.squid_log_name STRING squid
CONFIG proxy.config.log.squid_log_header STRING NULL
CONFIG proxy.config.log.common_log_enabled INT 0
CONFIG proxy.config.log.common_log_is_ascii INT 1
CONFIG proxy.config.log.common_log_name STRING common
CONFIG proxy.config.log.common_log_header STRING NULL
CONFIG proxy.config.log.extended_log_enabled INT 0
CONFIG proxy.config.log.extended_log_is_ascii INT 0
CONFIG proxy.config.log.extended_log_name STRING extended
CONFIG proxy.config.log.extended_log_header STRING NULL
CONFIG proxy.config.log.extended2_log_enabled INT 0
CONFIG proxy.config.log.extended2_log_is_ascii INT 1
CONFIG proxy.config.log.extended2_log_name STRING extended2
CONFIG proxy.config.log.extended2_log_header STRING NULL
CONFIG proxy.config.log.separate_icp_logs INT 0
CONFIG proxy.config.log.separate_host_logs INT 0
# Log collation allows you to do "remote logging"
LOCAL proxy.local.log.collation_mode INT 0
CONFIG proxy.config.log.collation_host STRING NULL
CONFIG proxy.config.log.collation_port INT 8085
CONFIG proxy.config.log.collation_secret STRING foobar
CONFIG proxy.config.log.collation_host_tagged INT 0
CONFIG proxy.config.log.collation_retry_sec INT 5
CONFIG proxy.config.log.rolling_enabled INT 1
CONFIG proxy.config.log.rolling_interval_sec INT 86400
CONFIG proxy.config.log.rolling_offset_hr INT 0
CONFIG proxy.config.log.rolling_size_mb INT 10
CONFIG proxy.config.log.auto_delete_rolled_files INT 1
CONFIG proxy.config.log.sampling_frequency INT 1
##############################################################################
#
# Reverse Proxy
#
##############################################################################
CONFIG proxy.config.reverse_proxy.enabled INT 1
CONFIG proxy.config.header.parse.no_host_url_redirect STRING NULL
##############################################################################
#
# URL Remap Rules
#
##############################################################################
CONFIG proxy.config.url_remap.default_to_server_pac INT 0
CONFIG proxy.config.url_remap.default_to_server_pac_port INT -1
# To enable forward proxy, you must turn off remap_required
CONFIG proxy.config.url_remap.remap_required INT 1
# Pristine host header is the "original" (request) header. Make sure your
# origin expects them in reverse proxy.
CONFIG proxy.config.url_remap.pristine_host_hdr INT 1
##############################################################################
#
# SSL Termination
#
##############################################################################
# The number of SSL threads is a multiplier of number of CPUs and
# proxy.config.exec_thread.autoconfig.scale by default. You can
# override that here (set it to a non-zero value).
CONFIG proxy.config.ssl.number.threads INT 0
# The following variables control SSL protocols.
CONFIG proxy.config.ssl.SSLv2 INT 0
CONFIG proxy.config.ssl.SSLv3 INT 0
CONFIG proxy.config.ssl.TLSv1 INT 1
CONFIG proxy.config.ssl.TLSv1_1 INT 1
CONFIG proxy.config.ssl.TLSv1_2 INT 1
# The following two variables control the Cipher Suite traffic Server
# uses for HTTPS connnections and whether to prefer the client
# selected (default) or the server selected
# Our default SSL Cipher Suite tries to be reasonably fast and strong.
CONFIG proxy.config.ssl.server.cipher_suite STRING ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:HIGH:!aNULL:!MD5
CONFIG proxy.config.ssl.server.honor_cipher_order INT 1
# Control if SSL should perform content compression or not
CONFIG proxy.config.ssl.compression INT 0
# Client certification level should be:
# 0 no client certificates
# 1 client certificates optional
# 2 client certificates required
CONFIG proxy.config.ssl.client.certification_level INT 0
# Server cert chain filename is the name of the global cert chain file
# that is added to every cert in ssl_multicert.config. This file is only
# loaded if there are configurations in ssl_multicert.config.
CONFIG proxy.config.ssl.server.cert_chain.filename STRING NULL
# This is the path that SSL certificates files are relative to. Certificate
# names specified in ssl_multicert.config will be located relative to this path.
CONFIG proxy.config.ssl.server.cert.path STRING etc/trafficserver
# If any private key is not contained in the certificate file, you must
# fill in the private key path. Private key names specified in
# ssl_multicert.config will be located relative to this path.
CONFIG proxy.config.ssl.server.private_key.path STRING etc/trafficserver
# The CA file name and path are the
# certificate authority certificate that
# client certificates will be verified against.
CONFIG proxy.config.ssl.CA.cert.filename STRING NULL
CONFIG proxy.config.ssl.CA.cert.path STRING etc/trafficserver
################################
# client related configuration #
################################
CONFIG proxy.config.ssl.client.verify.server INT 0
CONFIG proxy.config.ssl.client.cert.filename STRING NULL
CONFIG proxy.config.ssl.client.cert.path STRING etc/trafficserver
# Fill in private key file and path only if the client's
# private key is not contained in the client certificate file.
CONFIG proxy.config.ssl.client.private_key.filename STRING NULL
CONFIG proxy.config.ssl.client.private_key.path STRING etc/trafficserver
# The CA file name and path are the
# certificate authority certificate that
# server certificates will be verified against.
CONFIG proxy.config.ssl.client.CA.cert.filename STRING NULL
CONFIG proxy.config.ssl.client.CA.cert.path STRING etc/trafficserver
##############################################################################
#
# ICP Configuration. NOTE! ICP is currently broken NOTE!
#
##############################################################################
# icp modes
# enabled=0 ICP disabled
# enabled=1 Allow receive of ICP queries
# enabled=2 Allow send/receive of ICP queries
CONFIG proxy.config.icp.enabled INT 0
CONFIG proxy.config.icp.icp_interface STRING NULL
CONFIG proxy.config.icp.icp_port INT 3130
CONFIG proxy.config.icp.multicast_enabled INT 0
CONFIG proxy.config.icp.query_timeout INT 2
##############################################################################
#
# Scheduled Update Configuration
#
##############################################################################
CONFIG proxy.config.update.enabled INT 0
CONFIG proxy.config.update.force INT 0
CONFIG proxy.config.update.retry_count INT 10
CONFIG proxy.config.update.retry_interval INT 2
CONFIG proxy.config.update.concurrent_updates INT 100
##############################################################################
#
# Socket send/recv buffer sizes (0 == don't call setsockopt() )
#
##############################################################################
# out: proxy -> os connection
# in : ua -> proxy connection
CONFIG proxy.config.net.sock_send_buffer_size_in INT 262144
CONFIG proxy.config.net.sock_recv_buffer_size_in INT 0
CONFIG proxy.config.net.sock_send_buffer_size_out INT 0
CONFIG proxy.config.net.sock_recv_buffer_size_out INT 0
##############################################################################
#
# User Overridden Configurations Below
#
##############################################################################
CONFIG proxy.config.core_limit INT -1
##############################################################################
#
# Debugging
#
##############################################################################
# Uses a regular expression to match the debugging topic name, performance
# will be affected!
CONFIG proxy.config.diags.debug.enabled INT 0
CONFIG proxy.config.diags.debug.tags STRING http.*|dns.*
# Great for tracking down memory leaks, but you need to use the
# ink allocators
CONFIG proxy.config.dump_mem_info_frequency INT 0
# Log the source code location of diagnostic messages.
CONFIG proxy.config.diags.show_location INT 0
##############################################################################
#
# Configuration for Reclaimable InkFreeList memory pool
#
# NOTE: The following options are meaningfull only when Traffic Server is
# compiled with the following option to configure:
#
# --enable-reclaimable-freelist
#
##############################################################################
CONFIG proxy.config.allocator.enable_reclaim INT 1
# The value of reclaim_factor should be in the 0.0 to 1.0 range. Allocators
# use it to calculate size of unused memory, which is used to determine when
# to reclaim memory. The larger the value, the more aggressive reclaims.
CONFIG proxy.config.allocator.reclaim_factor FLOAT 0.300000
# Allocator will reclaim memory only when it continuously satisfy the reclaim
# condition for max_overage continuous checks.
CONFIG proxy.config.allocator.max_overage INT 3
# For debugging, enable debug_filter, which is a bit-map with these fields:
# bit 0: reclaim memory in ink_freelist_new
# bit 1: allocate memory from partial-free Chunks(if exist) or OS
CONFIG proxy.config.allocator.debug_filter INT 0
##############################################################################
#
# Slow Log
#
##############################################################################
# Log any request that takes more then x number of milliseconds, needs
# to be > 0 to be enabled
CONFIG proxy.config.http.slow.log.threshold INT 0
##############################################################################
#
# Thread pool for "misc" tasks, plugins etc. 2 is a good minimum.
#
##############################################################################
CONFIG proxy.config.task_threads INT 2
{{ ats_configuration.get("cache-path") }} {{ ats_configuration.get("disk-cache-size") }} volume={{ ats_configuration.get("hostname") }}
\ No newline at end of file
#!${dash-output:dash}
{{ content }}
\ No newline at end of file
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment