Skip to content

slapos
slapos
Commit dddb7942 authored by Kazuhiko Shiozaki's avatar Kazuhiko Shiozaki
Browse files
Options

Merge branch 'erp5-component' into erp5

parents 5cf88baf 7ce4fdd6
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 2572 additions and 317 deletions
.gitignore
View file @ dddb7942
......@@ -9,3 +9,5 @@ eggs/
parts/
slapos.cookbook.egg-info
.*.swp
*~
\#*\#
\ No newline at end of file
CHANGES.txt
View file @ dddb7942
Changes
=======
0.78.1 (2013-05-31)
0.78.4.dev (2013-07-18)
-----------------------
0.78.3 (2013-07-18)
-------------------
* slapos.cookbook:publish: Add support to publish information for slaves. [Cedric Le Ninivin]
0.78.2 (2013-07-18)
-------------------
* Fix slapos.cookbook:request: Add backward compatiblity about getInstanceGuid(). [Cedric de Saint Martin]
* slapos.cookbook:check_* promises: Add timeout to curl that is not otherwise killed by slapos promise subsystem. [Cedric de Saint Martin]
* Cloudooo: Allow any environment variables. [Yusei Tahara]
* ERP5: disable MariaDB query cache completely by 'query_cache_type = 0' for ERP5. [Kazuhiko Shiozaki]
* ERP5: enable haproxy admin socket and install haproxyctl script. [Kazuhiko Shiozaki]
* ERP5: increase the maximum number of open file descriptors before starting mysqld. [Kazuhiko Shiozaki]
* python 2.7: updated to 2.7.5 [Cedric de Saint Martin]
0.78.1 (2013-05-31)
-------------------
* Add boinc recipe: Allow to deploy an empty BOINC project. [Alain Takoudjou]
* Add boinc.app recipe: Allow to deploy and update a BOINC application into existing BOINC server instance . [Alain Takoudjou]
* Add boinc.client recipe: Allow to deploy a BOINC Client instance on SlapOS. [Alain Takoudjou]
......@@ -15,10 +34,6 @@ Changes
* Add trac recipe: for deploying Trac and manage project with support of SVN and GIT. [Alain Takoudjou]
* Add bonjourgrid recipe: for deploying BonjourGrid Master and submit BOINC or Condor project. [Alain Takoudjou]
* Add bonjourgrid.client recipe: for deploying BonjourGrid Worker instance and execute BOINC or Condor Jobs. [Alain Takoudjou]
* Cloudooo: Allow any environment variables. [Yusei Tahara]
* ERP5: disable MariaDB query cache completely by 'query_cache_type = 0' for ERP5. [Kazuhiko Shiozaki]
* ERP5: enable haproxy admin socket and install haproxyctl script. [Kazuhiko Shiozaki]
* ERP5: increase the maximum number of open file descriptors before starting mysqld. [Kazuhiko Shiozaki]
0.78.0 (2013-04-28)
-------------------
......
component/apache/buildout.cfg
View file @ dddb7942
......@@ -120,8 +120,9 @@ make-targets =
[apache-2.2]
# inspired on http://old.aclark.net/team/aclark/blog/a-lamp-buildout-for-wordpress-and-other-php-apps/
recipe = slapos.recipe.cmmi
url = http://mir2.ovh.net/ftp.apache.org/dist//httpd/httpd-2.2.24.tar.gz
md5sum = 64a3392018ad60583209a16d728180d3
version = 2.2.25
url = http://mir2.ovh.net/ftp.apache.org/dist/httpd/httpd-${:version}.tar.bz2
md5sum = 9ebe3070c0bb4311f21a0cd0e34f0045
patch-options = -p1
configure-options = --disable-static
--enable-authn-alias
......@@ -182,6 +183,8 @@ environment =
recipe = slapos.recipe.cmmi
url = http://sourceforge.net/projects/mod-antiloris/files/mod_antiloris-0.4.tar.bz2/download
md5sum = 66862bf10e9be3a023e475604a28a0b4
depends =
${apache-2.2:version}
configure-command = ${apache-2.2:location}/bin/apxs
configure-options = -c mod_antiloris.c
make-binary = ${:configure-command}
......
component/coreutils/buildout.cfg
View file @ dddb7942
......@@ -2,7 +2,7 @@
extends =
../xz-utils/buildout.cfg
parts =
coreutils
coreutils-output
[coreutils]
recipe = slapos.recipe.cmmi
......@@ -13,3 +13,13 @@ configure-options =
environment =
PATH=${xz-utils:location}/bin:%(PATH)s
LDFLAGS =-Wl,--as-needed
[coreutils-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${:test} -x ${:test} -a -x ${:cat} -a -x ${:rm}
test = ${coreutils:location}/bin/test
cat = ${coreutils:location}/bin/cat
rm = ${coreutils:location}/bin/rm
component/dash/buildout.cfg
View file @ dddb7942
[buildout]
extends =
../coreutils/buildout.cfg
parts = dash
parts = dash-output
[dash]
recipe = slapos.recipe.cmmi
......@@ -10,3 +12,11 @@ configure-options =
--disable-static
--disable-fnmatch
--disable-glob
[dash-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${coreutils-output:test} -x ${:dash}
dash = ${dash:location}/bin/dash
component/dcron/buildout.cfg
View file @ dddb7942
[buildout]
parts = dcron
extends =
../coreutils/buildout.cfg
parts = dcron-output
[dcron-patch-nonroot]
recipe = hexagonit.recipe.download
......@@ -18,3 +21,12 @@ patches =
patch-options = -p1
make-options =
PREFIX=${buildout:parts-directory}/${:_buildout_section_name_}
[dcron-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${coreutils-output:test} -x ${:crond} -a -x ${:crontab}
crond = ${dcron:location}/sbin/crond
crontab = ${dcron:location}/bin/crontab
component/dropbear/buildout.cfg
View file @ dddb7942
......@@ -7,9 +7,10 @@
[buildout]
extends =
../zlib/buildout.cfg
../coreutils/buildout.cfg
parts =
dropbear
dropbear-output
[dropbear-userspace-patch]
recipe = hexagonit.recipe.download
......@@ -63,3 +64,12 @@ patches=
patch-options=
-p1
[dropbear-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${coreutils-output:test} -x ${:ssh} -a -x ${:keygen}
ssh = ${dropbear:location}/bin/dbclient
keygen = ${dropbear:location}/bin/dropbearkey
component/ffmpeg/buildout.cfg
View file @ dddb7942
......@@ -2,6 +2,7 @@
extends =
../bzip2/buildout.cfg
../libpng/buildout.cfg
../patch/buildout.cfg
../pkgconfig/buildout.cfg
../zlib/buildout.cfg
......@@ -28,16 +29,25 @@ environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${libogg:location}/lib/pkgconfig
[libtheora-png_sizeof.patch]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:filename}
filename = libtheora-png_sizeof.patch
md5sum = eaa1454081b50f05b59495a12f52b0d5
download-only = true
[libtheora]
recipe = slapos.recipe.cmmi
url = http://downloads.xiph.org/releases/theora/libtheora-1.1.1.tar.bz2
md5sum = 292ab65cedd5021d6b7ddd117e07cd8e
depends =
${libpng:so_version}
patches = ${libtheora-png_sizeof.patch:location}/${libtheora-png_sizeof.patch:filename}
patch-options = -p1
configure-options =
--disable-static
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PATH=${patch:location}/bin:${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${libogg:location}/lib/pkgconfig:${libpng:location}/lib/pkgconfig:${libvorbis:location}/lib/pkgconfig
[yasm]
......
component/ffmpeg/libtheora-png_sizeof.patch 0 → 100644
View file @ dddb7942
--- libtheora-1.1.1/examples/png2theora.c.orig 2009-08-23 03:14:04.000000000 +0900
+++ libtheora-1.1.1/examples/png2theora.c 2013-07-16 12:40:07.629087870 +0900
@@ -462,9 +462,9 @@
png_set_strip_alpha(png_ptr);
row_data = (png_bytep)png_malloc(png_ptr,
- 3*height*width*png_sizeof(*row_data));
+ 3*height*width*sizeof(*row_data));
row_pointers = (png_bytep *)png_malloc(png_ptr,
- height*png_sizeof(*row_pointers));
+ height*sizeof(*row_pointers));
for(y = 0; y < height; y++) {
row_pointers[y] = row_data + y*(3*width);
}
component/gcc/buildout.cfg
View file @ dddb7942
......@@ -33,7 +33,7 @@ environment =
[ppl]
recipe = slapos.recipe.cmmi
# we should use version 0.10.x for gcc-4.5
url = http://www.cs.unipr.it/ppl/Download/ftp/releases/0.10.2/ppl-0.10.2.tar.bz2
url = http://bugseng.com/products/ppl/download/ftp/releases/0.10.2/ppl-0.10.2.tar.bz2
md5sum = 5667111f53150618b0fa522ffc53fc3e
configure-options =
--with-libgmp-prefix=${gmp:location}
......@@ -85,9 +85,6 @@ patches =
${gcc-multiarch.patch:location}/${gcc-multiarch.patch:filename}
patch-options = -p2
configure-command = make clean \\; make distclean \\; ./configure
# GMP does not correctly detect achitecture so it have to be given
# as slapos.recipe.cmmi is using shell expansion in subproceses
# backticks are working
configure-options =
--disable-bootstrap
--enable-languages="c,c++"
......@@ -97,12 +94,30 @@ configure-options =
--with-mpc=${mpc:location}
--with-ppl=${ppl:location}
--with-cloog=${cloog-ppl:location}
--with-ecj-jar=${ecj:location}/${ecj:filename}
--prefix=${buildout:parts-directory}/${:_buildout_section_name_}
environment =
LDFLAGS=-Wl,-rpath=${mpfr:location}/lib -Wl,-rpath=${gmp:location}/lib -Wl,-rpath=${mpc:location}/lib -Wl,-rpath=${ppl:location}/lib -Wl,-rpath=${cloog-ppl:location}/lib
PATH=${zip:location}/bin:%(PATH)s
# make install does not work when several core are used
make-targets = install -j1
[gcc-minimal]
recipe = slapos.recipe.cmmi
url = http://ftp.gnu.org/gnu/gcc/gcc-4.5.4/gcc-core-4.5.4.tar.bz2
md5sum = ca62e442629a9a7710f5d797bf1b521c
patches =
${gcc-multiarch.patch:location}/${gcc-multiarch.patch:filename}
patch-options = -p2
configure-options =
--disable-bootstrap
--enable-languages=c
--disable-multilib
--with-gmp=${gmp:location}
--with-mpfr=${mpfr:location}
--with-mpc=${mpc:location}
--without-ppl
--without-cloog
environment =
LDFLAGS=-Wl,-rpath=${mpfr:location}/lib -Wl,-rpath=${gmp:location}/lib -Wl,-rpath=${mpc:location}/lib
# make install does not work when several core are used
make-targets = install -j1
......
component/git/buildout.cfg
View file @ dddb7942
......@@ -4,6 +4,7 @@
[buildout]
extends =
../curl/buildout.cfg
../gettext/buildout.cfg
../libexpat/buildout.cfg
../openssl/buildout.cfg
../zlib/buildout.cfg
......
component/glib/buildout.cfg
View file @ dddb7942
......@@ -17,3 +17,7 @@ environment =
CPPFLAGS=-I${zlib:location}/include -I${gettext:location}/include
LDFLAGS=-L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${gettext:location}/lib -Wl,-rpath=${gettext:location}/lib
PATH=${gettext:location}/bin:%(PATH)s
[x86-cygwin-glib]
patches =
${:_profile_base_location_}/cygwin-glib-2.28.8-src.patch
component/glib/cygwin-glib-2.28.8-src.patch 0 → 100644
View file @ dddb7942
--- glib/gstrfuncs.c~ 2012-12-30 14:51:30.000000000 +0800
+++ glib/gstrfuncs.c 2012-12-30 14:51:50.203125000 +0800
@@ -1423,7 +1423,7 @@
#ifdef HAVE_STRSIGNAL
const char *msg_locale;
-#if defined(G_OS_BEOS) || defined(G_WITH_CYGWIN)
+#if defined(G_OS_BEOS)
extern const char *strsignal(int);
#else
/* this is declared differently (const) in string.h on BeOS */
component/grep/buildout.cfg
View file @ dddb7942
[buildout]
extends =
../pcre/buildout.cfg
../coreutils/buildout.cfg
../xz-utils/buildout.cfg
parts =
grep
grep-output
[grep]
recipe = slapos.recipe.cmmi
......@@ -13,3 +14,11 @@ environment =
PATH=${xz-utils:location}/bin:%(PATH)s
CPPFLAGS=-I${pcre:location}/include
LDFLAGS=-L${pcre:location}/lib -Wl,-rpath=${pcre:location}/lib
[grep-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${coreutils-output:test} -x ${:grep}
grep = ${grep:location}/bin/grep
component/nodejs/buildout.cfg
View file @ dddb7942
......@@ -14,8 +14,8 @@ parts =
[nodejs-0.8]
# Server-side Javascript.
recipe = slapos.recipe.cmmi
url = http://nodejs.org/dist/v0.8.14/node-v0.8.14.tar.gz
md5sum = 284fd2c7578064c339d9cf6a3a475ac7
url = http://nodejs.org/dist/v0.8.23/node-v0.8.23.tar.gz
md5sum = 22fe54ac365f52d3d80ecf748e7323d5
configure-options =
--openssl-includes=${openssl:location}/include
--openssl-libpath=${openssl:location}/lib
......
component/openssl/buildout.cfg
View file @ dddb7942
......@@ -51,4 +51,4 @@ configure-options =
make-options =
-j1
make-targets =
install && rm -f ${buildout:parts-directory}/${:_buildout_section_name_}/etc/ssl/certs/* && for i in ${ca-certificates:location}/certs/*/*.crt; do ln -sv $i ${buildout:parts-directory}/${:_buildout_section_name_}/etc/ssl/certs/`${buildout:parts-directory}/${:_buildout_section_name_}/bin/openssl x509 -hash -noout -in $i`.0; done; true
all install_sw && rm -f ${buildout:parts-directory}/${:_buildout_section_name_}/etc/ssl/certs/* && for i in ${ca-certificates:location}/certs/*/*.crt; do ln -sv $i ${buildout:parts-directory}/${:_buildout_section_name_}/etc/ssl/certs/`${buildout:parts-directory}/${:_buildout_section_name_}/bin/openssl x509 -hash -noout -in $i`.0; done; true
component/popt/buildout.cfg
View file @ dddb7942
......@@ -8,3 +8,6 @@ url = ftp://anduin.linuxfromscratch.org/BLFS/svn/p/popt-1.16.tar.gz
md5sum = 3743beefa3dd6247a73f8f7a32c14c33
configure-options =
--disable-static
[x86-cygwin-popt]
patches = ${:_profile_base_location_}/cygwin-popt-1.16-src.patch
\ No newline at end of file
component/popt/cygwin-popt-1.16-src.patch 0 → 100644
View file @ dddb7942
--- poptconfig.c~ 2011-09-10 22:49:30.802250000 +0800
+++ poptconfig.c 2011-09-10 23:46:30.572048000 +0800
@@ -45,8 +45,9 @@ extern int glob_pattern_p (const char *_
#if !defined(__GLIBC__)
/* Return nonzero if PATTERN contains any metacharacters.
Metacharacters can be quoted with backslashes if QUOTE is nonzero. */
+#define glob_pattern_p glob_pattern_p_s
static int
-glob_pattern_p (const char * pattern, int quote)
+glob_pattern_p_s(const char * pattern, int quote)
/*@*/
{
const char * p;
component/rdiff-backup/buildout.cfg
View file @ dddb7942
[buildout]
extends =
../librsync/buildout.cfg
../coreutils/buildout.cfg
parts =
rdiff-backup
rdiff-backup-output
[rdiff-backup-build]
recipe = zc.recipe.egg:custom
......@@ -24,3 +25,11 @@ eggs =
entry-points =
rdiff-backup=rdiff_backup.Main:Main
arguments = sys.argv[1:]
[rdiff-backup-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${coreutils-output:test} -x ${:rdiff-backup}
rdiff-backup = ${buildout:directory}/bin/rdiff-backup
setup.py
View file @ dddb7942
......@@ -28,7 +28,7 @@ from setuptools import setup, find_packages
import glob
import os
version = '0.78.2.dev'
version = '0.78.4.dev'
name = 'slapos.cookbook'
long_description = open("README.txt").read() + "\n" + \
open("CHANGES.txt").read() + "\n"
......@@ -188,6 +188,8 @@ setup(name=name,
'slapreport = slapos.recipe.slapreport:Recipe',
'slaprunner = slapos.recipe.slaprunner:Recipe',
'slaprunner.test = slapos.recipe.slaprunner:Test',
'slaprunner.export = slapos.recipe.slaprunner.backup:ExportRecipe',
'slaprunner.import = slapos.recipe.slaprunner.backup:ImportRecipe',
'softwaretype = slapos.recipe.softwaretype:Recipe',
'sphinx= slapos.recipe.sphinx:Recipe',
'sshkeys_authority = slapos.recipe.sshkeys_authority:Recipe',
......
slapos/recipe/README.generic_varnish.txt
View file @ dddb7942
......@@ -12,15 +12,14 @@ On slap console, you can instanciate varnish like this:
instance = request(
software_type='varnish',
partition_parameter_kw={
'tidstorage-url':'http://[your tidstrage address]:your tid strage port',
'backend-url':'https://[your_backend_address]:your_backend_port',
'web-checker-frontend-url':'http://www.example.com',
'web-checker-mail-address':'web-checker-result@example.com',
'web-checker-smtp-host':'mail.example.com',
}
)
tidstrage-url is the backend url that varnish will cache. It is expected that
the backend is created by tidstorage recipe.
backend-url is the backend url that varnish will cache.
web-checker-frontend-url is the entry-point-url that web checker will check
the HTTP headers of all the pages in the web site.
......
slapos/recipe/README.kvm.txt
View file @ dddb7942
......@@ -31,10 +31,10 @@ KVM with Remote and gzipped Image
gzip = true
# Use -hda instead -drive arg
# Default is drive (see Options bellow)
# Default is drive (see Options below)
image_type = hda
### Common Configuration bellow. ###
### Common Configuration below. ###
# VNC is optional
kvm_vnc = <SOME-IP>:<VNC-DISPLAY>
......
slapos/recipe/davstorage/__init__.py
View file @ dddb7942
......@@ -27,6 +27,8 @@
import subprocess
import httplib
import base64
import os
import shutil
from slapos.recipe.librecipe import GenericBaseRecipe
......@@ -50,13 +52,29 @@ class Recipe(GenericBaseRecipe):
user, password
])
htdocs_location = self.options['htdocs']
if not (os.path.exists(htdocs_location) and os.listdir(htdocs_location)):
try:
os.rmdir(htdocs_location)
except:
pass
shutil.copytree(self.options['source'], htdocs_location)
# Install php.ini
php_ini = self.createFile(os.path.join(self.options['php-ini-dir'],
'php.ini'),
self.substituteTemplate(self.getTemplateFilename('php.ini.in'),
dict(tmp_directory=self.options['tmp-dir']))
)
path_list.append(php_ini)
apache_config = dict(
pid_file=self.options['pid-file'],
lock_file=self.options['lock-file'],
davlock_db=self.options['davdb-lock'],
ip=self.options['ip'],
port=self.options['port'],
port_webdav=self.options['port_webdav'],
port_ajax=self.options['port_ajax'],
error_log=self.options['error-log'],
access_log=self.options['access-log'],
document_root=self.options['htdocs'],
......@@ -67,6 +85,7 @@ class Recipe(GenericBaseRecipe):
htpasswd_file=htpasswd_file,
ssl_certificate=self.options['cert-file'],
ssl_key=self.options['key-file'],
php_ini_dir=self.options['php-ini-dir']
)
# Create logfiles
......@@ -86,7 +105,7 @@ class Recipe(GenericBaseRecipe):
promise = self.createPythonScript(self.options['promise'],
__name__ + '.promise',
dict(host=self.options['ip'], port=int(self.options['port']),
dict(host=self.options['ip'], port=int(self.options['port_webdav']),
user=self.options['user'], password=self.options['password'])
)
path_list.append(promise)
......
slapos/recipe/davstorage/template/httpd.conf.in
View file @ dddb7942
ServerRoot "%(server_root)s"
Listen [%(ip)s]:%(port)s
Listen [%(ip)s]:%(port_webdav)s
Listen [%(ip)s]:%(port_ajax)s
NameVirtualHost [%(ip)s]:%(port_webdav)s
NameVirtualHost [%(ip)s]:%(port_ajax)s
# Needed modules
LoadModule unixd_module modules/mod_unixd.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule unixd_module "%(modules_dir)s/mod_unixd.so"
LoadModule access_compat_module "%(modules_dir)s/mod_access_compat.so"
LoadModule authn_core_module "%(modules_dir)s/mod_authn_core.so"
LoadModule authz_core_module "%(modules_dir)s/mod_authz_core.so"
LoadModule authn_file_module "%(modules_dir)s/mod_authn_file.so"
LoadModule authz_host_module "%(modules_dir)s/mod_authz_host.so"
LoadModule authz_user_module "%(modules_dir)s/mod_authz_user.so"
LoadModule auth_basic_module "%(modules_dir)s/mod_auth_basic.so"
LoadModule auth_digest_module "%(modules_dir)s/mod_auth_digest.so"
# Comment auth_digest since we don't use it
#LoadModule auth_digest_module "%(modules_dir)s/mod_auth_digest.so"
LoadModule log_config_module "%(modules_dir)s/mod_log_config.so"
LoadModule headers_module "%(modules_dir)s/mod_headers.so"
LoadModule setenvif_module "%(modules_dir)s/mod_setenvif.so"
LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
LoadModule socache_shmcb_module "%(modules_dir)s/mod_socache_shmcb.so"
LoadModule ssl_module "%(modules_dir)s/mod_ssl.so"
LoadModule mime_module "%(modules_dir)s/mod_mime.so"
LoadModule dav_module "%(modules_dir)s/mod_dav.so"
LoadModule dav_fs_module "%(modules_dir)s/mod_dav_fs.so"
LoadModule dir_module "%(modules_dir)s/mod_dir.so"
LoadModule php5_module "%(modules_dir)s/libphp5.so"
ServerAdmin %(email_address)s
# Quiet Server header (if not, Apache give its life history)
# It's safer
ServerTokens ProductOnly
PidFile "%(pid_file)s"
PHPINIDir "%(php_ini_dir)s"
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
<VirtualHost [%(ip)s]:%(port_ajax)s>
#ServerName www.example.com
# Directory protection
<Directory />
Options FollowSymLinks
AllowOverride None
Require all denied
</Directory>
<Directory %(document_root)s>
Options FollowSymLinks
AllowOverride All
Require all granted
</Directory>
DocumentRoot "%(document_root)s"
DirectoryIndex index.html index.php
SSLEngine on
SSLCertificateFile "%(ssl_certificate)s"
SSLCertificateKeyFile "%(ssl_key)s"
</VirtualHost>
<VirtualHost [%(ip)s]:%(port_webdav)s>
DocumentRoot "%(document_root)s"
PidFile "%(pid_file)s"
DavLockDB "%(davlock_db)s"
<Directory />
......@@ -66,6 +103,12 @@ DavLockDB "%(davlock_db)s"
</Directory>
SSLEngine on
SSLCertificateFile "%(ssl_certificate)s"
SSLCertificateKeyFile "%(ssl_key)s"
</VirtualHost>
ErrorLog "%(error_log)s"
LogLevel warn
......@@ -77,9 +120,5 @@ DefaultType text/plain
TypesConfig "%(mime_types)s"
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLEngine on
SSLCertificateFile "%(ssl_certificate)s"
SSLCertificateKeyFile "%(ssl_key)s"
AddType application/x-httpd-php .php .phtml .php5 .php4
AddType application/x-httpd-php-source .phps
\ No newline at end of file
slapos/recipe/davstorage/template/php.ini.in 0 → 100644
View file @ dddb7942
[PHP]
engine = On
safe_mode = Off
expose_php = Off
error_reporting = E_ALL & ~(E_DEPRECATED|E_NOTICE|E_WARNING)
display_errors = On
display_startup_errors = Off
log_errors = On
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
session.save_path = "%(tmp_directory)s"
session.auto_start = 0
date.timezone = Europe/Paris
file_uploads = On
upload_max_filesize = 8M
post_max_size = 8M
magic_quotes_gpc=Off
output_buffering=Off
\ No newline at end of file
slapos/recipe/erp5_promise/__init__.py
View file @ dddb7942
......@@ -34,19 +34,23 @@ class Recipe(GenericBaseRecipe):
"""
def install(self):
promise_parser = ConfigParser.RawConfigParser()
promise_parser.add_section('portal_templates')
promise_parser.set('portal_templates', 'repository', self.options['bt5-repository-url'])
promise_parser.set('portal_templates', 'expected_bt5', self.options['bt5'])
promise_parser.add_section('external_service')
promise_parser.set('external_service', 'cloudooo_url', self.options['cloudooo-url'])
promise_parser.set('external_service', 'memcached_url', self.options['memcached-url'])
promise_parser.set('external_service', 'kumofs_url', self.options['kumofs-url'])
promise_parser.set('external_service', 'smtp_url', self.options['smtp-url'])
for section_name, option_id_list in (
('portal_templates', (
('repository', 'bt5-repository-url'),
('expected_bt5', 'bt5'),
)),
('external_service', (
('cloudooo_url', 'cloudooo-url'),
('memcached_url', 'memcached-url'),
('kumofs_url', 'kumofs-url'),
('smtp_url', 'smtp-url'),
)),
):
promise_parser.add_section(section_name)
for internal_id, option_id in option_id_list:
option = self.options.get(option_id)
if option:
promise_parser.set(section_name, internal_id, option)
promise_parser.write(open(self.options['promise-path'], 'w'))
return [self.options['promise-path']]
slapos/recipe/generic_varnish/__init__.py
View file @ dddb7942
......@@ -37,7 +37,10 @@ class Recipe(GenericSlapRecipe):
"""
def _install(self):
ip = self.options['ip']
backend_url = self.parameter_dict['tidstorage-url']
backend_url = self.parameter_dict.get(
'backend-url',
self.parameter_dict.get('tidstorage-url') # BBB
)
backend_server, backend_port = self._getBackendServer(backend_url)
path_list = []
if backend_url.startswith('https://'):
......@@ -70,6 +73,7 @@ class Recipe(GenericSlapRecipe):
varnishd_pid_file=self.options['pid-file'],
varnish_instance_name=self.options['varnish-instance-name'],
varnish_data=self.options['varnish-data'],
gcc_location=self.options['gcc-location'],
shell_path=self.options['shell-path'],
vcl_file=self.options['vcl-file'],
backend_port=backend_port,
......
slapos/recipe/generic_varnish/template/varnishd.in
View file @ dddb7942
......@@ -10,5 +10,5 @@ DAEMON_OPTS="-F \
PIDFILE=%(varnishd_pid_file)s
# exporting PATH here so that we will pass the PATH variable to the subprocess
export PATH
export PATH="%(gcc_location)s:$PATH"
exec %(varnishd_binary)s -P ${PIDFILE} ${DAEMON_OPTS} 2>&1
slapos/recipe/generic_zope/template/zope.conf.in
View file @ dddb7942
......@@ -25,6 +25,9 @@ rest-input-encoding utf-8
rest-output-encoding utf-8
default-zpublisher-encoding utf-8
# Disable ownership checking to execute codes generated by alarm
skip-ownership-checking on
# Temporary storage database (for sessions)
<zodb_db temporary>
<temporarystorage>
......
slapos/recipe/generic_zope_zeo_client/template/zope.conf.in
View file @ dddb7942
......@@ -25,6 +25,9 @@ rest-input-encoding utf-8
rest-output-encoding utf-8
default-zpublisher-encoding utf-8
# Disable ownership checking to execute codes generated by alarm
skip-ownership-checking on
# Temporary storage database (for sessions)
<zodb_db temporary>
<temporarystorage>
......
slapos/recipe/haproxy/__init__.py
View file @ dddb7942
......@@ -87,7 +87,11 @@ class Recipe(GenericBaseRecipe):
'haproxy-listen-snippet.cfg.in')
server_snippet = ""
ip = self.options['ip']
server_check_path = self.options['server-check-path']
server_check_path = self.options.get('server-check-path', None)
if server_check_path:
httpchk = 'option httpchk GET %s' % server_check_path
else:
httpchk = ''
# FIXME: maxconn must be provided per-backend, not globally
maxconn = self.options['maxconn']
i = 0
......@@ -97,7 +101,7 @@ class Recipe(GenericBaseRecipe):
'name': name,
'ip': ip,
'port': port,
'server_check_path': server_check_path,
'httpchk': httpchk,
})
for address in backend_list:
i += 1
......
slapos/recipe/haproxy/template/haproxy-listen-snippet.cfg.in
View file @ dddb7942
listen %(name)s %(ip)s:%(port)s
cookie SERVERID insert
balance roundrobin
option httpchk GET %(server_check_path)s
%(httpchk)s
stats uri /haproxy
stats realm Global\ statistics
slapos/recipe/neoppod.py
View file @ dddb7942
......@@ -50,8 +50,6 @@ class NeoBaseRecipe(GenericBaseRecipe):
#'-n', options['name'],
'-c', options['cluster'],
]
if options['verbose']:
option_list.append('-v')
option_list.extend(self._getOptionList())
return [self.createPythonScript(
options['wrapper'],
......
slapos/recipe/publish.py
View file @ dddb7942
......@@ -35,20 +35,22 @@ class Recipe(GenericSlapRecipe):
publish_dict = dict()
options = self.options.copy()
del options['recipe']
slave_reference = options.pop('-slave-reference', None)
for k, v in options.iteritems():
if k[:1] == '-':
continue
publish_dict[k] = v
self._setConnectionDict(publish_dict)
self._setConnectionDict(publish_dict, slave_reference)
return []
def _setConnectionDict(self, publish_dict):
return self.setConnectionDict(publish_dict)
def _setConnectionDict(self, publish_dict, slave_reference=None):
return self.setConnectionDict(publish_dict, slave_reference)
SERIALISED_MAGIC_KEY = '_'
class Serialised(Recipe):
def _setConnectionDict(self, publish_dict):
return super(Serialised, self)._setConnectionDict(wrap(publish_dict))
def _setConnectionDict(self, publish_dict, slave_reference=None):
return super(Serialised, self)._setConnectionDict(wrap(publish_dict), slave_reference)
......
slapos/recipe/request.py
View file @ dddb7942
......@@ -33,6 +33,12 @@ import traceback
DEFAULT_SOFTWARE_TYPE = 'RootSoftwareInstance'
def getListOption(option_dict, key, default=()):
result = option_dict.get(key, default)
if isinstance(result, basestring):
result = result.split()
return result
class Recipe(object):
"""
Request a partition to a slap master.
......@@ -91,18 +97,18 @@ class Recipe(object):
self.logger = logging.getLogger(name)
software_url = options['software-url']
name = options['name']
return_parameters = options.get('return', '').split()
return_parameters = getListOption(options, 'return')
if not return_parameters:
self.logger.debug("No parameter to return to main instance."
"Be careful about that...")
software_type = options.get('software-type', DEFAULT_SOFTWARE_TYPE)
filter_kw = dict(
(x, options['sla-' + x]) for x in options.get('sla', '').split()
(x, options['sla-' + x]) for x in getListOption(options, 'sla')
if options['sla-' + x]
)
partition_parameter_kw = self._filterForStorage(dict(
(x, options['config-' + x])
for x in options.get('config', '').split()
for x in getListOption(options, 'config')
))
slave = options.get('slave', 'false').lower() in \
librecipe.GenericBaseRecipe.TRUE_VALUES
......
slapos/recipe/slaprunner/backup.py 0 → 100644
View file @ dddb7942
##############################################################################
#
# Copyright (c) 2013 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
import textwrap
from slapos.recipe.librecipe import GenericBaseRecipe
class ExportRecipe(GenericBaseRecipe):
"""\
This recipe creates an exporter script for using with the resilient stack.
Required options:
backup-directory
folder that will contain the dump file.
srv-directory
folder that contain the runner directory.
wrapper
full path of the exporter script to create.
"""
def install(self):
wrapper = self.options['wrapper']
self.createBackupScript(wrapper)
return [wrapper]
def createBackupScript(self, wrapper):
"""\
Create a script to backup the database in 'custom' format.
"""
content = textwrap.dedent("""\
#!%(shell-binary)s
umask 077
sync_element () {
path=$1
backup_path=$2
shift 2
element_list=$*
for element in $element_list
do
cd $path;
if [ -f $element ] || [ -d $element ]; then
%(rsync-binary)s -avz --safe-links --delete $element $backup_path;
fi
done
}
sync_element %(srv-directory)s/runner %(backup-directory)s/runner/ instance project proxy.db softwareLink
sync_element %(etc-directory)s %(backup-directory)s/etc/ .rcode .project .users ssh
if [ -d %(backup-directory)s/runner/software ]; then
rm %(backup-directory)s/runner/software/*
fi
""" % self.options)
self.createExecutable(wrapper, content=content)
class ImportRecipe(GenericBaseRecipe):
"""\
This recipe creates an importer script for using with the resilient stack.
Required options:
backup-directory
folder that will contain the dump file.
srv-directory
folder that contain the runner directory.
wrapper
full path of the exporter script to create.
"""
def install(self):
wrapper = self.options['wrapper']
self.createRestoreScript(wrapper)
return [wrapper]
def createRestoreScript(self, wrapper):
"""\
Create a script to restore the database from 'custom' format.
"""
content = textwrap.dedent("""\
#!%(shell-binary)s
umask 077
restore_element () {
backup_path=$1
restore_path=$2
shift 2
element_list=$*
for element in $element_list
do
cd $backup_path;
if [ -f $element ] || [ -d $element ]; then
%(rsync-binary)s -avz --delete $backup_path/$element $restore_path;
fi
done
}
restore_element %(backup-directory)s/runner/ %(srv-directory)s/runner instance project proxy.db softwareLink
restore_element %(backup-directory)s/etc/ %(etc-directory)s .rcode .project .users ssh
ifs=$IFS IFS=';'
read user pass remaining < %(etc-directory)s/.users
IFS=$ifs
%(curl-binary)s -vg6L -F clogin="$user" -F cpwd="$pass" --dump-header login_cookie %(backend-url)s/doLogin;
%(curl-binary)s -vg6LX POST --cookie login_cookie --max-time 5 %(backend-url)s/runSoftwareProfile;
rm -f login_cookie
""" % self.options)
self.createExecutable(wrapper, content=content)
slapos/recipe/tidstorage/__init__.py
View file @ dddb7942
......@@ -43,7 +43,11 @@ class Recipe(GenericBaseRecipe):
repozo_wrapper = self.createPythonScript(
self.options['repozo-wrapper'],
'slapos.recipe.librecipe.execute.execute',
[self.options['tidstorage-repozo-binary'], '--config',
configuration_file, '--repozo', self.options['repozo-binary'], '-z'])
[self.options['tidstorage-repozo-binary'],
'--config', configuration_file,
'--repozo', self.options['repozo-binary'],
'--gzip',
'--quick',
])
return [configuration_file, tidstorage_wrapper, repozo_wrapper]
software/backupserver/instance-pullrdiffbackup.cfg.in 0 → 100644
View file @ dddb7942
{% set part_list = [] -%}
{% set crontab_line_list = [] -%}
###############################
#
# Instanciate dcron
#
###############################
[directory]
recipe = slapos.cookbook:mkdirectory
etc = $${buildout:directory}/etc
bin = $${buildout:directory}/bin
srv = $${buildout:directory}/srv
var = $${buildout:directory}/var
log = $${:var}/log
# scripts = $${:etc}/run
services = $${:etc}/service
cron-entries = $${:etc}/cron.d
cron-lines = $${:etc}/cron.lines
crontabs = $${:etc}/crontabs
cronstamps = $${:etc}/cronstamps
backup = $${:srv}/backup
backupscript = $${:etc}/backup
home = $${:etc}/home
ssh = $${:home}/.ssh
#################################
# Cron service
#################################
[dcron-service]
recipe = slapos.recipe.template
url = ${template-dcron-service:output}
output = $${directory:services}/crond
mode = 0700
logfile = $${directory:log}/crond.log
#################################
# Slave backup scripts and crontab
#################################
# Go throught slave list to set their configuration
{% for slave_instance in slave_instance_list -%}
{% set slave_reference = slave_instance.get('slave_reference') -%}
{% set frequency = slave_instance.get('frequency', '') -%}
{% set hostname = slave_instance.get('hostname', '') -%}
{% set connection = slave_instance.get('connection', '') -%}
{% set include = slave_instance.get('include', '') -%}
{% set include_string = "' --include='".join(include.split(' ')) -%}
{% set exclude = slave_instance.get('exclude', '') -%}
{% set exclude_string = '' -%}
{% set sudo = slave_instance.get('sudo', 'False') -%}
{% set remote_schema = 'rdiff-backup --server --restrict-read-only / -- "$@"' -%}
{% if (exclude != '') -%}
{% set exclude_string = "' --exclude='".join(exclude.split(' ')) -%}
{% set exclude_string = "--exclude='" + exclude_string + "'" -%}
{% endif -%}
{% if (sudo == 'True') -%}
{% set remote_schema = 'sudo backupagent_rdiff-backup' -%}
{% endif -%}
{% if (frequency != '') and (hostname != '') and (connection != '') and (include != '') -%}
[{{ slave_reference }}-backup-directory]
recipe = slapos.cookbook:mkdirectory
directory = $${directory:backup}/$${:_buildout_section_name_}
[{{ slave_reference }}-backup-private_key]
recipe = plone.recipe.command
stop-on-error = true
command = ${dropbear-output:keygen} -t $${:type} -s 2048 -f $${:key}
key = $${directory:ssh}/$${:_buildout_section_name_}
location = $${:key}
type = rsa
[{{ slave_reference }}-backup-public_key]
recipe = plone.recipe.command
stop-on-error = true
update-command = $${:command}
command = ${coreutils-output:rm} -f $${:key} && ${dropbear-output:keygen} -y -f {{ '$${' ~ slave_reference }}-backup-private_key:key} | ${grep-output:grep} {{ '$${' ~ slave_reference }}-backup-private_key:type} > $${:key}
key = {{ '$${' ~ slave_reference }}-backup-private_key:key}.pub
location = $${:key}
[{{ slave_reference }}-backup-check-public_key]
recipe = plone.recipe.command
stop-on-error = true
update-command = $${:command}
command = grep ssh-{{ '$${' ~ slave_reference }}-backup-private_key:type} {{ '$${' ~ slave_reference }}-backup-public_key:key}
[{{ slave_reference }}-backup-read-public_key]
recipe = slapos.cookbook:generate.password
storage-path = {{ '$${' ~ slave_reference }}-backup-public_key:key}
bytes = 8
# Publish slave {{ slave_reference }} information
[{{ slave_reference }}-backup-publish]
recipe = slapos.cookbook:publish
-slave-reference = {{ slave_reference }}
authorized_key = {{ '$${' ~ slave_reference }}-backup-read-public_key:passwd}
{% do part_list.append("%s-backup-publish" % slave_reference) -%}
[{{ slave_reference }}-backup-script]
recipe = slapos.recipe.template
url = ${template-backup-script:output}
output = $${directory:backupscript}/$${:_buildout_section_name_}
mode = 0700
datadirectory = {{ '$${' ~ slave_reference }}-backup-directory:directory}
sshkey = {{ '$${' ~ slave_reference }}-backup-private_key:key}
connection = {{ connection }}
hostname = {{ hostname }}
include = {{ include_string }}
exclude_string = {{ exclude_string }}
remote_schema = {{ remote_schema }}
[{{ slave_reference }}-backup-crontab-line]
recipe = slapos.recipe.template
url = ${template-crontab-line:output}
output = $${directory:cron-lines}/$${:_buildout_section_name_}
mode = 0600
script = {{ '$${' ~ slave_reference }}-backup-script:output}
frequency = {{ frequency }}
{% do crontab_line_list.append("$${%s-backup-crontab-line:output}" % slave_reference) -%}
{% endif -%}
{% endfor -%}
#################################
# Generate crontab file
#################################
{% set crontab_line_list_string = " ".join(crontab_line_list) -%}
[activate-crontab-file]
# XXX File is never removed
recipe = plone.recipe.command
stop-on-error = true
update-command = $${:command}
command = ${coreutils-output:cat} ${template-crontab:output} {{ crontab_line_list_string }} | ${dcron-output:crontab} -c $${directory:crontabs} -
# Add parts generated by template
[buildout]
parts =
dcron-service
activate-crontab-file
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
software/backupserver/instance.cfg.in 0 → 100644
View file @ dddb7942
[buildout]
parts =
switch-softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[dynamic-template-pullrdiffbackup]
recipe = slapos.recipe.template:jinja2
template = ${template-pullrdiffbackup:output}
rendered = $${buildout:parts-directory}/$${:_buildout_section_name_}/$${:filename}
filename = instance-pullrdiffbackup.cfg
extensions = jinja2.ext.do
context =
key slave_instance_list instance-parameter:slave-instance-list
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = $${:pullrdiffbackup}
# pullrdiffbackup = ${template-pullrdiffbackup:output}
pullrdiffbackup = $${dynamic-template-pullrdiffbackup:rendered}
[slap-connection]
# part to migrate to new - separated words
computer-id = $${slap_connection:computer_id}
partition-id = $${slap_connection:partition_id}
server-url = $${slap_connection:server_url}
software-release-url = $${slap_connection:software_release_url}
key-file = $${slap_connection:key_file}
cert-file = $${slap_connection:cert_file}
# [slap-parameter]
# slave-instance-list = []
[instance-parameter]
# Fetches parameters defined in SlapOS Master for this instance.
# Always the same.
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap_connection:computer_id}
partition = $${slap_connection:partition_id}
url = $${slap_connection:server_url}
key = $${slap_connection:key_file}
cert = $${slap_connection:cert_file}
software/backupserver/software.cfg 0 → 100644
View file @ dddb7942
This diff is collapsed.
software/backupserver/template-backup-script.sh.in 0 → 100644
View file @ dddb7942
#!${dash-output:dash}
# trap "echo Backing up failed for $${:hostname}" ERR
export HOME=$${directory:home}
set -e
cd $${:datadirectory}
${rdiff-backup-output:rdiff-backup} \
$${:exclude_string} \
--include='$${:include}' \
--exclude='**' \
--remote-schema '${dropbear-output:ssh} -T -y -i $${:sshkey} %s $${:remote_schema}' \
$${:connection}::/ ./
# python scripts/verify_with_sudo.py ./ $${:connection}:/
software/backupserver/template-crontab-line.in 0 → 100644
View file @ dddb7942
# $${:_buildout_section_name_}
$${:frequency} $${:script}
software/backupserver/template-crontab.in 0 → 100644
View file @ dddb7942
# min(0-59) hours(0-23) day(1-31) month(1-12) dow(0-7) command
MAILTO=admins@erp5.org
software/backupserver/template-dcron-service.sh.in 0 → 100644
View file @ dddb7942
#!${dash-output:dash}
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
exec ${dcron-output:crond} \
-s $${directory:cron-entries} \
-c $${directory:crontabs} \
-t $${directory:cronstamps} \
-f -l 5 \
-L $${dcron-service:logfile}
# -M cron_simplelogger
software/cloudooo/instance-cloudoo-input-schema.json 0 → 100644
View file @ dddb7942
{
"$schema": "http://json-schema.org/draft-04/schema#",
"extends": "./schema-definitions.json#",
"properties": {
"tcpv4-port": {
"allOf": [{
"$ref": "#/definitions/tcpv4port"
}, {
"description": "Start allocating ports at this value, going upward",
"default": 23000
}]
},
"font-url-list": {
"description": "List of URLs from which fonts are to be downloaded",
"default": [],
"items": {
"type": "string"
},
"type": "array"
}
"backend-count": {
"description": "Number of backend cloudooo instances",
"default": 1,
"type": "integer"
}
}
}
software/cloudooo/instance-cloudoo-output-schema.json 0 → 100644
View file @ dddb7942
{
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "Values returned by Cloudooo instanciation",
"properties": {
"url": {
"description": "Conversion service access information",
"type": "string"
}
},
"type": "object"
}
software/cloudooo/instance-cloudoo.cfg.in
View file @ dddb7942
{% set ipv4 = (ipv4_set | list)[0] -%}
{% set bin_directory = parameter_dict['buildout-bin-directory'] -%}
{% set section_list = [] -%}
{% macro section(name) %}{% do section_list.append(name) %}{{ name }}{% endmacro -%}
[buildout]
parts =
publish-cloudooo-connection-information
......@@ -9,35 +12,53 @@ develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[publish-cloudooo-connection-information]
recipe = slapos.cookbook:publishurl
url = cloudooo://${cloudooo-instance:ip}:${cloudooo-instance:port}/
recipe = slapos.cookbook:publish.serialised
url = cloudooo://${haproxy:ip}:${haproxy:port}/
[cloudooo-instance]
[cloudooo-base]
recipe = slapos.cookbook:generic.cloudooo
# Network options
ip = ${slap-network-information:local-ipv4}
port = 23000
openoffice-port = 23060
# Paths
configuration-file = ${rootdirectory:etc}/cloudooo.cfg
wrapper = ${basedirectory:services}/cloudooo
# Paths: Data
data-directory = ${directory:cloudooo-data}
ip = {{ ipv4 }}
environment =
LD_LIBRARY_PATH = {{ parameter_dict['file'] }}/lib:{{ parameter_dict['fontconfig'] }}/lib:{{ parameter_dict['freetype'] }}/lib:{{ parameter_dict['libICE'] }}/lib:{{ parameter_dict['libpng12'] }}/lib:{{ parameter_dict['libSM'] }}/lib:{{ parameter_dict['libX11'] }}/lib:{{ parameter_dict['libXau'] }}/lib:{{ parameter_dict['libXdmcp'] }}/lib:{{ parameter_dict['libXext'] }}/lib:{{ parameter_dict['libxcb'] }}/lib:{{ parameter_dict['libXrender'] }}/lib:{{ parameter_dict['zlib'] }}/lib
FONTCONFIG_FILE = ${fontconfig-instance:conf-path}
PATH = ${binary-link:target-directory}
# Binary information
# cloudooo specific configuration
ooo-binary-path = {{ parameter_dict['libreoffice-bin'] }}/program
ooo-paster = {{ bin_directory }}/cloudooo_paster
ooo-uno-path = {{ parameter_dict['libreoffice-bin'] }}/basis-link/program
{% set cloudooo_port = slapparameter_dict.get('tcpv4_port', 23000) | int -%}
{% set backend_count = slapparameter_dict.get('backend-count', 1) | int -%}
{% for index in range(backend_count) -%}
{% set name = 'cloudooo-' ~ index -%}
[{{ section(name) }}]
< = cloudooo-base
port = {{ cloudooo_port }}
openoffice-port = {{ cloudooo_port + 1 }}
configuration-file = ${directory:etc}/{{ name }}.cfg
data-directory = ${directory:srv}/{{ name }}
wrapper = ${directory:services}/{{ name }}
{% set cloudooo_port = cloudooo_port + 2 -%}
{% endfor -%}
[haproxy]
recipe = slapos.cookbook:haproxy
name = cloudooo
conf-path = ${directory:etc}/haproxy.cfg
socket-path = ${directory:run}/haproxy.sock
ip = {{ ipv4 }}
port = 8001
maxconn = 1
wrapper-path = ${directory:services}/haproxy
binary-path = {{ parameter_dict['haproxy'] }}/sbin/haproxy
ctl-path = ${directory:bin}/haproxy-ctl
backend-list =
{%- for section_name in section_list %}
{{ "${" ~ section_name ~ ":ip}:${" ~ section_name ~ ":port}" }}
{%- endfor %}
[cloudooo-test-runner]
recipe = slapos.cookbook:cloudooo.test
......@@ -45,24 +66,25 @@ prepend-path = ${buildout:bin-directory}
run-unit-test = ${buildout:bin-directory}/runUnitTest
run-test-suite = ${buildout:bin-directory}/runTestSuite
ooo-paster = ${cloudooo-instance:ooo-paster}
configuration-file = ${cloudooo-instance:configuration-file}
ooo-paster = ${cloudooo-0:ooo-paster}
configuration-file = ${cloudooo-0:configuration-file}
run-unit-test-binary = {{ bin_directory }}/runCloudoooUnitTest
run-test-suite-binary = {{ bin_directory }}/runCloudoooTestSuite
[fontconfig-instance]
recipe = slapos.cookbook:fontconfig
conf-path = ${rootdirectory:etc}/font.conf
conf-path = ${directory:etc}/font.conf
font-system-folder = {{ parameter_dict['fonts'] }}
font-folder = ${directory:font}
url-list =
service-folder = ${basedirectory:services}
{# XXX: violates "instanciation happens offline" rule -#}
url-list = {{ slapparameter_dict.get('font-url-list', []) | join(' ') }}
service-folder = ${directory:services}
onetimedownload_path = {{ bin_directory }}/onetimedownload
[binary-link]
recipe = slapos.cookbook:symbolic.link
target-directory = ${rootdirectory:bin}
target-directory = ${directory:bin}
link-binary =
{{ parameter_dict['coreutils'] }}/bin/basename
{{ parameter_dict['coreutils'] }}/bin/cat
......@@ -78,17 +100,12 @@ link-binary =
{{ parameter_dict['poppler'] }}/bin/pdftohtml
# rest of parts are candidates for some generic stuff
[basedirectory]
recipe = slapos.cookbook:mkdirectory
services = ${rootdirectory:etc}/run
[directory]
recipe = slapos.cookbook:mkdirectory
cloudooo-data = ${rootdirectory:srv}/cloudooo
font = ${rootdirectory:srv}/font
[rootdirectory]
recipe = slapos.cookbook:mkdirectory
bin = ${buildout:directory}/bin
etc = ${buildout:directory}/etc
font = ${:srv}/font
run = ${:var}/run
services = ${:etc}/run
srv = ${buildout:directory}/srv
bin = ${buildout:directory}/bin
var = ${buildout:directory}/var
software/cloudooo/instance.cfg.in
View file @ dddb7942
......@@ -6,6 +6,14 @@ eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[slap-parameters]
recipe = slapos.cookbook:slapconfiguration
computer = ${slap-connection:computer-id}
partition = ${slap-connection:partition-id}
url = ${slap-connection:server-url}
key = ${slap-connection:key-file}
cert = ${slap-connection:cert-file}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = ${buildout:parts-directory}/${:_buildout_section_name_}/${:filename}
......@@ -13,12 +21,14 @@ extra-context =
context =
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slapparameter_dict slap-parameters:configuration
${:extra-context}
[dynamic-template-cloudooo-parameters]
file = {{ file_location }}
fontconfig = {{ fontconfig_location }}
freetype = {{ freetype_location }}
haproxy = {{ haproxy_location }}
libICE = {{ libICE_location }}
libpng12 = {{ libpng12_location }}
libSM = {{ libSM_location }}
......@@ -40,8 +50,10 @@ buildout-bin-directory = {{ buildout_bin_directory }}
< = jinja2-template-base
template = {{ template_cloudooo }}
filename = instance-cloudoo.cfg
extensions = jinja2.ext.do
extra-context =
section parameter_dict dynamic-template-cloudooo-parameters
key ipv4_set slap-parameters:ipv4
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
......
software/cloudooo/software.cfg
View file @ dddb7942
......@@ -16,6 +16,7 @@ parts +=
check-recipe
# Create instance template
template
slapos-cookbook
# XXX: Workaround of SlapOS limitation
# Unzippig of eggs is required, as SlapOS do not yet provide nicely working
......@@ -24,11 +25,10 @@ unzip = true
# Local development
[slapos.cookbook-repository]
recipe = plone.recipe.command
stop-on-error = true
location = ${buildout:parts-directory}/${:_buildout_section_name_}
command = ${git:location}/bin/git clone --branch cloudooo --quiet http://git.erp5.org/repos/slapos.git ${:location}
update-command = cd ${:location} && ${git:location}/bin/git pull --quiet
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.git
branch = master
git-executable = ${git:location}/bin/git
[check-recipe]
recipe = plone.recipe.command
......@@ -36,6 +36,14 @@ stop-on-error = true
update-command = ${:command}
command = grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link
[slap-parameters]
recipe = slapos.cookbook:slapconfiguration
computer = ${slap-connection:computer-id}
partition = ${slap-connection:partition-id}
url = ${slap-connection:server-url}
key = ${slap-connection:key-file}
cert = ${slap-connection:cert-file}
[template-jinja2-base]
recipe = slapos.recipe.template:jinja2
template = ${:_profile_base_location_}/${:filename}.in
......@@ -54,7 +62,7 @@ context =
# XXX: "template.cfg" is hardcoded in instanciation recipe
filename = template.cfg
template = ${:_profile_base_location_}/instance.cfg.in
md5sum = 694205787e78c5d615d72d7b4b26d174
md5sum = 425cb2e76d46d53bb0b0eebdb8c1aa95
extra-context =
key buildout_bin_directory buildout:bin-directory
key dcron_location dcron:location
......@@ -63,6 +71,7 @@ extra-context =
key fonts_location fonts:location
key freetype_location freetype:location
key git_location git:location
key haproxy_location haproxy:location
key imagemagick_location imagemagick:location
key libICE_location libICE:location
key libSM_location libSM:location
......@@ -83,10 +92,9 @@ extra-context =
[template-cloudooo]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/instance-cloudoo.cfg.in
md5sum = 4c8608f9525be0f01a09d60b240315a9
md5sum = bbe84b4c9022db62c926e8a8a4bf02a1
mode = 640
[networkcache]
# signature certificates of the following uploaders.
# Romain Courteaud
......
software/davstorage/common.cfg 0 → 100644
View file @ dddb7942
[buildout]
versions = versions
extends =
../../component/apache-php/buildout.cfg
../../component/apache/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
../../stack/shacache-client.cfg
../../component/lxml-python/buildout.cfg
../../component/python-2.7/buildout.cfg
../../component/gzip/buildout.cfg
find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/
http://www.nexedi.org/static/packages/source/
http://www.nexedi.org/static/packages/source/hexagonit.recipe.download/
# Use only quite well working sites.
allow-hosts +=
*.googlecode.com
*.nexedi.org
*.python.org
*.sourceforge.net
alastairs-place.net
bitbucket.org
dist.repoze.org
effbot.org
github.com
launchpad.net
peak.telecommunity.com
sourceforge.net
www.dabeaz.com
www.owlfish.com
parts =
apache-php
application
template
lxml-python
eggs
instance-recipe-egg
unzip= true
[eggs]
recipe = zc.recipe.egg
eggs =
[instance-recipe]
egg = slapos.cookbook
module = davstorage
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[application]
recipe = hexagonit.recipe.download
url = http://garr.dl.sourceforge.net/project/ajaxplorer/ajaxplorer/dev-channel/4.3.4/ajaxplorer-core-4.3.4.tar.gz
md5sum = 2f2ff8bda7bbe841ef0e870c724eb74f
strip-top-level-dir = true
[template]
# Default template for the instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = bed788dee6daf05349c4577e7a7f1299
output = ${buildout:directory}/template.cfg
mode = 0644
[instance-davstorage]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-davstorage.cfg
md5sum = 699ecf4678386667f58a3391bab7af0f
output = ${buildout:directory}/template-davstorage.cfg
mode = 0644
[lxml-python]
python = python2.7
software/davstorage/development.cfg 0 → 100644
View file @ dddb7942
[buildout]
extends =
../../component/git/buildout.cfg
common.cfg
parts +=
slapos.cookbook-repository
slapos.toolbox-repository
check-recipe
develop =
${:parts-directory}/slapos.cookbook-repository
${:parts-directory}/slapos.toolbox-repository
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.git
branch = davstorage-ajaxplorer
git-executable = ${git:location}/bin/git
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.toolbox.git
branch = master
git-executable = ${git:location}/bin/git
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link
[versions]
zc.buildout = 1.6.0-dev-SlapOS-002
Jinja2 = 2.6
Werkzeug = 0.8.3
buildout-versions = 1.7
hexagonit.recipe.cmmi = 1.6
hexagonit.recipe.download = 1.6nxd002
meld3 = 0.6.10
openssl = 1.0.1c
# Required by:
# slapos.core
Flask = 0.9
# Required by:
# slapos.cookbook
PyXML = 0.8.4
slapos.recipe.template = 2.4.2
# Required by:
# slapos.cookbook==0.24
# slapos.core==0.14
# xml-marshaller==0.9.7
lxml = 3.1.0
# Required by:
# slapos.cookbook==0.24
netaddr = 0.7.10
# Required by:
# slapos.core==0.14
netifaces = 0.8
# Required by:
# slapos.cookbook==0.24
# slapos.core==0.14
# zc.buildout==1.5.3-dev-SlapOS-009
# zc.recipe.egg==1.3.2
setuptools = 0.6c12dev-r88846
# Required by:
# slapos.cookbook==0.73.1
slapos.core = 0.35.1
# Required by:
# slapos.core==0.35.1
supervisor = 3.0b1
# Required by:
# slapos.cookbook==0.73.1
xml-marshaller = 0.9.7
# Required by:
# slapos.cookbook==0.24
zc.recipe.egg = 1.3.2
# Required by:
# slapos.core==0.35.1
zope.interface = 4.0.5
software/davstorage/instance-davstorage.cfg
View file @ dddb7942
[buildout]
parts =
davstorage
url
publish-connection-informations
certificate-authority
ca-davstorage
cron
cron-entry-logrotate
logrotate
logrotate-entry-davstorage
request-frontend
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[url]
recipe = slapos.cookbook:publishurl
scheme = webdavs
user = $${davstorage:user}
port = $${davstorage:port}
host = $${davstorage:ip}
password = $${davstorage:password}
[publish-connection-informations]
recipe = slapos.cookbook:publish
webdav_access = https://$${request-frontend:connection-domain}
url = https://$${request-frontend-ajaxupload:connection-domain}
webdav_user = $${davstorage:user}
webdav_password = $${davstorage:password}
[davstorage]
recipe = slapos.cookbook:davstorage
user = user
port = 8080
password = insecure
port_webdav = 8080
port_ajax = 8070
ip = $${slap-network-information:global-ipv6}
# Path
......@@ -42,12 +44,15 @@ root = $${buildout:directory}
email-address = admin+davstorage@vifib.net
htpasswd-file = $${directory:davstorage-conf}/davstorage.htpasswd
promise = $${basedirectory:promises}/davstorage
php-ini-dir = $${directory:php-ini-dir}
tmp-dir = $${directory:tmp-php}
# Binaries
apache-binary = ${apache:location}/bin/httpd
apache-modules-dir = ${apache:location}/modules/
apache-mime-file = ${apache:location}/conf/mime.types
apache-htpasswd = ${apache:location}/bin/htpasswd
source = ${application:location}
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
......@@ -66,7 +71,7 @@ dcrond-binary = ${dcron:location}/sbin/crond
cron-entries = $${directory:cron-entries}
crontabs = $${directory:crontabs}
cronstamps = $${directory:cronstamps}
catcher = $${cron-simplelogger:binary}
catcher = $${cron-simplelogger:wrapper}
binary = $${basedirectory:services}/crond
[logrotate]
......@@ -91,8 +96,8 @@ command = $${logrotate:wrapper}
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
binary = $${rootdirectory:bin}/cron_simplelogger
output = $${directory:cronoutput}
wrapper = $${rootdirectory:bin}/cron_simplelogger
log = $${basedirectory:log}/crond.log
[logrotate-entry-davstorage]
......@@ -128,6 +133,7 @@ etc = $${buildout:directory}/etc/
var = $${buildout:directory}/var/
srv = $${buildout:directory}/srv/
bin = $${buildout:directory}/bin/
tmp = $${buildout:directory}/tmp/
[basedirectory]
recipe = slapos.cookbook:mkdirectory
......@@ -149,3 +155,38 @@ cron-entries = $${rootdirectory:etc}/cron.d/
crontabs = $${rootdirectory:etc}/crontabs/
cronstamps = $${rootdirectory:etc}/cronstamps/
cronoutput = $${basedirectory:log}/cron/
php-ini-dir = $${rootdirectory:etc}/php
tmp-php = $${rootdirectory:tmp}/php
# Request frontend
[request-frontend-ajaxupload]
<= slap-connection
recipe = slapos.cookbook:request
name = Frontend Ajax
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config = url https-only
config-https-only = true
config-url = https://[$${davstorage:ip}]:$${davstorage:port_ajax}/
return = domain
[request-frontend]
<= slap-connection
recipe = slapos.cookbook:request
name = Frontend Webdav
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config = url https-only
config-https-only = true
config-url = https://$${davstorage:user}:$${davstorage:password}@[$${davstorage:ip}]:$${davstorage:port_webdav}/
return = domain
# XXX Vivien: promise not working for now
#[frontend-ajaxupload-promise]
#recipe = slapos.cookbook:check_url_available
#path = $${basedirectory:promises}/frontend-ajaxupload
#url = $${request-frontend-ajaxupload:connection-site_url}
#dash_path = ${dash:location}/bin/dash
#curl_path = ${curl:location}/bin/curl
\ No newline at end of file
software/davstorage/software.cfg
View file @ dddb7942
[buildout]
find-links +=
http://www.nexedi.org/static/packages/source/slapos.buildout/
versions = versions
extends =
../../component/apache/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
../../stack/shacache-client.cfg
../../component/lxml-python/buildout.cfg
../../component/python-2.7/buildout.cfg
../../component/gzip/buildout.cfg
# Use only quite well working sites.
allow-hosts =
*.nexedi.org
*.python.org
*.sourceforge.net
dist.repoze.org
effbot.org
github.com
peak.telecommunity.com
psutil.googlecode.com
www.dabeaz.com
alastairs-place.net
parts =
template
lxml-python
apache
logrotate
dcron
eggs
gzip
instance-recipe-egg
unzip= true
[eggs]
recipe = zc.recipe.egg
eggs =
[instance-recipe]
egg = slapos.cookbook
module = davstorage
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[template]
# Default template for the instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = bed788dee6daf05349c4577e7a7f1299
output = ${buildout:directory}/template.cfg
mode = 0644
[instance-davstorage]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-davstorage.cfg
md5sum = 49c8e049a78e233d3a553e64c8914592
output = ${buildout:directory}/template-davstorage.cfg
mode = 0644
[lxml-python]
python = python2.7
../../component/git/buildout.cfg
common.cfg
parts +=
slapos.cookbook-repository
slapos.toolbox-repository
check-recipe
develop =
${:parts-directory}/slapos.cookbook-repository
${:parts-directory}/slapos.toolbox-repository
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.git
branch = davstorage-ajaxplorer
git-executable = ${git:location}/bin/git
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.toolbox.git
branch = master
git-executable = ${git:location}/bin/git
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link
[versions]
zc.buildout = 1.6.0-dev-SlapOS-002
Jinja2 = 2.6
Werkzeug = 0.7.1
buildout-versions = 1.6
hexagonit.recipe.cmmi = 1.5.0
Werkzeug = 0.8.3
buildout-versions = 1.7
hexagonit.recipe.cmmi = 1.6
hexagonit.recipe.download = 1.6nxd002
meld3 = 0.6.7
slapos.cookbook = 0.26
meld3 = 0.6.10
openssl = 1.0.1c
# Required by:
# slapos.core==0.14
Flask = 0.7.2
# slapos.core
Flask = 0.9
# Required by:
# slapos.cookbook==0.24
# slapos.cookbook
PyXML = 0.8.4
# Required by:
# slapos.recipe.template==1.1
collective.recipe.template = 1.9
slapos.recipe.template = 2.4.2
# Required by:
# slapos.cookbook==0.24
# slapos.core==0.14
# xml-marshaller==0.9.7
lxml = 2.3
lxml = 3.1.0
# Required by:
# slapos.cookbook==0.24
netaddr = 0.7.6
netaddr = 0.7.10
# Required by:
# slapos.core==0.14
#netifaces = 0.4
netifaces = 0.8
# Required by:
# slapos.cookbook==0.24
......@@ -115,15 +73,15 @@ netaddr = 0.7.6
setuptools = 0.6c12dev-r88846
# Required by:
# slapos.cookbook==0.24
slapos.core = 0.14
# slapos.cookbook==0.73.1
slapos.core = 0.35.1
# Required by:
# slapos.core==0.14
supervisor = 3.0a10
# slapos.core==0.35.1
supervisor = 3.0b1
# Required by:
# slapos.cookbook==0.24
# slapos.cookbook==0.73.1
xml-marshaller = 0.9.7
# Required by:
......@@ -131,5 +89,6 @@ xml-marshaller = 0.9.7
zc.recipe.egg = 1.3.2
# Required by:
# slapos.core==0.14
zope.interface = 3.7.0
# slapos.core==0.35.1
zope.interface = 4.0.5
software/davstorage/todo.txt 0 → 100644
View file @ dddb7942
TODO:
+ Move Ajaxplorer to a separate directory (currently in the www/ directory
accessed by webdav client, and so might delete files)
+ Make configuration part disappear ! (configuration when instance deploy,
no need for user to do it at each start of a new instance)
software/etherpad-lite/common.cfg 0 → 100644
View file @ dddb7942
[buildout]
versions = versions
extends =
../../stack/slapos.cfg
../../component/gcc/buildout.cfg
../../component/openssl/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/python-2.7/buildout.cfg
../../component/gzip/buildout.cfg
../../component/git/buildout.cfg
../../component/nodejs/buildout.cfg
../../component/postgresql/buildout.cfg
parts =
postgresql
nodejs
etherpad-lite-repository
install-deps
template
lxml-python
eggs
instance-recipe-egg
unzip= true
[eggs]
recipe = zc.recipe.egg
eggs =
[instance-recipe]
egg = slapos.cookbook
module = etherpad-lite
[instance-recipe-egg]
recipe = zc.recipe.egg
python = python2.7
eggs = ${instance-recipe:egg}
[etherpad-lite-repository]
recipe = slapos.recipe.build:gitclone
repository = http://github.com/ether/etherpad-lite.git
branch = develop
git-executable = ${git:location}/bin/git
[template]
# Default template for the instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = 7ab2a242df988bf5c10bf8002acac3bd
output = ${buildout:directory}/template.cfg
mode = 0644
[instance-etherpad-lite]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-etherpad-lite.cfg
md5sum = fd7249be8988155110234c7bb877abb9
output = ${buildout:directory}/template-etherpad-lite.cfg
mode = 0644
[template-conf]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/templates/${:filename}
mode = 0644
filename = settings.json.in
md5sum = 19ab39e6b3256c82fd54ce074488b136
location = ${buildout:parts-directory}/${:_buildout_section_name_}
[template-run-script]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/templates/${:filename}
mode = 0644
filename = run.sh.in
md5sum = eac870b5f30e735e109a48913af2fae3
location = ${buildout:parts-directory}/${:_buildout_section_name_}
[template-deps-script]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/templates/${:filename}
etherpad-location = ${etherpad-lite-repository:location}
nodejs-location = ${nodejs:location}
curl-location = ${curl:location}
postgre-location = ${postgresql:location}
mode = 0755
md5sum = 53d0d53d419bd9ee592d3e1a1c84c758
filename = installDeps.sh.in
output = ${etherpad-lite-repository:location}/bin/installDeps.sh
[install-deps]
recipe = plone.recipe.command
command = ${template-deps-script:output}
update-command = command
[lxml-python]
python = python2.7
software/etherpad-lite/development.cfg 0 → 100644
View file @ dddb7942
[buildout]
extends =
common.cfg
parts +=
slapos.cookbook-repository
slapos.toolbox-repository
check-recipe
develop =
${:parts-directory}/slapos.cookbook-repository
${:parts-directory}/slapos.toolbox-repository
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.git
branch = etherpad-lite
git-executable = ${git:location}/bin/git
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.toolbox.git
branch = master
git-executable = ${git:location}/bin/git
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link
\ No newline at end of file
software/etherpad-lite/instance-etherpad-lite.cfg 0 → 100644
View file @ dddb7942
[buildout]
parts =
etherpad-lite
publish-connection-informations
# frontend-etherpad
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[rootdirectory]
recipe = slapos.cookbook:mkdirectory
etc = $${buildout:directory}/etc/
var = $${buildout:directory}/var/
srv = $${buildout:directory}/srv/
bin = $${buildout:directory}/bin/
tmp = $${buildout:directory}/tmp/
[basedirectory]
recipe = slapos.cookbook:mkdirectory
log = $${rootdirectory:var}/log/
services = $${rootdirectory:etc}/run/
run = $${rootdirectory:etc}/run/
backup = $${rootdirectory:srv}/backup/
promises = $${rootdirectory:etc}/promise/
[directory]
recipe = slapos.cookbook:mkdirectory
etherpad-conf = $${rootdirectory:etc}/etherpad/
etherpad-repository-location = $${buildout:directory}/parts/etherpad-lite-repository
[publish-connection-informations]
recipe = slapos.cookbook:publish
url = $${request-frontend:connection-site_url}
[etherpad-conf-generation]
recipe = slapos.recipe.template
url = ${template-conf:location}/${template-conf:filename}
ip = $${slap-network-information:global-ipv6}
dirtydb-location = $${rootdirectory:var}/dirty.db
port = 9001
mode = 0644
output = $${directory:etherpad-conf}/settings.json
[etherpad-run-script]
recipe = slapos.recipe.template
url = ${template-run-script:location}/${template-run-script:filename}
etherpad-location = ${etherpad-lite-repository:location}
etherpad-repository-location = $${directory:etherpad-repository-location}
nodejs-location = ${nodejs:location}
etherpad-deps-script-location = ${template-deps-script:output}
etherpad-conf-location = $${etherpad-conf-generation:output}
etherpad-conf-name = settings-$${slap-connection:partition-id}.json
mode = 0755
output = $${rootdirectory:bin}/run.sh
# Command line comes from the run script of etherpad-lite
[etherpad-lite]
recipe = slapos.cookbook:wrapper
wrapper-path = $${basedirectory:run}/etherpad-lite
command-line = $${etherpad-run-script:output} -s $${etherpad-run-script:etherpad-conf-name}
[request-frontend]
<= slap-connection
recipe = slapos.cookbook:request
name = Frontend
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config = url
config-url = http://$${etherpad-conf-generation:ip}:$${etherpad-conf-generation:port}
return = site_url
[frontend-etherpad]
recipe = slapos.cookbook:check_url_available
path = $${basedirectory:promises}/frontend-etherpad
url = $${request-frontend:connection-site_url}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
\ No newline at end of file
software/etherpad-lite/instance.cfg 0 → 100644
View file @ dddb7942
[buildout]
parts =
switch_softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[switch_softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${instance-etherpad-lite:output}
software/etherpad-lite/software.cfg 0 → 100644
View file @ dddb7942
[buildout]
extends =
common.cfg
parts +=
slapos.cookbook-repository
slapos.toolbox-repository
check-recipe
develop =
${:parts-directory}/slapos.cookbook-repository
${:parts-directory}/slapos.toolbox-repository
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.git
branch = etherpad-lite
git-executable = ${git:location}/bin/git
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.toolbox.git
branch = master
git-executable = ${git:location}/bin/git
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link
\ No newline at end of file
software/etherpad-lite/templates/installDeps.sh.in 0 → 100755
View file @ dddb7942
#!/bin/sh
# Ugly setup
cd ${:etherpad-location}
export PATH=$PATH:${:postgre-location}/bin
#Is gnu-grep (ggrep) installed on SunOS (Solaris)
if [ $(uname) = "SunOS" ]; then
hash ggrep > /dev/null 2>&1 || {
echo "Please install ggrep (pkg install gnu-grep)" >&2
exit 1
}
fi
#Is wget installed?
hash ${:curl-location}/bin/curl > /dev/null 2>&1 || {
echo "Please install curl" >&2
exit 1
}
#Is node installed?
hash ${:nodejs-location}/bin/node > /dev/null 2>&1 || {
echo "Please install node.js ( http://nodejs.org )" >&2
exit 1
}
#Is npm installed?
hash ${:nodejs-location}/bin/npm > /dev/null 2>&1 || {
echo "Please install npm ( http://npmjs.org )" >&2
exit 1
}
echo "Ensure that all dependencies are up to date... If this is the first time you have run Etherpad please be patient."
(
mkdir -p node_modules
cd node_modules
[ -e ep_etherpad-lite ] || ln -s ../src ep_etherpad-lite
cd ep_etherpad-lite
${:nodejs-location}/bin/npm install --loglevel warn
) || {
rm -rf node_modules
exit 1
}
echo "Ensure jQuery is downloaded and up to date..."
DOWNLOAD_JQUERY="true"
NEEDED_VERSION="1.9.1"
if [ $DOWNLOAD_JQUERY = "true" ]; then
${:curl-location}/bin/curl -lo src/static/js/jquery.js http://code.jquery.com/jquery-$NEEDED_VERSION.js || exit 1
fi
#Remove all minified data to force node creating it new
echo "Clear minfified cache..."
rm -f var/minified*
echo "ensure custom css/js files are created..."
for f in "index" "pad" "timeslider"
do
if [ ! -f "src/static/custom/$f.js" ]; then
cp "src/static/custom/js.template" "src/static/custom/$f.js" || exit 1
fi
if [ ! -f "src/static/custom/$f.css" ]; then
cp "src/static/custom/css.template" "src/static/custom/$f.css" || exit 1
fi
done
exit 0
software/etherpad-lite/templates/run.sh.in 0 → 100755
View file @ dddb7942
#!/bin/sh
# Copy repository content from the software release
mkdir ${:etherpad-repository-location}
cp -R ${:etherpad-location}/* ${:etherpad-repository-location}
#Move to the folder where ep-lite is installed
cd ${:etherpad-repository-location}
# XXX Vivien: very ugly, definitely need to find a cleaner way
cp ${:etherpad-conf-location} ${:etherpad-repository-location}/${:etherpad-conf-name}
#Move to the node folder and start
echo "start..."
${:nodejs-location}/bin/node node_modules/ep_etherpad-lite/node/server.js $*
software/etherpad-lite/templates/settings.json.in 0 → 100644
View file @ dddb7942
/*
This file must be valid JSON. But comments are allowed
Please edit settings.json, not settings.json.template
*/
{
// Name your instance!
"title": "Etherpad",
// favicon default name
// alternatively, set up a fully specified Url to your own favicon
"favicon": "favicon.ico",
//IP and port which etherpad should bind at
"ip": "${:ip}",
"port" : "${:port}",
// Session Key, used for reconnecting user sessions
// Set this to a secure string at least 10 characters long. Do not share this value.
"sessionKey" : "",
/*
// Node native SSL support
// this is disabled by default
//
// make sure to have the minimum and correct file access permissions set
// so that the Etherpad server can access them
"ssl" : {
"key" : "/path-to-your/epl-server.key",
"cert" : "/path-to-your/epl-server.crt"
},
*/
//The Type of the database. You can choose between dirty, postgres, sqlite and mysql
//You shouldn't use "dirty" for for anything else than testing or development
"dbType" : "dirty",
//the database specific settings
"dbSettings" : {
"filename" : "${:dirtydb-location}"
},
/* An Example of MySQL Configuration
"dbType" : "mysql",
"dbSettings" : {
"user" : "root",
"host" : "localhost",
"password": "",
"database": "store"
},
*/
//the default text of a pad
"defaultPadText" : "Welcome to Etherpad!\n\nThis pad text is synchronized as you type, so that everyone viewing this page sees the same text. This allows you to collaborate seamlessly on documents!\n\nGet involved with Etherpad at http:\/\/etherpad.org\n",
/* Users must have a session to access pads. This effectively allows only group pads to be accessed. */
"requireSession" : false,
/* Users may edit pads but not create new ones. Pad creation is only via the API. This applies both to group pads and regular pads. */
"editOnly" : false,
/* if true, all css & js will be minified before sending to the client. This will improve the loading performance massivly,
but makes it impossible to debug the javascript/css */
"minify" : true,
/* How long may clients use served javascript code (in seconds)? Without versioning this
may cause problems during deployment. Set to 0 to disable caching */
"maxAge" : 21600, // 60 * 60 * 6 = 6 hours
/* This is the path to the Abiword executable. Setting it to null, disables abiword.
Abiword is needed to advanced import/export features of pads*/
"abiword" : null,
/* This setting is used if you require authentication of all users.
Note: /admin always requires authentication. */
"requireAuthentication": false,
/* Require authorization by a module, or a user with is_admin set, see below. */
"requireAuthorization": false,
/* Users for basic authentication. is_admin = true gives access to /admin.
If you do not uncomment this, /admin will not be available! */
/*
"users": {
"admin": {
"password": "changeme1",
"is_admin": true
},
"user": {
"password": "changeme1",
"is_admin": false
}
},
*/
// restrict socket.io transport methods
"socketTransportProtocols" : ["xhr-polling"],
/* The log level we are using, can be: DEBUG, INFO, WARN, ERROR */
"loglevel": "INFO",
//Logging configuration. See log4js documentation for further information
// https://github.com/nomiddlename/log4js-node
// You can add as many appenders as you want here:
"logconfig" :
{ "appenders": [
{ "type": "console",
// "category": "access"// only logs pad access
}
/*
, { "type": "file"
, "filename": "your-log-file-here.log"
, "maxLogSize": 1024
, "backups": 3 // how many log files there're gonna be at max
//, "category": "test" // only log a specific category
}*/
/*{ "type": "logLevelFilter",
"level": "warn" // filters out all log messages that have a lower level than "error"
, "appender":
{ Use whatever appender you want here }
}*/
/*
, { "type": "logLevelFilter"
, "level": "error" // filters out all log messages that have a lower level than "error"
, "appender":
{ "type": "smtp"
, "subject": "An error occured in your EPL instance!"
, "recipients": "bar@blurdybloop.com, baz@blurdybloop.com"
, "sendInterval": 60*5 // in secs -- will buffer log messages; set to 0 to send a mail for every message
, "transport": "SMTP", "SMTP": { // see https://github.com/andris9/Nodemailer#possible-transport-methods
"host": "smtp.example.com", "port": 465,
"secureConnection": true,
"auth": {
"user": "foo@example.com",
"pass": "bar_foo"
}
}
}
}*/
] }
}
software/etherpad-lite/todo.txt 0 → 100644
View file @ dddb7942
TODO:
+ Modify etherpad-lite to read absolute path correctly, so that we aren't
forced to move everything to the software release directory for it to work
software/neoppod/README.txt
View file @ dddb7942
......@@ -37,11 +37,6 @@ Defautls to 12.
Number of replicates.
Defaults to 0 (no resilience).
'verbose' (int, optional)
~~~~~~~~~~~~~~~~~~~~~~~~~
Non-zero to increase verbosity.
Defaults to 0.
'mysql-storage-count' (int, optional)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Number of MySQL-based storage nodes to deploy. One master node is deployed
......
software/neoppod/cluster.cfg.in
View file @ dddb7942
......@@ -19,10 +19,9 @@ partition-id = ${slap-connection:partition-id}
config-cluster = {{ slapparameter_dict['cluster'] }}
config-partitions = {{ slapparameter_dict.get('partitions', 12) }}
config-replicas = {{ slapparameter_dict.get('replicas', 0) }}
config-verbose = {{ dumps(slapparameter_dict.get('verbose', 0)) }}
[request-config]
base = cluster verbose
base = cluster
master = partitions replicas
node = masters
......
software/neoppod/instance-neo-admin.cfg.in
View file @ dddb7942
......@@ -16,7 +16,6 @@ ip = ${publish:ip}
port = ${publish:port}
cluster = ${slap-parameter:cluster}
masters = ${slap-parameter:masters}
verbose = ${slap-parameter:verbose}
[publish]
recipe = slapos.cookbook:publish
......@@ -33,6 +32,7 @@ link-binary = ${neo-admin:wrapper}
[directory]
recipe = slapos.cookbook:mkdirectory
promises = ${buildout:directory}/etc/promises
run = ${buildout:directory}/etc/run
log = ${buildout:directory}/var/log
bin = ${buildout:directory}/bin
......
software/neoppod/instance-neo-master.cfg.in
View file @ dddb7942
......@@ -17,7 +17,6 @@ cluster = ${slap-parameter:cluster}
partitions = ${slap-parameter:partitions}
replicas = ${slap-parameter:replicas}
masters = ${slap-parameter:masters}
verbose = ${slap-parameter:verbose}
[publish]
recipe = slapos.cookbook:publish
......@@ -34,6 +33,7 @@ link-binary = ${neo-master:wrapper}
[directory]
recipe = slapos.cookbook:mkdirectory
promises = ${buildout:directory}/etc/promises
run = ${buildout:directory}/etc/run
log = ${buildout:directory}/var/log
bin = ${buildout:directory}/bin
......
software/neoppod/instance-neo-storage-mysql.cfg.in
View file @ dddb7942
......@@ -10,6 +10,7 @@ database = neo
conf-file = ${directory:etc}/mariadb.cnf
# No networking (ip & port options)
data-directory = ${directory:srv_mariadb}
bin-directory = ${directory:bin}
pid-file = ${directory:var_run}/mariadb.pid
socket = ${directory:var_run}/mariadb.sock
error-log = ${directory:log}/mariadb_error.log
......@@ -33,7 +34,6 @@ logfile = ${directory:log}/neostorage.log
ip = [${slap-network-information:global-ipv6}]
cluster = ${slap-parameter:cluster}
masters = ${slap-parameter:masters}
verbose = ${slap-parameter:verbose}
database-adapter = MySQL
database-parameters = ${mariadb-instance:user}:${mariadb-instance:password}@${mariadb-instance:database}${mariadb-instance:socket}
wait-database = 60
......
software/neoppod/software.cfg
View file @ dddb7942
# Note on COOKBOOK/END COOKBOOK: the areas delimited by those tags should be
# dropped when a cookbook version containing neoppod recipe is released as an
# egg on pypi.
# Note on LXML/END LXML: they delimit areas where lxml magic is needed. lxml is
# a slapos.cookbook dependency, so it should be fetched automatically. But when
# automatically fetched, it gets built against system headers/libs, which is
......@@ -10,9 +7,6 @@
[buildout]
extends =
../../stack/slapos.cfg
#COOKBOOK
../../component/git/buildout.cfg
#END COOKBOOK
#LXML
../../component/lxml-python/buildout.cfg
#END LXML
......@@ -20,19 +14,9 @@ extends =
../../component/mariadb/buildout.cfg
../../component/mysql-python/buildout.cfg
develop =
#COOKBOOK
# Note: as we develop a path we fetch, first buildout execution will fail as
# develop couldn't find the clone. Second (and later) runs will succeed.
${:parts-directory}/slapos.cookbook-repository
#END COOKBOOK
parts =
#COOKBOOK
slapos.cookbook-repository
check-recipe
#END COOKBOOK
slapos-deps-eggs
slapos-cookbook
# NEO & dependencies
python2.7
mariadb
......@@ -47,29 +31,12 @@ eggs =
#LXML
${lxml-python:egg}
#END LXML
#COOKBOOK
slapos.cookbook
#END COOKBOOK
#LXML
[lxml-python]
python = python2.7
#END LXML
#COOKBOOK
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.git
branch = neoppod
git-executable = ${git:location}/bin/git
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = test -e ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link
#END COOKBOOK
[base-template]
recipe = slapos.recipe.template:jinja2
template = ${:_profile_base_location_}/${:filename}.in
......@@ -85,12 +52,12 @@ context =
[template-neo-master]
< = base-template
filename = instance-neo-master.cfg
md5sum = b48e9446fb8fdce24d954d7d4f4c7460
md5sum = 8947620b15535b95163c11a0efab546e
[template-neo-storage-mysql]
< = base-template
filename = instance-neo-storage-mysql.cfg
md5sum = e767c66e7de58232de30bdf26188efe1
md5sum = 518c8c41ce73f0b608fbb1c3b889ca1b
extra-context =
key mariadb_location mariadb:location
key master_cfg template-neo-master:rendered
......@@ -98,12 +65,12 @@ extra-context =
[template-neo-admin]
< = base-template
filename = instance-neo-admin.cfg
md5sum = fcf1a5c937a609a934c8ccd7c1326726
md5sum = 987f69333c563898cab21455509f8024
[template-cluster]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/cluster.cfg.in
md5sum = fe37b2ff36fee65b44945393f63e51c3
md5sum = 7177d129e171b20044bf435ed549af43
[template-instance]
< = base-template
......
software/slaprunner/TODO.txt 0 → 100644
View file @ dddb7942
- Use https to connect to the runner. !!! Critical
- resilient sr: Cloned instances should not launch slapgrid-sr if it was not launched on export instance
software/slaprunner/common.cfg
View file @ dddb7942
[buildout]
extends =
../../component/bash/buildout.cfg
../../component/cloud9/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/dropbear/buildout.cfg
../../component/git/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/rsync/buildout.cfg
../../stack/flask.cfg
../../stack/shacache-client.cfg
../../stack/resilient/buildout.cfg
../../stack/slapos.cfg
parts =
slapos.cookbook-repository
rdiff-backup
template
slapos-cookbook
eggs
instance-runner-import
instance-runner-export
# slapos-cookbook
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
output = ${buildout:directory}/template.cfg
md5sum = 5de75f295f9382a587343718bb1be124
mode = 0644
md5sum = 5307e4200f044ae57b504ad68444491c
[template-runner]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner.cfg
output = ${buildout:directory}/template-runner.cfg
md5sum = 91d6550c43b7a43a999724af4650ae40
md5sum = e1d9aeeb3b02dfb3578eddfddd44d053
mode = 0644
[instance-resilient]
recipe = slapos.recipe.template:jinja2
template = ${:_profile_base_location_}/instance-resilient.cfg.jinja2
rendered = ${buildout:directory}/instance-resilient.cfg
md5sum = f533d354da36e1bb10819fab8a90109a
context = key buildout buildout:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
import-list = file parts template-parts:destination
file replicated template-replicated:destination
mode = 0644
[instance-runner-import]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner-import.cfg.in
output = ${buildout:directory}/instance-runner-import.cfg
md5sum = b37ec3af1898834041d8032ff755bac3
mode = 0644
[instance-runner-export]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner-export.cfg.in
output = ${buildout:directory}/instance-runner-export.cfg
md5sum = 4028924d0edb61bdcfbf03bb2bac43b8
mode = 0644
[eggs]
recipe = z3c.recipe.scripts
eggs =
${lxml-python:egg}
cns.recipe.symlink
hexagonit.recipe.download
inotifyx
netaddr
slapos.libnetworkcache
slapos.toolbox[flask_auth]
slapos.core
cns.recipe.symlink
xml_marshaller
pytz
software/slaprunner/development.cfg
View file @ dddb7942
......@@ -10,21 +10,24 @@ extends = common.cfg
parts +=
slapos.cookbook-repository
slapos.core-repository
slapos.toolbox-repository
check-recipe
# slapos.toolbox-repository
# slapos.core-repository
# check-recipe
develop =
${:parts-directory}/slapos.cookbook-repository
${:parts-directory}/slapos.core-repository
${:parts-directory}/slapos.toolbox-repository
# ${:parts-directory}/slapos.toolbox-repository
# ${:parts-directory}/slapos.core-repository
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.toolbox.git
branch = slaprunner
git-executable = ${git:location}/bin/git
#[slapos.toolbox-repository]
#recipe = slapos.recipe.build:gitclone
#repository = http://git.erp5.org/repos/slapos.toolbox.git
#branch = slaprunner-resiliency
#git-executable = ${git:location}/bin/git
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
......@@ -32,17 +35,18 @@ repository = http://git.erp5.org/repos/slapos.git
branch = slaprunner
git-executable = ${git:location}/bin/git
[slapos.core-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.core.git
branch = master
git-executable = ${git:location}/bin/git
#[slapos.core-repository]
#recipe = slapos.recipe.build:gitclone
#repository = http://git.erp5.org/repos/slapos.core.git
#branch = master
#git-executable = ${git:location}/bin/git
#
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.core.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link
# grep parts ${buildout:develop-eggs-directory}/slapos.core.egg-link &&
software/slaprunner/instance-resilient.cfg.jinja2 0 → 100644
View file @ dddb7942
# vim: set ft=cfg:
{% import 'parts' as parts %}
{% import 'replicated' as replicated %}
[buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
# += because we need to take up parts (like instance-custom, slapmonitor etc) from the profile we extended
parts +=
{{ parts.replicate("runner", "3") }}
publish-connection-informations
{{ replicated.replicate("runner", "3", "runner-export", "runner-import") }}
# Bubble up the parameters
[request-runner]
return = url ssh-public-key ssh-url notification-id ip backend_url url cloud9_url ssh_command password_recovery_code
config = instance-amount debug domain number authorized-key notify ip-list namebase runner1-computer-guid pbs-runner1-computer-guid runner2-computer-guid pbs-runner2-computer-guid runner3-computer-guid pbs-runner3-computer-guid
# XXX Cedric LN Ugly hack, resilient stack and slaprunner stack sharing too much ssh sections
config-authorized-key = ${request-pbs-runner-1:connection-ssh-key} ${request-pbs-runner-2:connection-ssh-key} ${slap-parameter:authorized-key}
config-instance-amount = ${slap-parameter:instance-amount}
config-debug = ${slap-parameter:debug}
config-runner1-computer-guid = ${slap-parameter:runner1-computer-guid}
config-pbs-runner1-computer-guid = ${slap-parameter:pbs-runner1-computer-guid}
config-runner2-computer-guid = ${slap-parameter:runner2-computer-guid}
config-pbs-runner2-computer-guid = ${slap-parameter:pbs-runner2-computer-guid}
config-runner3-computer-guid = ${slap-parameter:runner3-computer-guid}
config-pbs-runner3-computer-guid = ${slap-parameter:pbs-runner3-computer-guid}
config-domain = ${slap-parameter:domain}
[publish-connection-informations]
recipe = slapos.cookbook:publish
backend_url = ${request-runner:connection-backend_url}
url = ${request-runner:connection-url}
cloud9_url = ${request-runner:connection-cloud9_url}
ssh_command = ${request-runner:connection-ssh_command}
password_recovery_code = ${request-runner:connection-password_recovery_code}
[slap-parameter]
# Default parameters for distributed deployment
# I.e state "backup1 of maria should go there, ..."
# XXX-Cedric: Hardcoded number of backups. Should be dynamically generated.
runner1-computer-guid =
pbs-runner1-computer-guid =
runner2-computer-guid =
pbs-runner2-computer-guid =
runner3-computer-guid =
pbs-runner3-computer-guid =
# XXX-Cedric: Hardcoded parameters. Should be dynamically generated.
domain =
authorized-key =
instance-amount = 10
debug = false
software/slaprunner/instance-runner-export.cfg.in 0 → 100644
View file @ dddb7942
[buildout]
extends = ${template-runner:output}
${pbsready-export:output}
parts +=
urls
slaprunner
cron-entry-backup
[exporter]
recipe = slapos.cookbook:slaprunner.export
wrapper = $${directory:bin}/$${slap-parameter:namebase}-exporter
srv-directory = $${directory:srv}
etc-directory = $${directory:etc}
backup-directory = $${directory:backup}
shell-binary = ${dash:location}/bin/dash
rsync-binary = ${rsync:location}/bin/rsync
# Extends publish section with resilient parameters
[urls]
<= resilient-publish-connection-parameter
backend_url = $${slaprunner:access-url}
url = $${request-frontend:connection-site_url}
cloud9_url = $${cloud9:access-url}
ssh_command = ssh $${dropbear-runner-server:host} -p $${dropbear-runner-server:port}
password_recovery_code = $${recovery-code:passwd}
software/slaprunner/instance-runner-import.cfg.in 0 → 100644
View file @ dddb7942
[buildout]
extends = ${template-runner:output}
${pbsready-import:output}
parts +=
slaprunner
# have to repeat the next one, as it's not inherited from pbsready-import
import-on-notification
[importer]
recipe = slapos.cookbook:slaprunner.import
wrapper = $${directory:bin}/$${slap-parameter:namebase}-importer
srv-directory = $${directory:srv}
etc-directory = $${directory:etc}
backup-directory = $${directory:backup}
shell-binary = ${dash:location}/bin/dash
rsync-binary = ${rsync:location}/bin/rsync
curl-binary = ${curl:location}/bin/curl
backend-url = $${slaprunner:access-url}
software/slaprunner/instance-runner.cfg
View file @ dddb7942
......@@ -3,7 +3,7 @@ parts =
cloud9
slaprunner
test-runner
sshkeys-dropbear
sshkeys-dropbear-runner
dropbear-server-add-authorized-key
sshkeys-authority
publish-connection-informations
......@@ -13,7 +13,6 @@ parts =
dropbear-promise
symlinks
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
......@@ -82,9 +81,9 @@ instance-directory = $${runnerdirectory:instance-root}
etc_dir = $${directory:etc}
log_dir = $${directory:log}
run_dir = $${directory:run}
ssh_client = $${sshkeys-dropbear:wrapper}
public_key = $${sshkeys-dropbear:public-key}
private_key = $${sshkeys-dropbear:private-key}
ssh_client = $${sshkeys-dropbear-runner:wrapper}
public_key = $${sshkeys-dropbear-runner:public-key}
private_key = $${sshkeys-dropbear-runner:private-key}
ipv4 = $${slap-network-information:local-ipv4}
ipv6 = $${slap-network-information:global-ipv6}
proxy_port = 50000
......@@ -121,28 +120,28 @@ keys-directory = $${sshkeys-directory:keys}
wrapper = $${directory:services}/sshkeys_authority
keygen-binary = ${dropbear:location}/bin/dropbearkey
[dropbear-server]
[dropbear-runner-server]
recipe = slapos.cookbook:dropbear
host = $${slap-network-information:global-ipv6}
port = 2222
port = 22222
home = $${directory:ssh}
wrapper = $${directory:bin}/raw_sshd
shell = /bin/bash
wrapper = $${directory:bin}/runner_sshd
shell = ${bash:location}/bin/bash
rsa-keyfile = $${directory:ssh}/server_key.rsa
dropbear-binary = ${dropbear:location}/sbin/dropbear
[sshkeys-dropbear]
[sshkeys-dropbear-runner]
<= sshkeys-authority
recipe = slapos.cookbook:sshkeys_authority.request
name = dropbear
type = rsa
executable = $${dropbear-server:wrapper}
public-key = $${dropbear-server:rsa-keyfile}.pub
private-key = $${dropbear-server:rsa-keyfile}
wrapper = $${directory:services}/sshd
executable = $${dropbear-runner-server:wrapper}
public-key = $${dropbear-runner-server:rsa-keyfile}.pub
private-key = $${dropbear-runner-server:rsa-keyfile}
wrapper = $${directory:services}/runner_sshd
[dropbear-server-add-authorized-key]
<= dropbear-server
<= dropbear-runner-server
recipe = slapos.cookbook:dropbear.add_authorized_key
key = $${slap-parameter:authorized-key}
......@@ -166,7 +165,7 @@ recipe = slapos.cookbook:publish
backend_url = $${slaprunner:access-url}
url = $${request-frontend:connection-site_url}
cloud9_url = $${cloud9:access-url}
ssh_command = ssh $${dropbear-server:host} -p $${dropbear-server:port}
ssh_command = ssh $${dropbear-runner-server:host} -p $${dropbear-runner-server:port}
password_recovery_code = $${recovery-code:passwd}
......@@ -194,8 +193,8 @@ curl_path = ${curl:location}/bin/curl
[dropbear-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/dropbear
hostname = $${dropbear-server:host}
port = $${dropbear-server:port}
hostname = $${dropbear-runner-server:host}
port = $${dropbear-runner-server:port}
[symlinks]
recipe = cns.recipe.symlink
......
software/slaprunner/instance.cfg
View file @ dddb7942
......@@ -8,4 +8,11 @@ offline = true
[switch_softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${template-runner:output}
\ No newline at end of file
default = ${template-runner:output}
resilient = ${instance-resilient:rendered}
runner = ${template-runner:output}
runner-import = ${instance-runner-import:output}
runner-export = ${instance-runner-export:output}
frozen = ${instance-frozen:output}
pull-backup = ${template-pull-backup:output}
software/slaprunner/software.cfg
View file @ dddb7942
......@@ -5,7 +5,7 @@
# 2/ Define list of trusted certificates for the cache.
[buildout]
extends = common.cfg
extends = development.cfg
[networkcache]
# signature certificates of the following uploaders.
......@@ -55,13 +55,13 @@ meld3 = 0.6.10
netaddr = 0.7.10
plone.recipe.command = 1.1
pycrypto = 2.6
pytz = 2012j
slapos.cookbook = 0.71.1
slapos.core = 0.34
slapos.libnetworkcache = 0.13.3
pytz = 2013b
#slapos.cookbook = 0.71.1
slapos.core = 0.35.1
slapos.libnetworkcache = 0.13.4
slapos.recipe.build = 0.11.5
slapos.recipe.template = 2.4.2
slapos.toolbox = 0.34.0
#slapos.toolbox = 0.34.0
smmap = 0.8.2
xml-marshaller = 0.9.7
z3c.recipe.scripts = 1.0.1
......
software/varnish/instance-varnish.cfg.in 0 → 100644
View file @ dddb7942
{% set web_checker_parameter = slapparameter_dict['web-checker-mail-address'] -%}
[buildout]
parts =
publish-varnish-connection-information
varnish-instance
cron
cron-entry-logrotate
{# When web_checker related parameter is given, web_checker will be enabled.#}
{% if web_checker_parameter is defined %}
web-checker
cron-entry-web-checker
logrotate-entry-web-checker
{% endif %}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[publish-varnish-connection-information]
recipe = slapos.cookbook:publishurl
url = http://${varnish-instance:ip}:${varnish-instance:server-port}/
[varnish-instance]
recipe = slapos.cookbook:generic.varnish
# Network options
ip = ${slap-network-information:local-ipv4}
server-port = 6001
manager-port = 6002
stunnel-port = 6003
# Paths: Running wrappers
varnishd-wrapper = ${basedirectory:services}/varnishd
varnishlog-wrapper = ${rootdirectory:bin}/varnishlog
stunnel-wrapper = ${basedirectory:services}/stunnel
# Binary information
varnishd-binary = {{ parameter_dict['varnish'] }}/sbin/varnishd
varnishlog-binary = {{ parameter_dict['varnish'] }}/bin/varnishlog
shell-path = {{ parameter_dict['dash'] }}/bin/dash
stunnel-binary = {{ parameter_dict['stunnel'] }}/bin/stunnel
gcc-location = {{ parameter_dict['gcc'] }}/bin
# Configuration by VCL
vcl-file = ${rootdirectory:etc}/default.vcl
pid-file = ${basedirectory:run}/varnishd.pid
stunnel-conf-file = ${rootdirectory:etc}/stunnel.conf
stunnel-pid-file = ${basedirectory:run}/stunnel.pid
varnish-data = ${directory:varnish-data}
# this will pass at -n option
varnish-instance-name = ${directory:varnish-instance}
web-checker = {{ parameter_dict['buildout-bin-directory'] }}/web_checker_utility
[cron-entry-web-checker]
<= cron
recipe = slapos.cookbook:cron.d
name = web-checker
frequency = 0 0 * * *
command = ${varnish-instance:web-checker} ${web-checker:web-checker-config}
[web-checker]
recipe = slapos.cookbook:webchecker
web-checker-config = ${rootdirectory:etc}/web_checker.cfg
web-checker-working-directory = ${directory:web-checker}
frontend-url = ${varnish-instance:ip}:${varnish-instance:server-port}
wget-binary-path = {{ parameter_dict['wget'] }}/bin/wget
varnishlog-binary-path = ${varnish-instance:varnishlog-wrapper}
web-checker-log = ${basedirectory:log}/web-checker.log
[cron]
recipe = slapos.cookbook:cron
dcrond-binary = {{ parameter_dict['dcron'] }}/sbin/crond
cron-entries = ${directory:cron-entries}
crontabs = ${directory:crontabs}
cronstamps = ${directory:cronstamps}
binary = ${basedirectory:services}/crond
catcher = ${cron-simplelogger:wrapper}
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
wrapper = ${rootdirectory:bin}/cron_simplelogger
log = ${basedirectory:log}/cron.log
[cron-entry-logrotate]
<= cron
recipe = slapos.cookbook:cron.d
name = logrotate
frequency = 0 0 * * *
command = ${logrotate:wrapper}
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = {{ parameter_dict['logrotate'] }}/usr/sbin/logrotate
gzip-binary = {{ parameter_dict['gzip'] }}/bin/gzip
gunzip-binary = {{ parameter_dict['gzip'] }}/bin/gunzip
# Directories
wrapper = ${rootdirectory:bin}/logrotate
conf = ${rootdirectory:etc}/logrotate.conf
logrotate-entries = ${directory:logrotate-entries}
backup = ${directory:logrotate-backup}
state-file = ${rootdirectory:srv}/logrotate.status
[logrotate-entry-web-checker]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = web-checker
log = ${web-checker:web-checker-log}
frequency = daily
rotate-num = 30
sharedscripts = true
notifempty = true
create = true
[basedirectory]
recipe = slapos.cookbook:mkdirectory
services = ${rootdirectory:etc}/run
run = ${rootdirectory:var}/run
backup = ${rootdirectory:srv}/backup
log = ${rootdirectory:var}/log
backup = ${rootdirectory:srv}/backup
[directory]
recipe = slapos.cookbook:mkdirectory
varnish-data = ${rootdirectory:srv}/varnish
varnish-instance = ${directory:varnish-data}/instance
cron-entries = ${rootdirectory:etc}/cron.d
crontabs = ${rootdirectory:etc}/crontabs
cronstamps = ${rootdirectory:etc}/cronstamps
logrotate-backup = ${basedirectory:backup}/logrotate
logrotate-entries = ${rootdirectory:etc}/logrotate.d
web-checker = ${rootdirectory:srv}/web-checker
[rootdirectory]
recipe = slapos.cookbook:mkdirectory
etc = ${buildout:directory}/etc
var = ${buildout:directory}/var
srv = ${buildout:directory}/srv
bin = ${buildout:directory}/bin
software/varnish/instance.cfg.in 0 → 100644
View file @ dddb7942
[buildout]
parts =
switch-softwaretype
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[slap-parameters]
recipe = slapos.cookbook:slapconfiguration
computer = ${slap-connection:computer-id}
partition = ${slap-connection:partition-id}
url = ${slap-connection:server-url}
key = ${slap-connection:key-file}
cert = ${slap-connection:cert-file}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = ${buildout:parts-directory}/${:_buildout_section_name_}/${:filename}
extra-context =
context =
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slapparameter_dict slap-parameters:configuration
${:extra-context}
[dynamic-template-varnish-parameters]
dash = {{ dash_location }}
dcron = {{ dcron_location }}
gcc = {{ gcc_location }}
gzip = {{ gzip_location }}
logrotate = {{ logrotate_location }}
stunnel = {{ stunnel_location }}
varnish = {{ varnish_location }}
wget = {{ wget_location }}
buildout-bin-directory = {{ buildout_bin_directory }}
[dynamic-template-varnish]
< = jinja2-template-base
template = {{ template_varnish }}
filename = instance-varnish.cfg
extra-context =
section parameter_dict dynamic-template-varnish-parameters
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${dynamic-template-varnish:rendered}
[slap-connection]
# part to migrate to new - separated words
computer-id = ${slap_connection:computer_id}
partition-id = ${slap_connection:partition_id}
server-url = ${slap_connection:server_url}
software-release-url = ${slap_connection:software_release_url}
key-file = ${slap_connection:key_file}
cert-file = ${slap_connection:cert_file}
software/varnish/software.cfg 0 → 100644
View file @ dddb7942
This diff is collapsed.
stack/cloudooo.cfg
View file @ dddb7942
......@@ -49,6 +49,7 @@ extends =
../component/dcron/buildout.cfg
../component/coreutils/buildout.cfg
../component/cloudooo/buildout.cfg
../component/haproxy/buildout.cfg
versions = versions
......@@ -77,6 +78,7 @@ parts =
ffmpeg
rdiff-backup
haproxy
cloudooo
# Local development
......
stack/erp5/buildout.cfg
View file @ dddb7942
......@@ -19,7 +19,7 @@ allow-hosts += pybrary.net
extends =
# Exact version of Zope
https://raw.github.com/zopefoundation/Zope/2.13.20/versions.cfg
https://raw.github.com/zopefoundation/Zope/2.13.21/versions.cfg
../../stack/slapos.cfg
../../component/logrotate/buildout.cfg
../../component/dcron/buildout.cfg
......
stack/resilient/README.txt
View file @ dddb7942
......@@ -127,6 +127,27 @@ wrapper. I suggest you only add options and specify your export/import recipe.
Checking that it works
----------------------
To check that your software instance is resilient you can proceed this way:
Once all instances are successfully deployed, go to your export instance, connect as the instance user and run:
$ ~/bin/exporter
It is the script responsible for triggering the resiliency stack on your instance. After doing a backup of your data, it will notify the pull-backup instances of a new backup, triggering the transfer of this data to the import instances.
Once this script is run successfully, go to your import instance, connect as its instance user and check ~/srv/backup/"your sofwtare"/, the location of the data you wanted to receive. The last part of the resiliency is up to your import script.
DEBUGGING:
Here is a partial list of things you can check to understand what is causing the problem:
- Check that your import script does not fail and successfully places your data in ~/srv/backup/"your software" (as the import instance user) by runnig:
$ ~/bin/"your software"-exporter
- Check the export instance script is run successfully as this instance user by running:
$ ~/bin/exporter
- Check the pull-instance system did its job by going to one of your pull-backup instance, connect as its user and check the log : ~/var/log/equeue.log
-----------------------------------------------------------------------------------------
Finally, instance-mysoftware-import.cfg.in and
......
stack/resilient/buildout.cfg
View file @ dddb7942
[buildout]
extends =
../../component/gzip/buildout.cfg
../../component/rdiff-backup/buildout.cfg
parts =
rdiff-backup
pbsready
pbsready-import
pbsready-export
......@@ -9,6 +13,9 @@ parts =
instance-frozen
template-resilient
# needed tools for resiliency
gzip
#----------------
#--
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7