Skip to content

slapos
slapos
Commit d007050d authored by Alain Takoudjou's avatar Alain Takoudjou
Browse files
Options

Merge branch 'openstack' into grid-computing

parents 55b2bae2 b7e318af
Hide whitespace changes
Inline Side-by-side
Showing with 63115 additions and 2371 deletions
CHANGES.txt
View file @ d007050d
Changes
=======
0.78.4.dev (2013-07-18)
-----------------------
0.84.2 (2013-10-04)
-------------------
* sshkeys_authority: don't allow to return None as parameter. [9e340a0]
0.84.1 (2013-10-03)
-------------------
* Resiliency: PBS: promise should NOT bang. [64886cd]
0.84 (2013-09-30)
-----------------
* Request.py: improve instance-state handling. [ba5f160]
* Resilient recipe: remove hashing of urls/names. [ee2aec8]
* Resilient pbs recipe: recover from rdiff-backup failures. [be7f2fc, 92ee0c3]
* Resilience: add pidfiles in PBS. [0b3ad5c]
* Resilient: don't hide exception, print it. [05b3d64, d2b0494]
* Resiliency: Only keep 10 increments of backup. [4e89e33]
* KVM SR: add fallback in case of download exception. [de8d796]
* slaprunner: don't check certificate for importer. [53dc772]
0.83.1 (2013-09-10)
------------------
* slapconfiguration: fixes previous releasei (don't encode tap_set because it's not a string). [Cedric de Saint Martin]
0.83 (2013-09-10)
-----------------
* slaprunner recipe: remove trailing / from master_url. [Cedric de Saint Martin]
* librecipe: add pidfile option for singletons. [Cedric de Saint Martin]
* Resiliency: Use new pidfile option. [Cedric de Saint Martin]
* Fix request.py for slave instances. [Cedric de Saint Martin]
* slapconfiguration recipe: cast some parameters from unicode to str. [Cedric de Saint Martin]
0.82 (2013-08-30)
-----------------
* Certificate Authority: Can receice certificate to install. [Cedric Le Ninivin]
* Squid: Add squid recipe. [Romain Courteaud]
* Request: Trasmit instace state to requested instances. [Benjamin Blanc / Cédric Le Ninivin]
* Slapconfiguration: Now return instance state. [Cédric Le Ninivin]
* Apache Frontend: Remove recipe
0.81 (2013-08-12)
-----------------
* KVM SR: implement resiliency test. [Cedric de Saint Martin]
0.80 (2013-08-06)
----------------
* Add a simple readline recipe. [f4fce7e]
0.79 (2013-08-06)
-----------------
* KVM SR: Add support for NAT based networking (User Mode Network). [627895fe35]
* KVM SR: add virtual-hard-drive-url support. [aeb5df40cd, 8ce5a9aa1d0, a5034801aa9]
* Fix regression in GenericBaseRecipe.generatePassword. [3333b07d33c]
0.78.5 (2013-08-06)
-------------------
* check_url_available: add option to check secure links [6cbce4d8231]
0.78.4 (2013-08-06)
-------------------
* slapos.cookbook:slaprunner: Update to use https. [Cedric Le Ninivin]
0.78.3 (2013-07-18)
......
component/cloud9/buildout.cfg
View file @ d007050d
......@@ -44,6 +44,13 @@ filename = cloud9-session-directory.patch
download-only = true
md5sum = 5dc8cc28447ed3747b8a53c768d872aa
[cloud9-socket.patch]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:filename}
filename = cloud9-socket.patch
download-only = true
#md5sum = 5dc8cc28447ed3747b8a53c768d872aa
[cloud9-git]
# Online IDE written in javascript/node.js
# URL : c9.io
......@@ -55,7 +62,7 @@ commit = f7d102bc225c922f116d2cea52a746d64343ea59
repository = https://github.com/ajaxorg/cloud9.git
location = ${buildout:parts-directory}/${:_buildout_section_name_}
environment = export GIT_SSL_NO_VERIFY=true; export PATH=${git:location}/bin:${nodejs:location}/bin:${node-sm:location}/node_modules/sm/bin:$PATH; export CPPFLAGS="-I${libxml2:location}/include -I${nodejs:location}/include"; export LDFLAGS="-L${libxml2:location}/lib -Wl,-rpath=${libxml2:location}/lib"; export HOME=${:location};
command = ${:environment} (git clone --quiet ${:repository} ${:location} && cd ${:location} && git reset --hard ${:commit} && ${node-sm:location}/node_modules/.bin/sm install && patch -p1 < ${cloud9-session-directory.patch:location}/${cloud9-session-directory.patch:filename}) || (rm -fr ${:location}; exit 1)
command = ${:environment} (git clone --quiet ${:repository} ${:location} && cd ${:location} && git reset --hard ${:commit} && ${node-sm:location}/node_modules/.bin/sm install && patch -p1 < ${cloud9-session-directory.patch:location}/${cloud9-session-directory.patch:filename} && ${node-sm:location}/node_modules/.bin/sm install && patch -p1 < ${cloud9-socket.patch:location}/${cloud9-socket.patch:filename}) || (rm -fr ${:location}; exit 1)
update-command =
executable = ${:location}/server.js
......
component/cloud9/cloud9-socket.patch 0 → 100644
View file @ d007050d
diff --git a/node_modules/smith.io/node_modules/engine.io/node_modules/engine.io-client/dist/engine.io-dev.js b/node_modules/smith.io/node_modules/engine.io/node_modules/engine.io-client/dist/engine.io-dev.js
index fa7e54a..14b8e67 100644
--- a/node_modules/smith.io/node_modules/engine.io/node_modules/engine.io-client/dist/engine.io-dev.js
+++ b/node_modules/smith.io/node_modules/engine.io/node_modules/engine.io-client/dist/engine.io-dev.js
@@ -2126,7 +2126,7 @@ Polling.prototype.uri = function () {
query = '?' + query;
}
- return schema + '://' + this.host + port + this.path + query;
+ return this.path + query;
};
});require.register("transports/websocket.js", function(module, exports, require, global){
component/coreutils/buildout.cfg
View file @ d007050d
......@@ -19,7 +19,11 @@ environment =
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${:test} -x ${:test} -a -x ${:cat} -a -x ${:rm}
command = ${:test} -x ${:test} -a -x ${:cat} -a -x ${:rm} -a -x ${:echo} -a -x ${:date} -a -x ${:md5sum} -a -x ${:basename}
test = ${coreutils:location}/bin/test
cat = ${coreutils:location}/bin/cat
rm = ${coreutils:location}/bin/rm
echo = ${coreutils:location}/bin/echo
date = ${coreutils:location}/bin/date
md5sum = ${coreutils:location}/bin/md5sum
basename = ${coreutils:location}/bin/basename
component/faketime/buildout.cfg 0 → 100644
View file @ d007050d
# faketime - report faked system time to programs without having to change the system-wide time
# http://www.code-wizards.com/projects/libfaketime
[buildout]
parts = faketime
[faketime]
recipe = slapos.recipe.cmmi
url = http://www.code-wizards.com/projects/libfaketime/libfaketime-0.9.1.tar.gz
md5sum = ce3f996dfd5826b4ac62f1a7cc36ea27
configure-command = true
make-options =
PREFIX=${buildout:parts-directory}/${:_buildout_section_name_}
make-binary = make -e -C src
make-targets = install
post-install = sed -i -e "16c\FTPL_PATH=${buildout:parts-directory}/${:_buildout_section_name_}/lib/faketime" ${buildout:parts-directory}/${:_buildout_section_name_}/bin/faketime
component/gnutls/buildout.cfg
View file @ d007050d
[buildout]
extends =
../readline/buildout.cfg
../gmp/buildout.cfg
../nettle/buildout.cfg
../ncurses/buildout.cfg
../readline/buildout.cfg
../zlib/buildout.cfg
parts = gnutls
......@@ -22,14 +24,13 @@ environment =
LDFLAGS=-lgpg-error -L${gpg-error:location}/lib -Wl,-rpath=${gpg-error:location}/lib
[gnutls]
# XXX-Cedric : update to latest gnutls
recipe = slapos.recipe.cmmi
url = ftp://ftp.gnutls.org/gcrypt/gnutls/v2.8/gnutls-2.8.6.tar.bz2
md5sum = eb0a6d7d3cb9ac684d971c14f9f6d3ba
url = ftp://ftp.gnutls.org/gcrypt/gnutls/v3.2/gnutls-3.2.0.tar.xz
md5sum = e0cba4ddd923420026ff9739b3bc069a
configure-options =
--with-libgcrypt-prefix=${gcrypt:location}
--disable-static
environment =
CPPFLAGS=-I${zlib:location}/include -I${readline:location}/include -I${ncurses:location}/include -I${ncurses:location}/include/ncursesw -I${gcrypt:location}/include -I${gpg-error:location}/include
LDFLAGS=-lgcrypt -L${readline:location}/lib -Wl,-rpath=${readline:location}/lib -L${ncurses:location}/lib -Wl,-rpath=${ncurses:location}/lib -L${gcrypt:location}/lib -Wl,-rpath=${gcrypt:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${gpg-error:location}/lib -Wl,-rpath=${gpg-error:location}/lib
PKG_CONFIG=${zlib:location}/lib/pkgconfig
CPPFLAGS=-I${zlib:location}/include -I${readline:location}/include -I${ncurses:location}/include -I${ncurses:location}/include/ncursesw -I${gmp:location}/include -I${gcrypt:location}/include -I${gpg-error:location}/include -I${nettle:location}/include
LDFLAGS=-lgcrypt -L${gmp:location}/lib -Wl,-rpath=${gmp:location}/lib -L${readline:location}/lib -Wl,-rpath=${readline:location}/lib -L${ncurses:location}/lib -Wl,-rpath=${ncurses:location}/lib -L${gcrypt:location}/lib -Wl,-rpath=${gcrypt:location}/lib -L${nettle:location}/lib -Wl,-rpath=${nettle:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${gpg-error:location}/lib -Wl,-rpath=${gpg-error:location}/lib
component/groonga/buildout.cfg
View file @ d007050d
......@@ -11,9 +11,9 @@ extends =
[groonga]
recipe = slapos.recipe.cmmi
version = 3.0.5
version = 3.0.9
url = http://packages.groonga.org/source/groonga/groonga-${:version}.tar.gz
md5sum = 2894bbdd2275cb3c62aea14446dc2561
md5sum = 2e9f7090f40876233c1f077fd25c26ee
configure-options =
--disable-static
--disable-glibtest
......
component/gtk-2/buildout.cfg
View file @ d007050d
......@@ -22,12 +22,12 @@ depends =
${libpng:so_version}
configure-options =
--enable-tee=yes
--enable-xlib=no
--enable-xlib=yes
environment =
PATH=${freetype:location}/bin:${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${fontconfig:location}/lib/pkgconfig:${freetype:location}/lib/pkgconfig:${zlib:location}/lib/pkgconfig:${libpng:location}/lib/pkgconfig:${pixman:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libXrender:location}/lib/pkgconfig
CPPFLAGS=-I${libpng:location}/include/ -I${zlib:location}/include -I${libX11:location}/include/ -I${xproto:location}/include -I${kbproto:location}/include -I${libXrender:location}/include -I${render:location}/include
LDFLAGS=-L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${libXrender:location}/lib -Wl,-rpath=${libXrender:location}/lib
LDFLAGS=-L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${libXrender:location}/lib -Wl,-rpath=${libXrender:location}/lib -L${libX11:location}/lib
LD_LIBRARY_PATH=${render:location}/lib:${libX11:location}/lib:${libXrender:location}/lib
[pango]
......
component/libpng/buildout.cfg
View file @ d007050d
......@@ -28,6 +28,6 @@ md5sum = fd85af68f84cbdf549147811006488c1
[libpng]
<= libpng-common
url = http://download.sourceforge.net/libpng/libpng-1.6.2.tar.xz
md5sum = 9d838f6fca9948a9f360a0cc1b516d5f
url = http://download.sourceforge.net/libpng/libpng-1.6.6.tar.xz
md5sum = 3a41dcd58bcac7cc191c2ec80c7fb2ac
so_version = 16
component/make/buildout.cfg 0 → 100644
View file @ d007050d
[buildout]
extends =
../autoconf/buildout.cfg
../automake/buildout.cfg
[make]
# make 3.82 breaks too many things. Stick with 3.81.
# See http://lists.gnu.org/archive/html/make-alpha/2010-07/msg00025.html
# for all incompatible changes.
# Moreover, vanilla 3.81 does some seg faults, so use Debian patched version.
<= make3.81-debian
[make-dfsg_3.81-8.2.diff]
# Debian patch coming from:
# http://ftp.de.debian.org/debian/pool/main/m/make-dfsg/make-dfsg_3.81-8.2.diff.gz
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:_buildout_section_name_}
md5sum = fa77bb989a096fafbe7c78582e9415e3
download-only = true
[make3.81-debian]
recipe = slapos.recipe.cmmi
url = http://ftp.de.debian.org/debian/pool/main/m/make-dfsg/make-dfsg_3.81.orig.tar.gz
md5sum = 7c93b1ab4680eb21c2c13f4f47741e2d
patches =
${make-dfsg_3.81-8.2.diff:location}/make-dfsg_3.81-8.2.diff
patch-options = -p1
component/make/make-dfsg_3.81-8.2.diff 0 → 100644
View file @ d007050d
This source diff could not be displayed because it is too large. You can view the blob instead.
component/mariadb/buildout.cfg
View file @ d007050d
......@@ -25,10 +25,10 @@ download-only = true
[mariadb]
recipe = slapos.recipe.cmmi
version = 5.5.32
version = 5.5.33a
revision = 1
url = http://downloads.askmonty.org/f/mariadb-${:version}/kvm-tarbake-jaunty-x86/mariadb-${:version}.tar.gz/from/http://ftp.osuosl.org/pub/mariadb
md5sum = 565c2dce6a2fb027c9d0ffbae4934135
md5sum = 00449a034b88490f16bd679b800bb850
# compile directory is required to build mysql plugins.
keep-compile-dir = true
patch-options = -p0
......@@ -56,14 +56,14 @@ environment =
CMAKE_PROGRAM_PATH=${cmake:location}/bin
CMAKE_INCLUDE_PATH=${libaio:location}/include:${ncurses:location}/include:${openssl:location}/include:${readline5:location}/include:${zlib:location}/include
CMAKE_LIBRARY_PATH=${libaio:location}/lib:${ncurses:location}/lib:${openssl:location}/lib:${readline5:location}/lib:${zlib:location}/lib
LDFLAGS=-L${libaio:location}/lib
LDFLAGS=-L${libaio:location}/lib -L${zlib:location}/lib
[mroonga-mariadb]
# mroonga - a storage engine for MySQL. It provides fast fulltext search feature to all MySQL users.
# http://mroonga.github.com/
recipe = slapos.recipe.cmmi
url = http://packages.groonga.org/source/mroonga/mroonga-3.05.tar.gz
md5sum = ba4cbd79274d832b9343a0b2fe7d0787
url = http://packages.groonga.org/source/mroonga/mroonga-3.09.tar.gz
md5sum = bdcd1d86185a64520881d33e12b7d7a7
configure-options =
--with-mysql-source=${mariadb:location}__compile__/mariadb-${mariadb:version}
--with-mysql-config=${mariadb:location}/bin/mysql_config
......
component/nano/buildout.cfg 0 → 100644
View file @ d007050d
[buildout]
parts =
nano
extends =
../ncurses/buildout.cfg
[nano]
recipe = slapos.recipe.cmmi
version = 2.2.6
url = http://www.nano-editor.org/dist/v2.2/nano-2.2.6.tar.gz
md5sum = 03233ae480689a008eb98feb1b599807
environment=
CFLAGS=-I${ncurses:location}/include
LDFLAGS=-L${ncurses:location}/lib/ -Wl,-rpath=${ncurses:location}/lib/
\ No newline at end of file
component/nettle/buildout.cfg 0 → 100644
View file @ d007050d
[buildout]
extends =
../gmp/buildout.cfg
../m4/buildout.cfg
[nettle-lib-location.patch]
recipe = hexagonit.recipe.download
download-only = true
filename = ${:_buildout_section_name_}
url = ${:_profile_base_location_}/${:filename}
md5sum = 41dd0ce2a73487929bdc637b75dd62c9
[nettle]
recipe = slapos.recipe.cmmi
url = http://www.lysator.liu.se/~nisse/archive/nettle-2.7.1.tar.gz
md5sum = 003d5147911317931dd453520eb234a5
patches =
${nettle-lib-location.patch:location}/${nettle-lib-location.patch:filename}
configure-option =
--disable-static
--disable-assembler
--disable-openssl
environment =
PATH=${m4:location}/bin:%(PATH)s
CPPFLAGS=-I${gmp:location}/include
LDFLAGS=-L${gmp:location}/lib -Wl,-rpath=${gmp:location}/lib
component/nettle/nettle-lib-location.patch 0 → 100644
View file @ d007050d
--- configure.orig 2013-07-05 15:37:28.000000000 +0200
+++ configure 2013-07-05 15:47:48.000000000 +0200
@@ -4680,52 +4680,6 @@
if test "x$ABI" != xstandard ; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: Compiler uses $ABI-bit ABI. To change, set CC." >&5
$as_echo "$as_me: Compiler uses $ABI-bit ABI. To change, set CC." >&6;}
- if test "$libdir" = '${exec_prefix}/lib' ; then
- # Try setting a better default
- case "$host_cpu:$host_os:$ABI" in
- *:solaris*:32|*:sunos*:32)
- libdir='${exec_prefix}/lib'
- ;;
- *:solaris*:64|*:sunos*:64)
- libdir='${exec_prefix}/lib/64'
- ;;
- # Linux conventions are a mess... According to the Linux File
- # Hierarchy Standard, all architectures except IA64 puts 32-bit
- # libraries in lib, and 64-bit in lib64. Some distributions,
- # e.g., Fedora and Gentoo, adhere to this standard, while at
- # least Debian has decided to put 64-bit libraries in lib and
- # 32-bit libraries in lib32.
-
- # We try to figure out the convention, except if we're cross
- # compiling. We use lib${ABI} if /usr/lib${ABI} exists and
- # appears to not be a symlink to a different name.
- *:linux*:32|*:linux*:64)
- if test "$cross_compiling" = yes ; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Cross compiling for linux. Can't guess if libraries go in lib${ABI} or lib." >&5
-$as_echo "$as_me: WARNING: Cross compiling for linux. Can't guess if libraries go in lib${ABI} or lib." >&2;}; else
- # The dash builtin pwd tries to be "helpful" and remember
- # symlink names. Use -P option, and hope it's portable enough.
- test -d /usr/lib${ABI} \
- && (cd /usr/lib${ABI} && pwd -P | grep >/dev/null "/lib${ABI}"'$') \
- && libdir='${exec_prefix}/'"lib${ABI}"
- fi
- ;;
- # On freebsd, it seems 32-bit libraries are in lib32,
- # and 64-bit in lib. Don't know about "kfreebsd", does
- # it follow the Linux fhs conventions?
- *:freebsd*:32)
- libdir='${exec_prefix}/lib32'
- ;;
- *:freebsd*:64)
- libdir='${exec_prefix}/lib'
- ;;
- *)
- { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Don't know where to install $ABI-bit libraries on this system." >&5
-$as_echo "$as_me: WARNING: Don't know where to install $ABI-bit libraries on this system." >&2;};
- esac
- { $as_echo "$as_me:${as_lineno-$LINENO}: Libraries to be installed in $libdir." >&5
-$as_echo "$as_me: Libraries to be installed in $libdir." >&6;}
- fi
fi
# Select assembler code
component/nginx/buildout.cfg
View file @ d007050d
......@@ -3,11 +3,14 @@ extends =
../pcre/buildout.cfg
../zlib/buildout.cfg
../openssl/buildout.cfg
../coreutils/buildout.cfg
parts = nginx-output
[nginx]
recipe = slapos.recipe.cmmi
url = http://nginx.org/download/nginx-1.2.7.tar.gz
md5sum = d252f5c689a14a668e241c744ccf5f06
url = http://nginx.org/download/nginx-1.5.3.tar.gz
md5sum = 1e735dd6a6ade2b5c20e924b67c3d355
configure-options=
--with-ipv6
--with-http_ssl_module
......@@ -16,6 +19,15 @@ configure-options=
--with-ld-opt="-L ${zlib:location}/lib -L ${openssl:location}/lib -L ${pcre:location}/lib -Wl,-rpath=${pcre:location}/lib -Wl,-rpath=${zlib:location}/lib -Wl,-rpath=${openssl:location}/lib"
--with-cc-opt="-I ${pcre:location}/include -I ${openssl:location}/include -I ${zlib:location}/include"
[nginx-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${coreutils-output:test} -x ${:nginx} -a -f ${:mime}
nginx = ${nginx:location}/sbin/nginx
mime = ${nginx:location}/conf/mime.types
[nginx-unstable]
<= nginx
url = http://nginx.org/download/nginx-1.3.15.tar.gz
......
component/openssl/buildout.cfg
View file @ d007050d
......@@ -8,9 +8,10 @@ extends =
../ca-certificates/buildout.cfg
../zlib/buildout.cfg
../patch/buildout.cfg
../coreutils/buildout.cfg
parts =
openssl
openssl-output
[openssl-nodoc.patch]
# Disable doc generation part in Makefile
......@@ -52,3 +53,11 @@ make-options =
-j1
make-targets =
all install_sw && rm -f ${buildout:parts-directory}/${:_buildout_section_name_}/etc/ssl/certs/* && for i in ${ca-certificates:location}/certs/*/*.crt; do ln -sv $i ${buildout:parts-directory}/${:_buildout_section_name_}/etc/ssl/certs/`${buildout:parts-directory}/${:_buildout_section_name_}/bin/openssl x509 -hash -noout -in $i`.0; done; true
[openssl-output]
# Shared binary location to ease migration
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command = ${coreutils-output:test} -x ${:openssl}
openssl = ${openssl:location}/bin/openssl
component/percona-toolkit/buildout.cfg
View file @ d007050d
......@@ -10,8 +10,8 @@ parts =
recipe = slapos.recipe.cmmi
depends =
${perl:version}
version = 2.1.9
version = 2.2.5
url = http://www.percona.com/redir/downloads/percona-toolkit/${:version}/percona-toolkit-${:version}.tar.gz
md5sum = 94545d0fe6a4893dcad8a3411531107d
md5sum = 66165271fc3ddf8311e5ff3b1a954595
configure-command =
${perl:location}/bin/perl Makefile.PL
component/python-2.7/buildout.cfg
View file @ d007050d
......@@ -10,6 +10,7 @@ extends =
../sqlite3/buildout.cfg
../zlib/buildout.cfg
../file/buildout.cfg
../xz-utils/buildout.cfg
parts =
python2.7
......@@ -25,9 +26,9 @@ python = python2.7
[python2.7]
recipe = slapos.recipe.cmmi
package_version = 2.7.5
package_version_suffix =
md5sum = 6334b666b7ff2038c761d7b27ba699c1
package_version = 2.7.6
package_version_suffix = rc1
md5sum = 74e1f6abe529350ac0d239387ee98616
depends =
${gdbm:version}
......@@ -38,7 +39,7 @@ version = 2.7
executable = ${:prefix}/bin/python${:version}
url =
http://python.org/ftp/python/${:package_version}/Python-${:package_version}${:package_version_suffix}.tar.bz2
http://python.org/ftp/python/${:package_version}/Python-${:package_version}${:package_version_suffix}.tar.xz
configure-options =
--enable-ipv6
--enable-unicode=ucs4
......@@ -51,5 +52,6 @@ make-targets = make profile-opt && make install
# the entry "-Wl,-rpath=${file:location}/lib" below is needed by python-magic,
# which would otherwise load the system libmagic.so with ctypes
environment =
PATH=${xz-utils:location}/bin:%(PATH)s
CPPFLAGS=-I${zlib:location}/include -I${readline:location}/include -I${libexpat:location}/include -I${ncurses:location}/include -I${ncurses:location}/include/ncursesw -I${bzip2:location}/include -I${gdbm:location}/include -I${openssl:location}/include -I${sqlite3:location}/include -I${gettext:location}/include
LDFLAGS=-L${zlib:location}/lib -L${readline:location}/lib -L${libexpat:location}/lib -L${ncurses:location}/lib -L${bzip2:location}/lib -L${gdbm:location}/lib -L${openssl:location}/lib -L${sqlite3:location}/lib -Wl,-rpath=${zlib:location}/lib -Wl,-rpath=${readline:location}/lib -Wl,-rpath=${libexpat:location}/lib -Wl,-rpath=${ncurses:location}/lib -Wl,-rpath=${bzip2:location}/lib -Wl,-rpath=${gdbm:location}/lib -Wl,-rpath=${openssl:location}/lib -Wl,-rpath=${sqlite3:location}/lib -L${gettext:location}/lib -Wl,-rpath=${gettext:location}/lib -Wl,-rpath=${file:location}/lib
component/python-openssl/buildout.cfg
View file @ d007050d
......@@ -8,6 +8,8 @@ parts =
[python-openssl]
recipe = zc.recipe.egg:custom
egg = pyOpenSSL
include-dirs =
${openssl:location}/include/
library-dirs =
${openssl:location}/lib/
rpath =
......
component/qemu-kvm/buildout.cfg
View file @ d007050d
[buildout]
extends =
extends =
../../component/gnutls/buildout.cfg
../../component/libpng/buildout.cfg
../../component/libuuid/buildout.cfg
../../component/pkgconfig/buildout.cfg
../../component/xorg/buildout.cfg
../../component/zlib/buildout.cfg
[kvm]
# Backward compatibility
<= qemu-kvm
# XXX Change all reference to kvm section to qemu section, then
# use qemu as main name section.
[qemu]
<= kvm
[qemu-kvm]
[kvm]
recipe = slapos.recipe.cmmi
# qemu-kvm and qemu are now the same since 1.3.
url = http://wiki.qemu-project.org/download/qemu-1.4.1.tar.bz2
md5sum = eb2d696956324722b5ecfa46e41f9a75
url = http://wiki.qemu-project.org/download/qemu-1.6.1.tar.bz2
md5sum = 3a897d722457c5a895cd6ac79a28fda0
depends =
${libpng:so_version}
configure-options =
--target-list=""
--target-list=x86_64-softmmu
--enable-system
--with-system-pixman
--disable-sdl
--disable-xen
--enable-vnc-tls
......@@ -36,4 +40,26 @@ configure-options =
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${gnutls:location}/lib/pkgconfig:${glib:location}/lib/pkgconfig:${pixman:location}/lib/pkgconfig
LDFLAGS=-L${pixman:location}/lib -Wl,-rpath=${pixman:location}/lib
# The following is only available in buildout2, which we don't use yet.
[kvm-bits64]
configure-options =
--target-list=x86_64-softmmu
${kvm:configure-options}
[kvm-bits32]
configure-options =
--target-list=i386-softmmu
${kvm:configure-options}
[debian-amd64-netinst.iso]
# Download the installer of Debian 7 (Wheezy)
recipe = hexagonit.recipe.download
url = http://cdimage.debian.org/debian-cd/7.2.0/amd64/iso-cd/debian-7.2.0-amd64-netinst.iso
filename = ${:_buildout_section_name_}
md5sum = b86774fe4de88be6378ba3d71b8029bd
download-only = true
mode = 0644
location = ${buildout:parts-directory}/${:_buildout_section_name_}
component/slapos/buildout.cfg
View file @ d007050d
......@@ -17,6 +17,7 @@ extends =
../pkgconfig/buildout.cfg
../popt/buildout.cfg
../python-2.7/buildout.cfg
../python-openssl/buildout.cfg
../readline/buildout.cfg
../sqlite3/buildout.cfg
../swig/buildout.cfg
......@@ -90,10 +91,14 @@ output = ${buildout:directory}/environment.sh
[lxml-python]
python = python2.7
[python-openssl]
python = python2.7
[slapos]
recipe = z3c.recipe.scripts
python = python2.7
eggs =
${python-openssl:egg}
slapos.libnetworkcache
zc.buildout
${lxml-python:egg}
......@@ -131,46 +136,64 @@ scripts = py
[versions]
# Use our own buildout version
zc.buildout = 1.6.0-dev-SlapOS-010
zc.buildout = 1.6.0-dev-SlapOS-012
# Force to use zc.recipe.egg 1.x
zc.recipe.egg = 1.3.2
# Use own version of h.r.download to be able to open archives not supported by python2.x: .xz
hexagonit.recipe.download = 1.6nxd002
hexagonit.recipe.download = 1.7nxd002
Jinja2 = 2.7
Jinja2 = 2.7.1
MarkupSafe = 0.18
Werkzeug = 0.8.3
Pygments = 1.6
Werkzeug = 0.9.4
buildout-versions = 1.7
cmd2 = 0.6.7
collective.recipe.template = 1.10
lxml = 3.1.2
itsdangerous = 0.23
lxml = 3.2.3
meld3 = 0.6.10
netaddr = 0.7.10
prettytable = 0.7.2
pyOpenSSL = 0.13.1
pyparsing = 2.0.1
setuptools = 1.1.6
slapos.core = 1.0.0rc6
slapos.libnetworkcache = 0.13.4
slapos.recipe.cmmi = 0.1.1
xml-marshaller = 0.9.7
z3c.recipe.scripts = 1.0.1
# Required by:
# slapos.core==0.35.2-dev
Flask = 0.9
# slapos.core==1.0.0rc6
Flask = 0.10.1
# Required by:
# slapos.core==0.35.2-dev
# slapos.core==1.0.0rc6
bpython = 0.12
# Required by:
# slapos.core==1.0.0rc6
cliff = 1.4.5
# Required by:
# slapos.core==1.0.0rc6
ipython = 1.1.0
# Required by:
# slapos.core==1.0.0rc6
netifaces = 0.8
# Required by:
# slapos.core==0.35.2-dev
# slapos.libnetworkcache==0.13.3
# supervisor==3.0b1
# zc.buildout==1.6.0-dev-SlapOS-010
# zope.interface==4.0.5
setuptools = 0.6c12dev-r88846
# slapos.core==1.0.0rc6
requests = 2.0.0
# Required by:
# slapos.core==0.35.2-dev
supervisor = 3.0b2
# slapos.core==1.0.0rc6
supervisor = 3.0
# Required by:
# slapos.core==0.35.2-dev
# slapos.core==1.0.0rc6
zope.interface = 4.0.5
component/slapos/testing.cfg 0 → 100644
View file @ d007050d
# This file is used to install testing, not-stable-yet, version of SlapOS Node
[buildout]
extends =
buildout.cfg
# Add hosting location of testing version of slapos.core
find-links +=
http://www.nexedi.org/static/packages/source/slapos.core-testing/
[versions]
slapos.core =
component/sqlite3/buildout.cfg
View file @ d007050d
......@@ -5,8 +5,8 @@ parts =
[sqlite3]
recipe = slapos.recipe.cmmi
url = http://www.sqlite.org/2013/sqlite-autoconf-3071700.tar.gz
md5sum = 18c285053e9562b848209cb0ee16d4ab
url = http://www.sqlite.org/2013/sqlite-autoconf-3080100.tar.gz
md5sum = 8b5a0a02dfcb0c7daf90856a5cfd485a
configure-options =
--disable-static
--enable-readline
......
component/squid/buildout.cfg 0 → 100644
View file @ d007050d
# Squid: Optimising Web Delivery
# http://squid-cache.org
[buildout]
parts =
squid
extends =
../pkgconfig/buildout.cfg
[squid]
recipe = hexagonit.recipe.cmmi
url = http://www.squid-cache.org/Versions/v3/3.2/squid-3.2.1.tar.gz
md5sum = 3fb81acc6b70a432e3f0d8a0491056dc
configure-options =
--disable-dependency-tracking
--disable-translation
--disable-htcp
--disable-snmp
--disable-loadable-modules
--disable-icmp
--disable-esi
--disable-icap-client
--disable-wccp
--disable-wccpv2
--disable-eui
--enable-http-violations
--disable-ipfw-transparent
--disable-ipf-transparent
--disable-pf-transparent
--disable-linux-netfilter
--enable-follow-x-forwarded-for
--disable-auth
--disable-url-rewrite-helpers
--disable-auto-locale
--disable-kerberos
--enable-x-accelerator-vary
--disable-external-acl-helpers
--disable-auth-ntlm
--with-krb5-config=no
Environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
component/texinfo/buildout.cfg 0 → 100644
View file @ d007050d
[buildout]
extends =
../ncurses/buildout.cfg
[texinfo]
# Most other components are not happy with texinfo 5, because it treats some
# used-to-be-warnings as errors.
<= texinfo4
[texinfo4]
recipe = slapos.recipe.cmmi
url = http://ftp.gnu.org/gnu/texinfo/texinfo-4.13.tar.gz
md5sum = 71ba711519209b5fb583fed2b3d86fcb
configure-options =
--disable-static
environment =
CFLAGS=-I${ncurses:location}/include
LDFLAGS=-L${ncurses:location}/lib -Wl,-rpath=${ncurses:location}/lib
component/xorg/buildout.cfg
View file @ d007050d
......@@ -28,13 +28,13 @@ ACLOCAL=${xorg-util-macros:location}/share/aclocal
[xorg-util-macros]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/util/util-macros-1.11.0.tar.bz2
md5sum = 22d5cdff672450cb6902e0d68c200dcb
url = http://www.x.org/releases/X11R7.7/src/everything/util-macros-1.17.tar.bz2
md5sum = 4f41667e1bf4938bb2b24fa09d517f77
[xproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/proto/xproto-7.0.20.tar.bz2
md5sum = 65633168e5315c19defb4652cd3d83c1
url = http://www.x.org/releases/X11R7.7/src/everything/xproto-7.0.23.tar.bz2
md5sum = d4d241a4849167e4e694fe73371c328c
configure-options =
--disable-specs
--without-xmlto
......@@ -45,8 +45,8 @@ environment =
[xextproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/proto/xextproto-7.1.2.tar.bz2
md5sum = 263ae968b223c23b2986603d84e5c30e
url = http://www.x.org/releases/X11R7.7/src/everything/xextproto-7.2.1.tar.bz2
md5sum = eaac343af094e6b608cf15cfba0f77c5
configure-options =
--disable-specs
--without-xmlto
......@@ -64,8 +64,8 @@ md5sum = 37d82a3b6009113023599632117a6855
[xtrans]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/xtrans-1.2.6.tar.bz2
md5sum = c66f9ffd2da4fb012220c6c40ebc7609
url = http://www.x.org/releases/X11R7.7/src/everything/xtrans-1.2.7.tar.bz2
md5sum = 84c66908cf003ad8c272b0eecbdbaee3
patches =
${xtrans-tmp-env-patch:location}/${xtrans-tmp-env-patch:filename}
patches_md5sum = ${xtrans-tmp-env-patch:md5sum}
......@@ -80,8 +80,8 @@ environment =
[libXau]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libXau-1.0.6.tar.bz2
md5sum = 4a2cbd83727682f9ee1c1e719bac6adb
url = http://www.x.org/releases/X11R7.7/src/everything/libXau-1.0.7.tar.bz2
md5sum = 2d241521df40d27034413436d1a1465c
configure-options =
--disable-static
environment =
......@@ -90,15 +90,15 @@ environment =
[xcbproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/xcb/xcb-proto-1.6.tar.bz2
md5sum = 04313e1d914b44d0e457f6c494fc178b
url = http://xcb.freedesktop.org/dist/xcb-proto-1.8.tar.bz2
md5sum = a5de3432cc6e43cc6a27f241dbb991b1
environment =
PATH=${libxml2:location}/bin:%(PATH)s
PYTHON=${buildout:executable}
[xorg-libpthread-stubs]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/xcb/libpthread-stubs-0.3.tar.bz2
url = http://xcb.freedesktop.org/dist/libpthread-stubs-0.3.tar.bz2
md5sum = e8fa31b42e13f87e8f5a7a2b731db7ee
[libxcb-tmp-env-patch]
......@@ -110,8 +110,8 @@ md5sum = 61f39878120ba434a169e24cae2af862
[libxcb]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/xcb/libxcb-1.7.tar.bz2
md5sum = 925699df361b99491165ebc12068056b
url = http://xcb.freedesktop.org/dist/libxcb-1.9.1.tar.bz2
md5sum = ed632cb0dc31b6fbd7ea5c0f931cf5a4
patches =
${libxcb-tmp-env-patch:location}/${libxcb-tmp-env-patch:filename}
patches_md5sum = ${libxcb-tmp-env-patch:md5sum}
......@@ -130,8 +130,8 @@ environment =
[libXext]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libXext-1.2.0.tar.bz2
md5sum = 9bb236ff0193e9fc1c1fb504dd840331
url = http://www.x.org/releases/X11R7.7/src/everything/libXext-1.3.1.tar.bz2
md5sum = 71251a22bc47068d60a95f50ed2ec3cf
environment =
PKG_CONFIG_PATH=${kbproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig
LD_LIBRARY_PATH=${libX11:location}/lib
......@@ -144,33 +144,21 @@ configure-options =
--without-xmlto
--without-fop
XEXT_LIBS=-lX11
patches =
${libXext-patch-link-error:location}/${libXext-patch-link-error:filename}
patch-options = -p1
[libXext-patch-link-error]
# Fixes libXext.la link error
# http://cgit.freedesktop.org/xorg/lib/libXext/patch/?id=700c7896b832d6e4fb0185f0d5382b01f94e7141
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/${:filename}
download-only = true
filename = 700c7896b832d6e4fb0185f0d5382b01f94e7141.patch
md5sum = 52635ef694ee6f1acb642a77ee8eb010
[inputproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/inputproto-2.0.1.tar.bz2
md5sum = da9bf9e5d174163f597d2d72757d9038
url = http://www.x.org/releases/X11R7.7/src/everything/inputproto-2.2.tar.bz2
md5sum = 13c8aedaf98a92e282b7e456c0a5bed9
[kbproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/kbproto-1.0.5.tar.bz2
md5sum = e7edb59a3f54af15f749e8f3e314ee62
url = http://www.x.org/releases/X11R7.7/src/everything/kbproto-1.0.6.tar.bz2
md5sum = 677ea8523eec6caca86121ad2dca0b71
[libX11]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libX11-1.4.0.tar.bz2
md5sum = b63d9f7493a61df51d0c0be04ac435e4
url = http://www.x.org/releases/X11R7.7/src/everything/libX11-1.5.0.tar.bz2
md5sum = 78b4b3bab4acbdf0abcfca30a8c70cc6
configure-options =
--disable-static
--enable-loadable-i18n
......@@ -190,8 +178,8 @@ environment =
[libXdmcp]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libXdmcp-1.1.0.tar.bz2
md5sum = 762b6bbaff7b7d0831ddb4f072f939a5
url = http://www.x.org/releases/X11R7.7/src/everything/libXdmcp-1.1.1.tar.bz2
md5sum = b94af6cef211cf3ee256f7e81f70fcd9
environment =
PKG_CONFIG_PATH=${xorg-util-macros:location}/share/pkgconfig:${xproto:location}/lib/pkgconfig
PATH=${pkgconfig:location}/bin:%(PATH)s
......@@ -202,103 +190,92 @@ configure-options =
[fixesproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/fixesproto-4.1.2.tar.bz2
md5sum = bdb58ecc313b509247036d5c11fa99df
url = http://www.x.org/releases/X11R7.7/src/everything/fixesproto-5.0.tar.bz2
md5sum = e7431ab84d37b2678af71e29355e101d
[bigreqsproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/bigreqsproto-1.1.1.tar.bz2
md5sum = 6f6c24436c2b3ab235eb14a85b9aaacf
url = http://www.x.org/releases/X11R7.7/src/everything/bigreqsproto-1.1.2.tar.bz2
md5sum = 1a05fb01fa1d5198894c931cf925c025
[xcmiscproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/xcmiscproto-1.2.1.tar.bz2
md5sum = cd7372cd827bfd7ca7e9238f2ce274b1
url = http://www.x.org/releases/X11R7.7/src/everything/xcmiscproto-1.2.2.tar.bz2
md5sum = 5f4847c78e41b801982c8a5e06365b24
[damageproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/damageproto-1.2.1.tar.bz2
url = http://www.x.org/releases/X11R7.7/src/everything/damageproto-1.2.1.tar.bz2
md5sum = 998e5904764b82642cc63d97b4ba9e95
[randrproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/randrproto-1.3.2.tar.bz2
url = http://www.x.org/releases/X11R7.7/src/everything/randrproto-1.3.2.tar.bz2
md5sum = 597491c0d8055e2a66f11350c4985775
[renderproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/renderproto-0.11.1.tar.bz2
url = http://www.x.org/releases/X11R7.7/src/everything/renderproto-0.11.1.tar.bz2
md5sum = a914ccc1de66ddeb4b611c6b0686e274
[videoproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/videoproto-2.3.1.tar.bz2
url = http://www.x.org/releases/X11R7.7/src/everything/videoproto-2.3.1.tar.bz2
md5sum = c3b348c6e2031b72b11ae63fc7f805c2
[fontsproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/fontsproto-2.1.1.tar.bz2
md5sum = 37102ffcaa73f77d700acd6f7a25d8f0
url = http://www.x.org/releases/X11R7.7/src/everything/fontsproto-2.1.2.tar.bz2
md5sum = c5f4f1fb4ba7766eedbc9489e81f3be2
[recordproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/recordproto-1.14.1.tar.bz2
md5sum = 24541a30b399213def35f48efd926c63
url = http://www.x.org/releases/X11R7.7/src/everything/recordproto-1.14.2.tar.bz2
md5sum = 1b4e5dede5ea51906f1530ca1e21d216
[resourceproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/resourceproto-1.1.1.tar.bz2
md5sum = 8ff0525ae7502b48597b78d00bc22284
url = http://www.x.org/releases/X11R7.7/src/everything/resourceproto-1.2.0.tar.bz2
md5sum = cfdb57dae221b71b2703f8e2980eaaf4
[xineramaproto]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/xineramaproto-1.2.tar.bz2
md5sum = a8aadcb281b9c11a91303e24cdea45f5
url = http://www.x.org/releases/X11R7.7/src/everything/xineramaproto-1.2.1.tar.bz2
md5sum = 9959fe0bfb22a0e7260433b8d199590a
[pixman]
recipe = slapos.recipe.cmmi
url = http://xorg.freedesktop.org/archive/individual/lib/pixman-0.23.2.tar.bz2
url = http://xorg.freedesktop.org/archive/individual/lib/pixman-0.23.2.tar.bz2
md5sum = 2e2805f5ca02edeb15a7862779670069
[libfontenc]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/libfontenc-1.1.0.tar.bz2
md5sum = 11d3c292f05a90f6f67840a9e9c3d9b8
url = http://www.x.org/releases/X11R7.7/src/everything/libfontenc-1.1.1.tar.bz2
md5sum = a2a861f142c3b4367f14fc14239fc1f7
environment =
PKG_CONFIG_PATH=${xproto:location}/lib/pkgconfig
LD_LIBRARY_PATH=${xproto:location}/lib/
LD_RUN_PATH=${xproto:location}/lib/
CPPFLAGS=-I${xproto:location}/include/ -I${zlib:location}/include
LDFLAGS=-L${xproto:location}/lib -Wl,-rpath=${xproto:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib
PATH=${pkgconfig:location}/bin:%(PATH)s
[libXfont]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/libXfont-1.4.3.tar.bz2
md5sum = 6fb689cfe13d8d9460f4abb5bd88588d
url = http://www.x.org/releases/X11R7.7/src/everything/libXfont-1.4.5.tar.bz2
md5sum = 6851da5dae0a6cf5f7c9b9e2b05dd3b4
environment =
PKG_CONFIG_PATH=${fontsproto:location}/lib/pkgconfig:${libfontenc:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${xtrans:location}/share/pkgconfig:${freetype:location}/lib/pkgconfig
LD_LIBRARY_PATH=${fontsproto:location}/lib:${libfontenc:location}/lib:${xtrans:location}/include/:${xproto:location}/lib/:${freetype:location}/lib
LD_RUN_PATH=${fontsproto:location}/lib:${libfontenc:location}/lib:${xproto:location}/lib/:${xtrans:location}/include/:${freetype:location}/lib
LDFLAGS=-L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${bzip2:location}/lib -Wl,-rpath=${bzip2:location}/lib -L${libfontenc:location}/lib -Wl,-rpath=${libfontenc:location}/lib
CPPFLAGS=-I${zlib:location}/include -I${bzip2:location}/include -I${libfontenc:location}/include
PATH=${pkgconfig:location}/bin:%(PATH)s
[libxkbfile]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/everything/libxkbfile-1.0.7.tar.bz2
md5sum = 59b4fe0bdf8d9b05e45b59e8fe9e7516
url = http://www.x.org/releases/X11R7.7/src/everything/libxkbfile-1.0.8.tar.bz2
md5sum = 19e6533ae64abba0773816a23f2b9507
environment =
PKG_CONFIG_PATH=${kbproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig
LD_LIBRARY_PATH=${kbproto:location}/lib:${libX11:location}/lib:${xproto:location}/lib/
LD_RUN_PATH=${kbproto:location}/lib:${libX11:location}/lib:${xproto:location}/lib/
CPPFLAGS=-I${xproto:location}/include/ -I${libX11:location}/include/ -I${kbproto:location}/include
PKG_CONFIG_PATH=${kbproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig
PATH=${pkgconfig:location}/bin:%(PATH)s
[xkeyboard-config]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/individual/data/xkeyboard-config/xkeyboard-config-2.1.tar.bz2
md5sum = 57ecc78bacd8ecc398d8ed6588f4d04b
url = http://www.x.org/releases/X11R7.7/src/everything/xkeyboard-config-2.6.tar.bz2
md5sum = e7e31da9adb56ba52b5c18226b8cbef3
depends =
${perl:version}
${perl-XML-Parser:location}
......@@ -313,17 +290,14 @@ environment =
[xkbcomp]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/app/xkbcomp-1.2.0.tar.bz2
md5sum = 0f55995cd8da9b2d88553e1a2e17cd0a
url = http://www.x.org/releases/X11R7.7/src/everything/xkbcomp-1.2.4.tar.bz2
md5sum = a0fc1ac3fc4fe479ade09674347c5aa0
# Hardcoded location for xkeyboard-config, needed during compile time
xkeyboard-config-location = ${libxkbfile:location}/../xkeyboard-config/share/X11/xkb
configure-options =
--with-xkb-config-root=${:xkeyboard-config-location}
environment =
PKG_CONFIG_PATH=${libxkbfile:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig
LD_LIBRARY_PATH=${libxkbfile:location}/lib/:${libX11:location}/lib
LD_RUN_PATH=${libxkbfile:location}/lib/:${libX11:location}/lib
CPPFLAGS=-I${libxkbfile:location}/include/ -I${libX11:location}/include -I${xproto:location}/include -I${kbproto:location}/include
PATH=${pkgconfig:location}/bin:%(PATH)s
[render]
......@@ -343,8 +317,8 @@ environment =
[libXinerama]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libXinerama-1.1.1.tar.bz2
md5sum = ecd4839ad01f6f637c6fb5327207f89b
url = http://www.x.org/releases/X11R7.7/src/everything/libXinerama-1.1.2.tar.bz2
md5sum = cb45d6672c93a608f003b6404f1dd462
environment =
PKG_CONFIG_PATH=${kbproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libXext:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${xineramaproto:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig
PATH=${pkgconfig:location}/bin:%(PATH)s
......@@ -353,8 +327,8 @@ configure-options =
[libICE]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libICE-1.0.7.tar.bz2
md5sum = bb72a732b15e9dc25c3036559387eed5
url = http://www.x.org/releases/X11R7.7/src/everything/libICE-1.0.8.tar.bz2
md5sum = 471b5ca9f5562ac0d6eac7a0bf650738
configure-options =
--disable-static
--without-xmlto
......@@ -365,8 +339,8 @@ environment =
[libSM]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libSM-1.2.0.tar.bz2
md5sum = e78c447bf1790552b644eca81b542742
url = http://www.x.org/releases/X11R7.7/src/everything/libSM-1.2.1.tar.bz2
md5sum = 766de9d1e1ecf8bf74cebe2111d8e2bd
configure-options =
--disable-static
--without-xmlto
......@@ -378,18 +352,17 @@ environment =
[libXt]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libXt-1.0.9.tar.bz2
md5sum = 8a414f8f2327aaa616ca2dcac1f5d8c3
url = http://www.x.org/releases/X11R7.7/src/everything/libXt-1.1.3.tar.bz2
md5sum = a6f137ae100e74ebe3b71eb4a38c40b3
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${libSM:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libICE:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig
CPPFLAGS=-I${libSM:location}/include -I${kbproto:location}/include -I${libICE:location}/include -I${xproto:location}/include -I${libX11:location}/include
PKG_CONFIG_PATH=${libSM:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libICE:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig
[dri2proto]
recipe = slapos.recipe.cmmi
url = http://xorg.freedesktop.org/releases/individual/proto/dri2proto-2.6.tar.gz
md5sum = 873142af5db695537cfe05e01d13541f
configure-options =
url = http://www.x.org/releases/X11R7.7/src/everything/dri2proto-2.6.tar.bz2
md5sum = 2eb74959684f47c862081099059a11ab
configure-options =
--disable-static
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
......@@ -398,9 +371,9 @@ environment =
[pciaccess]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libpciaccess-0.12.0.tar.bz2
md5sum = 285e07976274572e1f1e68edee09b70a
configure-options =
url = http://www.x.org/releases/X11R7.7/src/everything/libpciaccess-0.13.1.tar.bz2
md5sum = 399a419ac6a54f0fc07c69c9bdf452dc
configure-options =
--disable-static
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
......@@ -409,7 +382,7 @@ environment =
recipe = slapos.recipe.cmmi
url = http://xorg.freedesktop.org/releases/individual/util/makedepend-1.0.3.tar.bz2
md5sum = 4e6cb97bbecfbb34f3f644a75e513093
configure-options =
configure-options =
--disable-static
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
......@@ -419,7 +392,7 @@ environment =
recipe = slapos.recipe.cmmi
url = http://xorg.freedesktop.org/releases/individual/proto/glproto-1.4.14.tar.bz2
md5sum = f48257daf0017f7a7667e5bf48ca3578
configure-options =
configure-options =
--disable-static
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
......@@ -429,23 +402,21 @@ environment =
recipe = slapos.recipe.cmmi
url = http://xorg.freedesktop.org/releases/individual/lib/libXfixes-4.0.5.tar.bz2
md5sum = 1b4b8386bd5d1751b2c7177223ad4629
configure-options =
configure-options =
--disable-static
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${xproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${xorg-util-macros:location}/share/pkgconfig
CPPFLAGS=-I${libX11:location}/include -I${xproto:location}/include -I${fixesproto:location}/include -I${xextproto:location}/include
PKG_CONFIG_PATH=${xproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${xorg-util-macros:location}/share/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig
[xdamage]
recipe = slapos.recipe.cmmi
url = http://xorg.freedesktop.org/releases/individual/lib/libXdamage-1.1.3.tar.bz2
url = http://www.x.org/releases/X11R7.7/src/everything/libXdamage-1.1.3.tar.bz2
md5sum = 44774e1a065158b52f1a0da5100cebec
configure-options =
configure-options =
--disable-static
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${damageproto:location}/lib/pkgconfig:${xfixes:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig
CPPFLAGS=-I${libX11:location}/include -I${xproto:location}/include -I${fixesproto:location}/include -I${xextproto:location}/include -I${damageproto:location}/include -I${xfixes:location}/include
PKG_CONFIG_PATH=${damageproto:location}/lib/pkgconfig:${xfixes:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig
[xorg-tmp-env-patch]
recipe = hexagonit.recipe.download
......@@ -456,37 +427,35 @@ md5sum = 8b60ab8121f0564a681fc00d03101696
[libxmu]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libXmu-1.1.0.tar.gz
md5sum = 15e1cb9aaf97eae169bfe6944e0a541e
url = http://www.x.org/releases/X11R7.7/src/everything/libXmu-1.1.1.tar.bz2
md5sum = a4efff8de85bd45dd3da124285d10c00
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${xextproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libXext:location}/lib/pkgconfig:${libXt:location}/lib/pkgconfig
CPPFLAGS=-I${libX11:location}/include -I${xextproto:location}/include
PKG_CONFIG_PATH=${xextproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libXext:location}/lib/pkgconfig:${libXt:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libICE:location}/lib/pkgconfig:${libSM:location}/lib/pkgconfig
[libXcursor]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/lib/libXcursor-1.1.11.tar.gz
md5sum = 55558a41d0bcf9210636858844fa4d2f
url = http://www.x.org/releases/X11R7.7/src/everything/libXcursor-1.1.13.tar.bz2
md5sum = 52efa81b7f26c8eda13510a2fba98eea
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${render:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xfixes:location}/lib/pkgconfig:${libXrender:location}/lib/pkgconfig
CPPFLAGS=-I${render:location}/include -I${xproto:location}/include -I${fixesproto:location}/include -I${libX11:location}/include -I${xfixes:location}/include -I${libXrender:location}/include
PKG_CONFIG_PATH=${render:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${fixesproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${xfixes:location}/lib/pkgconfig:${libXrender:location}/lib/pkgconfig:${renderproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig
[xwd]
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/app/xwd-1.0.4.tar.gz
md5sum = f8da771069756210833524b5afa1832c
url = http://www.x.org/releases/X11R7.7/src/everything/xwd-1.0.5.tar.bz2
md5sum = 2113126f9ac9c02bb8547c112c5d037e
environment =
PATH=${pkgconfig:location}/bin:%(PATH)s
PKG_CONFIG_PATH=${xorg-util-macros:location}/share/pkgconfig:${xproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libxkbfile:location}/lib/pkgconfig
CPPFLAGS=-I${xproto:location}/include -I${libX11:location}/include -I${libxkbfile:location}/include# -I${xfixes:location}/include -I${libXrender:location}/include
LDFLAGS=-L${xproto:location}/lib -Wl,-rpath=${xproto:location}/lib -L${libX11:location}/lib -Wl,-rpath=${libX11:location}/lib -L${libxkbfile:location}/lib -Wl,-rpath=${libxkbfile:location}/lib -lX11
PKG_CONFIG_PATH=${xorg-util-macros:location}/share/pkgconfig:${xproto:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libxkbfile:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig
CPPFLAGS=-I${libXt:location}/include
LDFLAGS=-L${libXt:location}/lib -Wl,-rpath=${libXt:location}/lib
[xserver]
# Adds Xvfb functionnality
recipe = slapos.recipe.cmmi
url = http://www.x.org/releases/X11R7.6/src/xserver/xorg-server-1.9.3.tar.bz2
md5sum = 5bef6839a76d029204ab31aa2fcb5201
url = http://www.x.org/releases/X11R7.7/src/everything/xorg-server-1.12.2.tar.bz2
md5sum = 791f0323b886abb7954de7f042bb7dc6
patches =
${xorg-tmp-env-patch:location}/${xorg-tmp-env-patch:filename}
patches_md5sum = ${xorg-tmp-env-patch:md5sum}
......@@ -506,7 +475,7 @@ configure-options =
--with-xkb-path=${xkeyboard-config:location}/share/X11/xkb
--with-sha1=libgcrypt
environment =
PKG_CONFIG_PATH=${pixman:location}/lib/pkgconfig:${openssl:location}/lib/pkgconfig:${xorg-util-macros:location}/share/pkgconfig:${fixesproto:location}/lib/pkgconfig:${damageproto:location}/lib/pkgconfig:${xcmiscproto:location}/lib/pkgconfig:${xtrans:location}/share/pkgconfig:${bigreqsproto:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${randrproto:location}/lib/pkgconfig:${renderproto:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${inputproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${fontsproto:location}/lib/pkgconfig:${videoproto:location}/lib/pkgconfig:${recordproto:location}/lib/pkgconfig:${resourceproto:location}/lib/pkgconfig:${xineramaproto:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libxkbfile:location}/lib/pkgconfig:${libXfont:location}/lib/pkgconfig:${gpg-error:location}/lib/pkgconfig
CPPFLAGS=-I${xproto:location}/include -I${pixman:location}/include/pixman-1 -I${xextproto:location}/include -I${inputproto:location}/include -I${fontsproto:location}/include -I${kbproto:location}/include -I${xineramaproto:location}/include -I${renderproto:location}/include -I${bigreqsproto:location}/include -I${videoproto:location}/include -I${resourceproto:location}/include -I${xcmiscproto:location}/include -I${xtrans:location}/include -I${libXau:location}/include -I${randrproto:location}/include -I${libxkbfile:location}/include -I${recordproto:location}/include -I${fixesproto:location}/include -I${damageproto:location}/include -I${bzip2:location}/include -I${gcrypt:location}/include -I${zlib:location}/include -I${libfontenc:location}/include -I${gpg-error:location}/include
PKG_CONFIG_PATH=${pixman:location}/lib/pkgconfig:${openssl:location}/lib/pkgconfig:${xorg-util-macros:location}/share/pkgconfig:${fixesproto:location}/lib/pkgconfig:${damageproto:location}/lib/pkgconfig:${xcmiscproto:location}/lib/pkgconfig:${xtrans:location}/share/pkgconfig:${bigreqsproto:location}/lib/pkgconfig:${xproto:location}/lib/pkgconfig:${randrproto:location}/lib/pkgconfig:${renderproto:location}/lib/pkgconfig:${xextproto:location}/lib/pkgconfig:${inputproto:location}/lib/pkgconfig:${kbproto:location}/lib/pkgconfig:${fontsproto:location}/lib/pkgconfig:${videoproto:location}/lib/pkgconfig:${recordproto:location}/lib/pkgconfig:${resourceproto:location}/lib/pkgconfig:${xineramaproto:location}/lib/pkgconfig:${libXau:location}/lib/pkgconfig:${libxkbfile:location}/lib/pkgconfig:${libXfont:location}/lib/pkgconfig:${libX11:location}/lib/pkgconfig:${libxcb:location}/lib/pkgconfig:${xorg-libpthread-stubs:location}/lib/pkgconfig:${libfontenc:location}/lib/pkgconfig:${freetype:location}/lib/pkgconfig
CPPFLAGS=-I${bzip2:location}/include -I${gcrypt:location}/include -I${gpg-error:location}/include -I${zlib:location}/include
PATH=${pkgconfig:location}/bin:%(PATH)s
LDFLAGS=-L${gcrypt:location}/lib -L${bzip2:location}/lib -Wl,-rpath=${gcrypt:location}/lib -Wl,-rpath=${openssl:location}/lib -Wl,-rpath=${bzip2:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib -L${libfontenc:location}/lib -Wl,-rpath=${libfontenc:location}/lib -L${libXau:location}/lib -Wl,-rpath=${libXau:location}/lib -L${gpg-error:location}/lib -Wl,-rpath=${gpg-error:location}/lib
LDFLAGS=-L${bzip2:location}/lib -Wl,-rpath=${bzip2:location}/lib -L${gcrypt:location}/lib -Wl,-rpath=${gcrypt:location}/lib -L${gpg-error:location}/lib -Wl,-rpath=${gpg-error:location}/lib -L${zlib:location}/lib -Wl,-rpath=${zlib:location}/lib
component/zeromq/buildout.cfg 0 → 100644
View file @ d007050d
[buildout]
extends =
../libtool/buildout.cfg
../libuuid/buildout.cfg
[zeromq]
<= zeromq3
[zeromq3]
recipe = slapos.recipe.cmmi
url = http://download.zeromq.org/zeromq-3.2.3.tar.gz
md5sum = 1abf8246363249baf5931a065ee38203
configure-options = --without-documentation
environment =
PATH=${libtool:location}/bin:%(PATH)s
LDFLAGS=-L${libtool:location}/lib -Wl,-rpath -Wl,${libtool:location}/lib -L${libuuid:location}/lib -Wl,-rpath -Wl,${libuuid:location}/lib
[zeromq2]
recipe = slapos.recipe.cmmi
url = http://download.zeromq.org/zeromq-2.2.0.tar.gz
md5sum = 1b11aae09b19d18276d0717b2ea288f6
configure-options =
--without-documentation
environment =
PATH=${libtool:location}/bin:%(PATH)s
CXXFLAGS=-I${libuuid:location}/include
LDFLAGS=-L${libtool:location}/lib -Wl,-rpath -Wl,${libtool:location}/lib -L${libuuid:location}/lib -Wl,-rpath -Wl,${libuuid:location}/lib
setup.py
View file @ d007050d
......@@ -28,7 +28,7 @@ from setuptools import setup, find_packages
import glob
import os
version = '0.78.4.dev'
version = '0.84.2'
name = 'slapos.cookbook'
long_description = open("README.txt").read() + "\n" + \
open("CHANGES.txt").read() + "\n"
......@@ -70,7 +70,6 @@ setup(name=name,
'zc.buildout': [
'addresiliency = slapos.recipe.addresiliency:Recipe',
'agent = slapos.recipe.agent:Recipe',
'apache.frontend = slapos.recipe.apache_frontend:Recipe',
'apache.zope.backend = slapos.recipe.apache_zope_backend:Recipe',
'apacheperl = slapos.recipe.apacheperl:Recipe',
'apachephp = slapos.recipe.apachephp:Recipe',
......@@ -166,8 +165,7 @@ setup(name=name,
'publish.serialised = slapos.recipe.publish:Serialised',
'publishsection = slapos.recipe.publish:PublishSection',
'publishurl = slapos.recipe.publishurl:Recipe',
'pwgen = slapos.recipe.pwgen:Recipe',
'pwgen.stable = slapos.recipe.pwgen:StablePasswordGeneratorRecipe',
'readline = slapos.recipe.readline:Recipe',
'redis.server = slapos.recipe.redis:Recipe',
'request = slapos.recipe.request:Recipe',
'request.serialised = slapos.recipe.request:Serialised',
......@@ -193,6 +191,7 @@ setup(name=name,
'slaprunner.import = slapos.recipe.slaprunner.backup:ImportRecipe',
'softwaretype = slapos.recipe.softwaretype:Recipe',
'sphinx= slapos.recipe.sphinx:Recipe',
'squid = slapos.recipe.squid:Recipe',
'sshkeys_authority = slapos.recipe.sshkeys_authority:Recipe',
'sshkeys_authority.request = slapos.recipe.sshkeys_authority:Request',
'stunnel = slapos.recipe.stunnel:Recipe',
......
slapos/recipe/addresiliency/renamer.py deleted 100644 → 0
View file @ 55b2bae2
# -*- coding: utf-8 -*-
import logging
import time
import slapos
log = logging.getLogger(__name__)
logging.basicConfig(level=logging.DEBUG)
class Renamer(object):
def __init__(self, server_url, key_file, cert_file, computer_guid,
partition_id, software_release, namebase):
self.server_url = server_url
self.key_file = key_file
self.cert_file = cert_file
self.computer_guid = computer_guid
self.partition_id = partition_id
self.software_release = software_release
self.namebase = namebase
def _failover(self):
"""\
This method does
- retrieve the broken computer partition
- change its reference to 'broken-...' and its software type to 'frozen'
- retrieve the winner computer partition (attached to this process)
- change its reference to replace the broken one.
later, slapgrid will change its software_type as well.
Then, after running slapgrid-cp a few times, the winner takes over and
a new cp is created to replace it as an importer.
"""
slap = slapos.slap.slap()
slap.initializeConnection(self.server_url, self.key_file, self.cert_file)
# partition that will take over.
cp_winner = slap.registerComputerPartition(computer_guid=self.computer_guid,
partition_id=self.partition_id)
# XXX although we can already rename cp_winner, to change its software type we need to
# get hold of the root cp as well
cp_exporter_ref = self.namebase + '0' # this is ok. the boss is always number zero.
# partition to be deactivated
cp_broken = cp_winner.request(software_release=self.software_release,
software_type='frozen',
state='stopped',
partition_reference=cp_exporter_ref)
broken_new_ref = 'broken-{}'.format(time.strftime("%d-%b_%H:%M:%S", time.gmtime()))
log.debug("Renaming {}: {}".format(cp_broken.getId(), broken_new_ref))
cp_broken.rename(new_name=broken_new_ref)
cp_broken.stopped()
log.debug("Renaming {}: {}".format(cp_winner.getId(), cp_exporter_ref))
# update name (and later, software type) for the partition that will take over
cp_winner.rename(new_name=cp_exporter_ref)
cp_winner.bang(message='partitions have been renamed!')
def failover(self):
try:
self._failover()
log.info('Renaming done')
except slapos.slap.ServerError:
log.info('Internal server error')
slapos/recipe/addresiliency/takeover.py
View file @ d007050d
# -*- coding: utf-8 -*-
import logging
import time
import traceback
import slapos.recipe.addresiliency.renamer
import slapos
from slapos.slap.slap import NotFoundError
def run(args):
renamer = slapos.recipe.addresiliency.renamer.Renamer(server_url = args.pop('server_url'),
key_file = args.pop('key_file'),
cert_file = args.pop('cert_file'),
computer_guid = args.pop('computer_id'),
partition_id = args.pop('partition_id'),
software_release = args.pop('software'),
namebase = args.pop('namebase'))
log = logging.getLogger(__name__)
logging.basicConfig(level=logging.DEBUG)
def takeover(server_url, key_file, cert_file, computer_guid,
partition_id, software_release, namebase,
winner_instance_suffix = None):
"""
This function does
- retrieve the broken computer partition
- change its reference to 'broken-...' and its software type to 'frozen'
- retrieve the winner computer partition (attached to this process)
- change its reference to replace the broken one.
later, slapgrid will change its software_type as well.
Then, after running slapgrid-cp a few times, the winner takes over and
a new cp is created to replace it as an importer.
"""
slap = slapos.slap.slap()
slap.initializeConnection(server_url, key_file, cert_file)
current_partition = slap.registerComputerPartition(computer_guid=computer_guid,
partition_id=partition_id)
# partition that will take over.
if winner_instance_suffix:
winner_instance_name = namebase + winner_instance_suffix
# XXX: we hardcode a lot of values here, because request is a settergetter, all at once.
cp_winner = current_partition.request(software_release=software_release,
software_type='%s-import' % namebase,
partition_reference=winner_instance_name)
else:
# This script is run in the winning partition: use this one as winner
cp_winner = current_partition
# XXX although we can already rename cp_winner, to change its software type we need to
# get hold of the root cp as well
cp_exporter_ref = namebase + '0' # this is ok. the boss is always number zero.
renamer.failover()
# partition to be deactivated
cp_broken = cp_winner.request(software_release=software_release,
software_type='frozen',
state='stopped',
partition_reference=cp_exporter_ref)
broken_new_ref = 'broken-{}'.format(time.strftime("%d-%b_%H:%M:%S", time.gmtime()))
log.debug("Renaming {}: {}".format(cp_broken.getId(), broken_new_ref))
cp_broken.rename(new_name=broken_new_ref)
log.debug("Renaming {}: {}".format(cp_winner.getId(), cp_exporter_ref))
# update name (and later, software type) for the partition that will take over
while True:
time.sleep(10)
try:
cp_winner.rename(new_name=cp_exporter_ref)
break
except NotFoundError:
traceback.print_exc()
log.warning('Impossible to rename. Retrying in a few seconds...')
log.debug('Renamed.')
cp_winner.bang(message='partitions have been renamed!')
# Note: Root instance will reconfigure itself the winning instance (software_type
# and parameters.)
def run(args):
slapos.recipe.addresiliency.takeover.takeover(server_url = args.pop('server_url'),
key_file = args.pop('key_file'),
cert_file = args.pop('cert_file'),
computer_guid = args.pop('computer_id'),
partition_id = args.pop('partition_id'),
software_release = args.pop('software'),
namebase = args.pop('namebase'))
slapos/recipe/apache_frontend/__init__.py deleted 100644 → 0
View file @ 55b2bae2
##############################################################################
#
# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
from slapos.recipe.librecipe import BaseSlapRecipe
import os
import pkg_resources
import hashlib
import operator
import sys
import zc.buildout
import zc.recipe.egg
import ConfigParser
import re
import traceback
TRUE_VALUES = ['y', 'yes', '1', 'true']
class Recipe(BaseSlapRecipe):
def getTemplateFilename(self, template_name):
return pkg_resources.resource_filename(__name__,
'template/%s' % template_name)
def _install(self):
# Define directory not defined in deprecated lib
self.service_directory = os.path.join(self.etc_directory, 'service')
# Check for mandatory arguments
frontend_domain_name = self.parameter_dict.get("domain")
if frontend_domain_name is None:
raise zc.buildout.UserError('No domain name specified. Please define '
'the "domain" instance parameter.')
# Define optional arguments
frontend_port_number = self.parameter_dict.get("port", 4443)
frontend_plain_http_port_number = self.parameter_dict.get(
"plain_http_port", 8080)
base_varnish_port = 26009
slave_instance_list = self.parameter_dict.get("slave_instance_list", [])
self.path_list = []
self.requirements, self.ws = self.egg.working_set()
# self.cron_d is a directory, where cron jobs can be registered
self.cron_d = self.installCrond()
self.killpidfromfile = zc.buildout.easy_install.scripts(
[('killpidfromfile', 'slapos.toolbox.killpidfromfile',
'killpidfromfile')], self.ws, sys.executable, self.bin_directory)[0]
self.path_list.append(self.killpidfromfile)
rewrite_rule_list = []
rewrite_rule_https_only_list = []
rewrite_rule_zope_list = []
rewrite_rule_zope_path_list = []
slave_dict = {}
service_dict = {}
# Sort slave instance by reference to avoid most security issues
slave_instance_list = sorted(slave_instance_list,
key=operator.itemgetter('slave_reference'))
# dict of used domains, only used to track duplicates
domain_dict = {}
for slave_instance in slave_instance_list:
# Sanitize inputs
backend_url = slave_instance.get("url", None)
reference = slave_instance.get("slave_reference")
enable_cache = slave_instance.get('enable_cache', '').lower() in TRUE_VALUES
slave_type = slave_instance.get('type', '').lower() or None
https_only = slave_instance.get('https-only', '').lower() in TRUE_VALUES
# Set scheme (http? https?)
if https_only:
scheme = 'https://'
else:
scheme = 'http://'
self.logger.info('Processing slave instance: %s' % reference)
# Check for mandatory slave fields
if backend_url is None:
self.logger.warn('No "url" parameter is defined for %s slave'\
'instance. Ignoring it.' % reference)
continue
# Check for custom domain (like mypersonaldomain.com)
# If no custom domain, use generated one.
# Note: if we get an empty custom_domain parameter, we ignore it
domain = slave_instance.get('custom_domain')
if isinstance(domain, basestring):
domain = domain.strip()
if domain is None or domain.strip() == '':
domain = "%s.%s" % (reference.replace("-", "").lower(),
frontend_domain_name)
if domain_dict.get(domain):
# This domain already has been processed, skip this new one
continue
else:
domain_dict[domain] = True
# Define the URL where the instance will be available
# WARNING: we use default ports (443, 80) here.
slave_dict[reference] = "%s%s/" % (scheme, domain)
# Check if we want varnish+stunnel cache.
#if enable_cache:
# # XXX-Cedric : need to refactor to clean code? (to many variables)
# rewrite_rule = self.configureVarnishSlave(
# base_varnish_port, backend_url, reference, service_dict, domain)
# base_varnish_port += 2
#else:
# rewrite_rule = "%s %s" % (domain, backend_url)
# Temporary forbid activation of cache until it is properly tested
rewrite_rule = "%s %s" % (domain, backend_url)
# Finally, if successful, we add the rewrite rule to our list of rules
# We have 4 RewriteMaps:
# - One for generic (non-zope) websites, accepting both HTTP and HTTPS
# - One for generic websites that only accept HTTPS
# - Two for Zope-based websites
if rewrite_rule:
# We check if we have a zope slave. It requires different rewrite
# rule structure.
# So we will have one RewriteMap for normal websites, and one
# RewriteMap for Zope Virtual Host Monster websites.
if slave_type in ['zope']:
rewrite_rule_zope_list.append(rewrite_rule)
# For Zope, we have another dict containing the path e.g '/erp5/...
rewrite_rule_path = "%s %s" % (domain, slave_instance.get('path', ''))
rewrite_rule_zope_path_list.append(rewrite_rule_path)
else:
if https_only:
rewrite_rule_https_only_list.append(rewrite_rule)
else:
rewrite_rule_list.append(rewrite_rule)
# Certificate stuff
valid_certificate_str = self.parameter_dict.get("domain_ssl_ca_cert")
valid_key_str = self.parameter_dict.get("domain_ssl_ca_key")
if valid_certificate_str is None and valid_key_str is None:
ca_conf = self.installCertificateAuthority()
key, certificate = self.requestCertificate(frontend_domain_name)
else:
ca_conf = self.installValidCertificateAuthority(
frontend_domain_name, valid_certificate_str, valid_key_str)
key = ca_conf.pop("key")
certificate = ca_conf.pop("certificate")
if service_dict != {}:
if valid_certificate_str is not None and valid_key_str is not None:
self.installCertificateAuthority()
stunnel_key, stunnel_certificate = \
self.requestCertificate(frontend_domain_name)
else:
stunnel_key, stunnel_certificate = key, certificate
self.installStunnel(service_dict,
stunnel_certificate, stunnel_key,
ca_conf["ca_crl"],
ca_conf["certificate_authority_path"])
apache_parameter_dict = self.installFrontendApache(
ip_list=["[%s]" % self.getGlobalIPv6Address(),
self.getLocalIPv4Address()],
port=frontend_port_number,
plain_http_port=frontend_plain_http_port_number,
name=frontend_domain_name,
rewrite_rule_list=rewrite_rule_list,
rewrite_rule_https_only_list=rewrite_rule_https_only_list,
rewrite_rule_zope_list=rewrite_rule_zope_list,
rewrite_rule_zope_path_list=rewrite_rule_zope_path_list,
key=key, certificate=certificate)
# Send connection informations about each slave
for reference, url in slave_dict.iteritems():
self.logger.debug("Sending connection parameters of slave "
"instance: %s" % reference)
try:
connection_dict = {
# Send the public IPs (if possible) so that user knows what IP
# to bind to its domain name
'frontend_ipv6_address': self.getGlobalIPv6Address(),
'frontend_ipv4_address': self.parameter_dict.get("public-ipv4",
self.getLocalIPv4Address()),
'site_url': url,
}
self.setConnectionDict(connection_dict, reference)
except:
self.logger.fatal("Error while sending slave %s informations: %s",
reference, traceback.format_exc())
# Then set it for master instance
self.setConnectionDict(
dict(site_url=apache_parameter_dict["site_url"],
frontend_ipv6_address=self.getGlobalIPv6Address(),
frontend_ipv4_address=self.getLocalIPv4Address()))
# Promises
promise_config = dict(
hostname=self.getGlobalIPv6Address(),
port=frontend_port_number,
python_path=sys.executable,
)
promise_v6 = self.createPromiseWrapper(
'apache_ipv6',
self.substituteTemplate(
pkg_resources.resource_filename(
'slapos.recipe.check_port_listening',
'template/socket_connection_attempt.py.in'),
promise_config))
self.path_list.append(promise_v6)
promise_config = dict(
hostname=self.getLocalIPv4Address(),
port=frontend_port_number,
python_path=sys.executable,
)
promise_v4 = self.createPromiseWrapper(
'apache_ipv4',
self.substituteTemplate(
pkg_resources.resource_filename(
'slapos.recipe.check_port_listening',
'template/socket_connection_attempt.py.in'),
promise_config))
self.path_list.append(promise_v4)
return self.path_list
def configureVarnishSlave(self, base_varnish_port, url, reference,
service_dict, domain):
# Varnish should use stunnel to connect to the backend
base_varnish_control_port = base_varnish_port
base_varnish_port += 1
# Use regex
host_regex = "((\[\w*|[0-9]+\.)(\:|)).*(\]|\.[0-9]+)"
slave_host = re.search(host_regex, url).group(0)
port_regex = "\w+(\/|)$"
matcher = re.search(port_regex, url)
if matcher is not None:
slave_port = matcher.group(0)
slave_port = slave_port.replace("/", "")
elif url.startswith("https://"):
slave_port = 443
else:
slave_port = 80
service_name = "varnish_%s" % reference
varnish_ip = self.getLocalIPv4Address()
stunnel_port = base_varnish_port + 1
self.installVarnishCache(service_name,
ip=varnish_ip,
port=base_varnish_port,
control_port=base_varnish_control_port,
backend_host=varnish_ip,
backend_port=stunnel_port,
size="1G")
service_dict[service_name] = dict(public_ip=varnish_ip,
public_port=stunnel_port,
private_ip=slave_host.replace("[", "").replace("]", ""),
private_port=slave_port)
return "%s http://%s:%s" % \
(domain, varnish_ip, base_varnish_port)
def requestCertificate(self, name):
hash = hashlib.sha512(name).hexdigest()
key = os.path.join(self.ca_private, hash + self.ca_key_ext)
certificate = os.path.join(self.ca_certs, hash + self.ca_crt_ext)
parser = ConfigParser.RawConfigParser()
parser.add_section('certificate')
parser.set('certificate', 'name', name)
parser.set('certificate', 'key_file', key)
parser.set('certificate', 'certificate_file', certificate)
parser.write(open(os.path.join(self.ca_request_dir, hash), 'w'))
return key, certificate
def installCrond(self):
timestamps = self.createDataDirectory('cronstamps')
cron_output = os.path.join(self.log_directory, 'cron-output')
self._createDirectory(cron_output)
catcher = zc.buildout.easy_install.scripts([('catchcron',
__name__ + '.catdatefile', 'catdatefile')], self.ws, sys.executable,
self.bin_directory, arguments=[cron_output])[0]
self.path_list.append(catcher)
cron_d = os.path.join(self.etc_directory, 'cron.d')
crontabs = os.path.join(self.etc_directory, 'crontabs')
self._createDirectory(cron_d)
self._createDirectory(crontabs)
wrapper = zc.buildout.easy_install.scripts([('crond',
'slapos.recipe.librecipe.execute', 'execute')], self.ws, sys.executable,
self.service_directory, arguments=[
self.options['dcrond_binary'].strip(), '-s', cron_d, '-c', crontabs,
'-t', timestamps, '-f', '-l', '5', '-M', catcher]
)[0]
self.path_list.append(wrapper)
return cron_d
def installValidCertificateAuthority(self, domain_name, certificate, key):
ca_dir = os.path.join(self.data_root_directory, 'ca')
ca_private = os.path.join(ca_dir, 'private')
ca_certs = os.path.join(ca_dir, 'certs')
ca_crl = os.path.join(ca_dir, 'crl')
self._createDirectory(ca_dir)
for path in (ca_private, ca_certs, ca_crl):
self._createDirectory(path)
key_path = os.path.join(ca_private, domain_name + ".key")
certificate_path = os.path.join(ca_certs, domain_name + ".crt")
self._writeFile(key_path, key)
self._writeFile(certificate_path, certificate)
return dict(certificate_authority_path=ca_dir,
ca_crl=ca_crl,
certificate=certificate_path,
key=key_path)
def installCertificateAuthority(self, ca_country_code='XX',
ca_email='xx@example.com', ca_state='State', ca_city='City',
ca_company='Company'):
backup_path = self.createBackupDirectory('ca')
self.ca_dir = os.path.join(self.data_root_directory, 'ca')
self._createDirectory(self.ca_dir)
self.ca_request_dir = os.path.join(self.ca_dir, 'requests')
self._createDirectory(self.ca_request_dir)
config = dict(ca_dir=self.ca_dir, request_dir=self.ca_request_dir)
self.ca_private = os.path.join(self.ca_dir, 'private')
self.ca_certs = os.path.join(self.ca_dir, 'certs')
self.ca_crl = os.path.join(self.ca_dir, 'crl')
self.ca_newcerts = os.path.join(self.ca_dir, 'newcerts')
self.ca_key_ext = '.key'
self.ca_crt_ext = '.crt'
for d in [self.ca_private, self.ca_crl, self.ca_newcerts, self.ca_certs]:
self._createDirectory(d)
for f in ['crlnumber', 'serial']:
if not os.path.exists(os.path.join(self.ca_dir, f)):
open(os.path.join(self.ca_dir, f), 'w').write('01')
if not os.path.exists(os.path.join(self.ca_dir, 'index.txt')):
open(os.path.join(self.ca_dir, 'index.txt'), 'w').write('')
openssl_configuration = os.path.join(self.ca_dir, 'openssl.cnf')
config.update(
working_directory=self.ca_dir,
country_code=ca_country_code,
state=ca_state,
city=ca_city,
company=ca_company,
email_address=ca_email,
)
self._writeFile(openssl_configuration, pkg_resources.resource_string(
__name__, 'template/openssl.cnf.ca.in') % config)
# XXX-Cedric: Don't use this, but use slapos.recipe.certificate_authority
# from the instance profile.
self.path_list.extend(zc.buildout.easy_install.scripts([
('certificate_authority', __name__ + '.certificate_authority',
'runCertificateAuthority')],
self.ws, sys.executable, self.service_directory, arguments=[dict(
openssl_configuration=openssl_configuration,
openssl_binary=self.options['openssl_binary'],
certificate=os.path.join(self.ca_dir, 'cacert.pem'),
key=os.path.join(self.ca_private, 'cakey.pem'),
crl=os.path.join(self.ca_crl),
request_dir=self.ca_request_dir
)]))
# configure backup
backup_cron = os.path.join(self.cron_d, 'ca_rdiff_backup')
open(backup_cron, 'w').write(
'''0 0 * * * %(rdiff_backup)s %(source)s %(destination)s'''%dict(
rdiff_backup=self.options['rdiff_backup_binary'],
source=self.ca_dir,
destination=backup_path))
self.path_list.append(backup_cron)
return dict(
ca_certificate=os.path.join(config['ca_dir'], 'cacert.pem'),
ca_crl=os.path.join(config['ca_dir'], 'crl'),
certificate_authority_path=config['ca_dir']
)
def _getApacheConfigurationDict(self, name, ip_list, port):
apache_conf = dict()
apache_conf['server_name'] = name
apache_conf['pid_file'] = self.options['pid-file']
apache_conf['lock_file'] = os.path.join(self.run_directory,
name + '.lock')
apache_conf['document_root'] = os.path.join(self.data_root_directory,
'htdocs')
apache_conf['instance_home'] = os.path.join(self.work_directory)
apache_conf['httpd_home'] = self.options['httpd_home']
apache_conf['ip_list'] = ip_list
apache_conf['port'] = port
apache_conf['server_admin'] = 'admin@'
apache_conf['error_log'] = self.options['error-log']
apache_conf['access_log'] = self.options['access-log']
return apache_conf
def installVarnishCache(self, name, ip, port, control_port, backend_host,
backend_port, size="1G"):
"""
Install a varnish daemon for a certain address
"""
directory = self.createDataDirectory(name)
varnish_config = dict(
directory=directory,
pid = "%s/varnish.pid" % directory,
port="%s:%s" % (ip, port),
varnishd_binary=self.options["varnishd_binary"],
control_port="%s:%s" % (ip, control_port),
storage="file,%s/storage.bin,%s" % (directory, size))
config_file = self.createConfigurationFile("%s.conf" % name,
self.substituteTemplate(self.getTemplateFilename('varnish.vcl.in'),
dict(backend_host=backend_host, backend_port=backend_port)))
varnish_argument_list = [varnish_config['varnishd_binary'].strip(),
"-F", "-n", directory, "-P", varnish_config["pid"], "-p",
"cc_command=exec %s " % self.options["gcc_binary"] +\
"-fpic -shared -o %o %s",
"-f", config_file,
"-a", varnish_config["port"], "-T", varnish_config["control_port"],
"-s", varnish_config["storage"]]
environment = dict(PATH="%s:%s" % (self.options["binutils_directory"],
os.environ.get('PATH')))
wrapper = zc.buildout.easy_install.scripts([(name,
'slapos.recipe.librecipe.execute', 'executee')], self.ws,
sys.executable, self.service_directory, arguments=[varnish_argument_list,
environment])[0]
self.path_list.append(wrapper)
return varnish_config
def installStunnel(self, service_dict, certificate,
key, ca_crl, ca_path):
"""Installs stunnel
service_dict =
{ name: (public_ip, private_ip, public_port, private_port),}
"""
template_filename = self.getTemplateFilename('stunnel.conf.in')
template_entry_filename = self.getTemplateFilename('stunnel.conf.entry.in')
log = os.path.join(self.log_directory, 'stunnel.log')
pid_file = os.path.join(self.run_directory, 'stunnel.pid')
stunnel_conf = dict(
pid_file=pid_file,
log=log,
cert = certificate,
key = key,
ca_crl = ca_crl,
ca_path = ca_path,
entry_str=''
)
entry_list = []
for name, parameter_dict in service_dict.iteritems():
parameter_dict["name"] = name
entry_str = self.substituteTemplate(template_entry_filename,
parameter_dict)
entry_list.append(entry_str)
stunnel_conf["entry_str"] = "\n".join(entry_list)
stunnel_conf_path = self.createConfigurationFile("stunnel.conf",
self.substituteTemplate(template_filename,
stunnel_conf))
wrapper = zc.buildout.easy_install.scripts([('stunnel',
'slapos.recipe.librecipe.execute', 'execute_wait')], self.ws,
sys.executable, self.service_directory, arguments=[
[self.options['stunnel_binary'].strip(), stunnel_conf_path],
[certificate, key]]
)[0]
self.path_list.append(wrapper)
return stunnel_conf
def installFrontendApache(self, ip_list, key, certificate, name,
port=4443, plain_http_port=8080,
rewrite_rule_list=None,
rewrite_rule_zope_list=None,
rewrite_rule_https_only_list=None,
rewrite_rule_zope_path_list=None,
access_control_string=None):
if rewrite_rule_list is None:
rewrite_rule_list = []
if rewrite_rule_https_only_list is None:
rewrite_rule_zope_path_list = []
if rewrite_rule_zope_list is None:
rewrite_rule_zope_list = []
if rewrite_rule_zope_path_list is None:
rewrite_rule_zope_path_list = []
# Create htdocs, populate it with default 404 document
htdocs_location = os.path.join(self.data_root_directory, 'htdocs')
self._createDirectory(htdocs_location)
notfound_file_location = os.path.join(htdocs_location, 'notfound.html')
notfound_template_file_location = self.getTemplateFilename(
'notfound.html')
notfound_file_content = open(notfound_template_file_location, 'r').read()
self._writeFile(notfound_file_location, notfound_file_content)
# Create mod_ssl cache directory
cache_directory_location = os.path.join(self.var_directory, 'cache')
mod_ssl_cache_location = os.path.join(cache_directory_location,
'httpd_mod_ssl')
self._createDirectory(cache_directory_location)
self._createDirectory(mod_ssl_cache_location)
# Create "custom" apache configuration files if it does not exist.
# Note : Those files won't be erased or changed by slapgrid.
# It can be freely customized by node admin.
custom_apache_configuration_directory = os.path.join(
self.data_root_directory, 'apache-conf.d')
self._createDirectory(custom_apache_configuration_directory)
# First one is included in the end of the apache configuration file
custom_apache_configuration_file_location = os.path.join(
custom_apache_configuration_directory, 'apache_frontend.custom.conf')
if not os.path.exists(custom_apache_configuration_file_location):
open(custom_apache_configuration_file_location, 'w')
# Second one is included in the virtualhost of apache configuration file
custom_apache_virtual_configuration_file_location = os.path.join(
custom_apache_configuration_directory,
'apache_frontend.virtualhost.custom.conf')
if not os.path.exists(custom_apache_virtual_configuration_file_location):
open(custom_apache_virtual_configuration_file_location, 'w')
# Create backup of custom apache configuration
backup_path = self.createBackupDirectory('custom_apache_conf_backup')
backup_cron = os.path.join(self.cron_d, 'custom_apache_conf_backup')
open(backup_cron, 'w').write(
'''0 0 * * * %(rdiff_backup)s %(source)s %(destination)s'''%dict(
rdiff_backup=self.options['rdiff_backup_binary'],
source=custom_apache_configuration_directory,
destination=backup_path))
self.path_list.append(backup_cron)
# Create configuration file and rewritemaps
apachemap_path = self.createConfigurationFile(
"apache_rewritemap_generic.txt",
"\n".join(rewrite_rule_list)
)
apachemap_httpsonly_path = self.createConfigurationFile(
"apache_rewritemap_httpsonly.txt",
"\n".join(rewrite_rule_https_only_list)
)
apachemap_zope_path = self.createConfigurationFile(
"apache_rewritemap_zope.txt",
"\n".join(rewrite_rule_zope_list)
)
apachemap_zopepath_path = self.createConfigurationFile(
"apache_rewritemap_zopepath.txt",
"\n".join(rewrite_rule_zope_path_list)
)
apache_conf = self._getApacheConfigurationDict(name, ip_list, port)
apache_conf['ssl_snippet'] = self.substituteTemplate(
self.getTemplateFilename('apache.ssl-snippet.conf.in'),
dict(login_certificate=certificate,
login_key=key,
httpd_mod_ssl_cache_directory=mod_ssl_cache_location,
)
)
apache_conf["listen"] = "\n".join([
"Listen %s:%s" % (ip, port)
for port in (plain_http_port, port)
for ip in ip_list
])
path = self.substituteTemplate(
self.getTemplateFilename('apache.conf.path-protected.in'),
dict(path='/',
access_control_string='none',
document_root=apache_conf['document_root'],
)
)
apache_conf.update(**dict(
path_enable=path,
apachemap_path=apachemap_path,
apachemap_httpsonly_path=apachemap_httpsonly_path,
apachemapzope_path=apachemap_zope_path,
apachemapzopepath_path=apachemap_zopepath_path,
apache_domain=name,
https_port=port,
plain_http_port=plain_http_port,
custom_apache_conf=custom_apache_configuration_file_location,
custom_apache_virtualhost_conf=custom_apache_virtual_configuration_file_location,
))
apache_conf_string = self.substituteTemplate(
self.getTemplateFilename('apache.conf.in'), apache_conf)
apache_config_file = self.createConfigurationFile('apache_frontend.conf',
apache_conf_string)
self.path_list.append(apache_config_file)
self.path_list.extend(zc.buildout.easy_install.scripts([(
'frontend_apache', 'slapos.recipe.erp5.apache', 'runApache')], self.ws,
sys.executable, self.service_directory, arguments=[
dict(
required_path_list=[key, certificate],
binary=self.options['httpd_binary'],
config=apache_config_file)
]))
return dict(site_url="https://%s:%s/" % (name, port))
slapos/recipe/apache_frontend/certificate_authority.py deleted 100755 → 0
View file @ 55b2bae2
import os
import subprocess
import time
import ConfigParser
import uuid
def popenCommunicate(command_list, input=None):
subprocess_kw = dict(stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
if input is not None:
subprocess_kw.update(stdin=subprocess.PIPE)
popen = subprocess.Popen(command_list, **subprocess_kw)
result = popen.communicate(input)[0]
if popen.returncode is None:
popen.kill()
if popen.returncode != 0:
raise ValueError('Issue during calling %r, result was:\n%s' % (
command_list, result))
return result
class CertificateAuthority:
def __init__(self, key, certificate, openssl_binary,
openssl_configuration, request_dir):
self.key = key
self.certificate = certificate
self.openssl_binary = openssl_binary
self.openssl_configuration = openssl_configuration
self.request_dir = request_dir
def checkAuthority(self):
file_list = [ self.key, self.certificate ]
ca_ready = True
for f in file_list:
if not os.path.exists(f):
ca_ready = False
break
if ca_ready:
return
for f in file_list:
if os.path.exists(f):
os.unlink(f)
try:
# no CA, let us create new one
popenCommunicate([self.openssl_binary, 'req', '-nodes', '-config',
self.openssl_configuration, '-new', '-x509', '-extensions', 'v3_ca',
'-keyout', self.key, '-out', self.certificate, '-days', '10950'],
# Authority name will be random, so no instance has the same issuer
'Certificate Authority %s\n' % uuid.uuid1())
except:
try:
for f in file_list:
if os.path.exists(f):
os.unlink(f)
except:
# do not raise during cleanup
pass
raise
def _checkCertificate(self, common_name, key, certificate):
file_list = [key, certificate]
ready = True
for f in file_list:
if not os.path.exists(f):
ready = False
break
if ready:
return False
for f in file_list:
if os.path.exists(f):
os.unlink(f)
csr = certificate + '.csr'
try:
popenCommunicate([self.openssl_binary, 'req', '-config',
self.openssl_configuration, '-nodes', '-new', '-keyout',
key, '-out', csr, '-days', '3650'],
common_name + '\n')
try:
popenCommunicate([self.openssl_binary, 'ca', '-batch', '-config',
self.openssl_configuration, '-out', certificate,
'-infiles', csr])
finally:
if os.path.exists(csr):
os.unlink(csr)
except:
try:
for f in file_list:
if os.path.exists(f):
os.unlink(f)
except:
# do not raise during cleanup
pass
raise
else:
return True
def checkRequestDir(self):
for request_file in os.listdir(self.request_dir):
parser = ConfigParser.RawConfigParser()
parser.readfp(open(os.path.join(self.request_dir, request_file), 'r'))
if self._checkCertificate(parser.get('certificate', 'name'),
parser.get('certificate', 'key_file'), parser.get('certificate',
'certificate_file')):
print 'Created certificate %r' % parser.get('certificate', 'name')
def runCertificateAuthority(args):
ca_conf = args[0]
ca = CertificateAuthority(ca_conf['key'], ca_conf['certificate'],
ca_conf['openssl_binary'], ca_conf['openssl_configuration'],
ca_conf['request_dir'])
while True:
ca.checkAuthority()
ca.checkRequestDir()
time.sleep(60)
slapos/recipe/apache_frontend/template/apache.conf.path-protected.in deleted 100644 → 0
View file @ 55b2bae2
<Directory %(path)s>
Order Deny,Allow
Allow from %(access_control_string)s
</Directory>
<Directory %(document_root)s>
Order Allow,Deny
Allow from All
</Directory>
slapos/recipe/apache_frontend/template/apache.location-snippet.conf.in deleted 100644 → 0
View file @ 55b2bae2
<Location %(location)s>
Order Deny,Allow
Deny from all
Allow from %(allow_string)s
</Location>
slapos/recipe/apache_frontend/template/apache.ssl-snippet.conf.in deleted 100644 → 0
View file @ 55b2bae2
SSLCertificateFile %(login_certificate)s
SSLCertificateKeyFile %(login_key)s
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLSessionCache shmcb:/%(httpd_mod_ssl_cache_directory)s/ssl_scache(512000)
SSLSessionCacheTimeout 300
SSLRandomSeed startup /dev/urandom 256
SSLRandomSeed connect builtin
SSLProtocol -ALL +SSLv3 +TLSv1
SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!ADH
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
# Accept proxy to sites using self-signed SSL certificates
SSLProxyCheckPeerCN off
SSLProxyCheckPeerExpire off
slapos/recipe/apache_frontend/template/logrotate_entry.in deleted 100644 → 0
View file @ 55b2bae2
%(file_list)s {
daily
dateext
rotate 30
compress
notifempty
sharedscripts
create
postrotate
%(postrotate)s
endscript
olddir %(olddir)s
}
slapos/recipe/apache_frontend/template/openssl.cnf.ca.in deleted 100644 → 0
View file @ 55b2bae2
#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#
# This definition stops the following lines choking if HOME isn't
# defined.
HOME = .
RANDFILE = $ENV::HOME/.rnd
# Extra OBJECT IDENTIFIER info:
#oid_file = $ENV::HOME/.oid
oid_section = new_oids
# To use this configuration file with the "-extfile" option of the
# "openssl x509" utility, name here the section containing the
# X.509v3 extensions to use:
# extensions =
# (Alternatively, use a configuration file that has only
# X.509v3 extensions in its main [= default] section.)
[ new_oids ]
# We can add new OIDs in here for use by 'ca', 'req' and 'ts'.
# Add a simple OID like this:
# testoid1=1.2.3.4
# Or use config file substitution like this:
# testoid2=${testoid1}.5.6
# Policies used by the TSA examples.
tsa_policy1 = 1.2.3.4.1
tsa_policy2 = 1.2.3.4.5.6
tsa_policy3 = 1.2.3.4.5.7
####################################################################
[ ca ]
default_ca = CA_default # The default ca section
####################################################################
[ CA_default ]
dir = %(working_directory)s # Where everything is kept
certs = $dir/certs # Where the issued certs are kept
crl_dir = $dir/crl # Where the issued crl are kept
database = $dir/index.txt # database index file.
#unique_subject = no # Set to 'no' to allow creation of
# several ctificates with same subject.
new_certs_dir = $dir/newcerts # default place for new certs.
certificate = $dir/cacert.pem # The CA certificate
serial = $dir/serial # The current serial number
crlnumber = $dir/crlnumber # the current crl number
# must be commented out to leave a V1 CRL
crl = $dir/crl.pem # The current CRL
private_key = $dir/private/cakey.pem # The private key
RANDFILE = $dir/private/.rand # private random number file
x509_extensions = usr_cert # The extentions to add to the cert
# Comment out the following two lines for the "traditional"
# (and highly broken) format.
name_opt = ca_default # Subject Name options
cert_opt = ca_default # Certificate field options
# Extension copying option: use with caution.
# copy_extensions = copy
# Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs
# so this is commented out by default to leave a V1 CRL.
# crlnumber must also be commented out to leave a V1 CRL.
# crl_extensions = crl_ext
default_days = 3650 # how long to certify for
default_crl_days= 30 # how long before next CRL
default_md = default # use public key default MD
preserve = no # keep passed DN ordering
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy = policy_match
# For the CA policy
[ policy_match ]
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
# For the 'anything' policy
# At this point in time, you must list all acceptable 'object'
# types.
[ policy_anything ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
####################################################################
[ req ]
default_bits = 2048
default_md = sha1
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
#attributes = req_attributes
x509_extensions = v3_ca # The extentions to add to the self signed cert
# Passwords for private keys if not present they will be prompted for
# input_password = secret
# output_password = secret
# This sets a mask for permitted string types. There are several options.
# default: PrintableString, T61String, BMPString.
# pkix : PrintableString, BMPString (PKIX recommendation before 2004)
# utf8only: only UTF8Strings (PKIX recommendation after 2004).
# nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings).
# MASK:XXXX a literal mask value.
# WARNING: ancient versions of Netscape crash on BMPStrings or UTF8Strings.
string_mask = utf8only
# req_extensions = v3_req # The extensions to add to a certificate request
[ req_distinguished_name ]
countryName = Country Name (2 letter code)
countryName_value = %(country_code)s
countryName_min = 2
countryName_max = 2
stateOrProvinceName = State or Province Name (full name)
stateOrProvinceName_value = %(state)s
localityName = Locality Name (eg, city)
localityName_value = %(city)s
0.organizationName = Organization Name (eg, company)
0.organizationName_value = %(company)s
# we can do this but it is not needed normally :-)
#1.organizationName = Second Organization Name (eg, company)
#1.organizationName_default = World Wide Web Pty Ltd
commonName = Common Name (eg, your name or your server\'s hostname)
commonName_max = 64
emailAddress = Email Address
emailAddress_value = %(email_address)s
emailAddress_max = 64
# SET-ex3 = SET extension number 3
#[ req_attributes ]
#challengePassword = A challenge password
#challengePassword_min = 4
#challengePassword_max = 20
#
#unstructuredName = An optional company name
[ usr_cert ]
# These extensions are added when 'ca' signs a request.
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
basicConstraints=CA:FALSE
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
# This is OK for an SSL server.
# nsCertType = server
# For an object signing certificate this would be used.
# nsCertType = objsign
# For normal client use this is typical
# nsCertType = client, email
# and for everything including object signing:
# nsCertType = client, email, objsign
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
# This will be displayed in Netscape's comment listbox.
nsComment = "OpenSSL Generated Certificate"
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
# Copy subject details
# issuerAltName=issuer:copy
#nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
# This is required for TSA certificates.
# extendedKeyUsage = critical,timeStamping
[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
[ v3_ca ]
# Extensions for a typical CA
# PKIX recommendation.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer
# This is what PKIX recommends but some broken software chokes on critical
# extensions.
#basicConstraints = critical,CA:true
# So we do this instead.
basicConstraints = CA:true
# Key usage: this is typical for a CA certificate. However since it will
# prevent it being used as an test self-signed certificate it is best
# left out by default.
# keyUsage = cRLSign, keyCertSign
# Some might want this also
# nsCertType = sslCA, emailCA
# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy
# DER hex encoding of an extension: beware experts only!
# obj=DER:02:03
# Where 'obj' is a standard or added object
# You can even override a supported extension:
# basicConstraints= critical, DER:30:03:01:01:FF
[ crl_ext ]
# CRL extensions.
# Only issuerAltName and authorityKeyIdentifier make any sense in a CRL.
# issuerAltName=issuer:copy
authorityKeyIdentifier=keyid:always
[ proxy_cert_ext ]
# These extensions should be added when creating a proxy certificate
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
basicConstraints=CA:FALSE
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
# This is OK for an SSL server.
# nsCertType = server
# For an object signing certificate this would be used.
# nsCertType = objsign
# For normal client use this is typical
# nsCertType = client, email
# and for everything including object signing:
# nsCertType = client, email, objsign
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
# This will be displayed in Netscape's comment listbox.
nsComment = "OpenSSL Generated Certificate"
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
# Copy subject details
# issuerAltName=issuer:copy
#nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName
# This really needs to be in place for it to be a proxy certificate.
proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo
####################################################################
[ tsa ]
default_tsa = tsa_config1 # the default TSA section
[ tsa_config1 ]
# These are used by the TSA reply generation only.
dir = /etc/pki/tls # TSA root directory
serial = $dir/tsaserial # The current serial number (mandatory)
crypto_device = builtin # OpenSSL engine to use for signing
signer_cert = $dir/tsacert.pem # The TSA signing certificate
# (optional)
certs = $dir/cacert.pem # Certificate chain to include in reply
# (optional)
signer_key = $dir/private/tsakey.pem # The TSA private key (optional)
default_policy = tsa_policy1 # Policy if request did not specify it
# (optional)
other_policies = tsa_policy2, tsa_policy3 # acceptable policies (optional)
digests = md5, sha1 # Acceptable message digests (mandatory)
accuracy = secs:1, millisecs:500, microsecs:100 # (optional)
clock_precision_digits = 0 # number of digits after dot. (optional)
ordering = yes # Is ordering defined for timestamps?
# (optional, default: no)
tsa_name = yes # Must the TSA name be included in the reply?
# (optional, default: no)
ess_cert_id_chain = no # Must the ESS cert id chain be included?
# (optional, default: no)
slapos/recipe/apache_frontend/template/stunnel.conf.entry.in deleted 100644 → 0
View file @ 55b2bae2
[%(name)s]
accept = %(public_ip)s:%(public_port)s
connect = %(private_ip)s:%(private_port)s
slapos/recipe/apache_frontend/template/stunnel.conf.in deleted 100644 → 0
View file @ 55b2bae2
foreground = yes
output = %(log)s
pid = %(pid_file)s
syslog = no
client = yes
CApath = %(ca_path)s
key = %(key)s
CRLpath = %(ca_crl)s
cert = %(cert)s
sslVersion = SSLv3
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
%(entry_str)s
slapos/recipe/apache_frontend/template/varnish.vcl.in deleted 100644 → 0
View file @ 55b2bae2
# This is a basic VCL configuration file for varnish. See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# Default backend definition. Set this to point to your content
# server.
#
backend default {
.host = "%(backend_host)s";
.port = "%(backend_port)s";
.probe = {
.url = "/";
.timeout = 10s;
.interval = 10s;
.window = 4;
.threshold = 3;
}
}
#
# Below is a commented-out copy of the default VCL logic. If you
# redefine any of these subroutines, the built-in logic will be
# appended to your code.
#
# sub vcl_recv {
# if (req.http.x-forwarded-for) {
# set req.http.X-Forwarded-For =
# req.http.X-Forwarded-For ", " client.ip;
# } else {
# set req.http.X-Forwarded-For = client.ip;
# }
# if (req.request != "GET" &&
# req.request != "HEAD" &&
# req.request != "PUT" &&
# req.request != "POST" &&
# req.request != "TRACE" &&
# req.request != "OPTIONS" &&
# req.request != "DELETE") {
# /* Non-RFC2616 or CONNECT which is weird. */
# return (pipe);
# }
# if (req.request != "GET" && req.request != "HEAD") {
# /* We only deal with GET and HEAD by default */
# return (pass);
# }
# if (req.http.Authorization || req.http.Cookie) {
# /* Not cacheable by default */
# return (pass);
# }
# return (lookup);
# }
sub vcl_recv {
if (req.http.cache-control ~ "no-cache") {
purge_url(req.url);
}
if (req.url ~ "\.(css|js|ico)$") {
unset req.http.cookie;
}
# remove bogus cookies
if (req.http.Cookie) {
set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__utm.=[^;]+;? *", "\1");
set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__ac_name=\x22\x22;? *", "\1");
set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__ac=\x22Og.3D.3D\x22;? *", "\1");
}
if (req.http.Cookie == "") {
remove req.http.Cookie;
}
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For ", " client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.request != "GET" && req.request != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization) {
/* Not cacheable by default */
return (pass);
}
if (req.http.Cookie && req.http.Cookie ~ "(^|; ) *__ac=") {
/* Not cacheable for authorised users,
but KM images are cacheable */
if (!(req.url ~ "/km_img/.*\.(png|gif)$")) {
return (pass);
}
}
# XXX login form can defer based on __ac_name cookie value
if (req.url ~ "/(login_form|WebSite_viewLoginDialog)($|\?)") {
return (pass);
}
if (req.backend.healthy) {
set req.grace = 1h;
} else {
set req.grace = 1w;
}
return (lookup);
}
#
# sub vcl_pipe {
# # Note that only the first request to the backend will have
# # X-Forwarded-For set. If you use X-Forwarded-For and want to
# # have it set for all requests, make sure to have:
# # set req.http.connection = "close";
# # here. It is not set by default as it might break some broken web
# # applications, like IIS with NTLM authentication.
# return (pipe);
# }
#
# sub vcl_pass {
# return (pass);
# }
#
# sub vcl_hash {
# set req.hash += req.url;
# if (req.http.host) {
# set req.hash += req.http.host;
# } else {
# set req.hash += server.ip;
# }
# return (hash);
# }
#
# sub vcl_hit {
# if (!obj.cacheable) {
# return (pass);
# }
# return (deliver);
# }
#
# sub vcl_miss {
# return (fetch);
# }
#
# sub vcl_fetch {
# if (!beresp.cacheable) {
# return (pass);
# }
# if (beresp.http.Set-Cookie) {
# return (pass);
# }
# return (deliver);
# }
sub vcl_fetch {
# we only cache 200 (OK) and 304 (Not Modified) responses.
if (beresp.status != 200 && beresp.status != 304) {
set beresp.cacheable = false;
}
if (beresp.http.cache-control ~ "no-cache") {
set beresp.cacheable = false;
}
if (!beresp.cacheable) {
unset beresp.http.expires;
set beresp.http.cache-control = "no-cache";
return (pass);
}
# we don't care haproxy's cookie.
if (beresp.http.Set-Cookie && beresp.http.Set-Cookie !~ "^SERVERID=[^;]+; path=/$") {
return (pass);
}
if (req.url ~ "\.(css|js|ico)$") {
unset beresp.http.set-cookie;
set beresp.http.cache-control = regsub(beresp.http.cache-control, "^", "public,");
set beresp.http.cache-control = regsub(beresp.http.cache-control, ",$", "");
}
# remove some headers added by caching policy manager to avoid
# '304 Not Modified' in case of login <-> logout switching.
if (beresp.http.content-type ~ "^text/html") {
unset beresp.http.last-modified;
}
if (beresp.cacheable) {
/* Remove Expires from backend, it's not long enough */
unset beresp.http.expires;
/* Set the clients TTL on this object */
set beresp.http.cache-control = "max-age = 900";
/* Set how long Varnish will keep it */
set beresp.ttl = 1w;
/* marker for vcl_deliver to reset Age: */
set beresp.http.magicmarker = "1";
}
set beresp.grace = 1w;
return (deliver);
}
#
# sub vcl_deliver {
# return (deliver);
# }
sub vcl_deliver {
if (resp.http.magicmarker) {
/* Remove the magic marker */
unset resp.http.magicmarker;
/* By definition we have a fresh object */
set resp.http.age = "0";
}
if (obj.hits > 0) {
set resp.http.X-Cache = obj.hits;
} else {
set resp.http.X-Cache = "MISS";
}
return (deliver);
}
#
# sub vcl_error {
# set obj.http.Content-Type = "text/html; charset=utf-8";
# synthetic {"
# <?xml version="1.0" encoding="utf-8"?>
# <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
# "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
# <html>
# <head>
# <title>"} obj.status " " obj.response {"</title>
# </head>
# <body>
# <h1>Error "} obj.status " " obj.response {"</h1>
# <p>"} obj.response {"</p>
# <h3>Guru Meditation:</h3>
# <p>XID: "} req.xid {"</p>
# <hr>
# <p>Varnish cache server</p>
# </body>
# </html>
# "};
# return (deliver);
# }
slapos/recipe/certificate_authority/__init__.py
View file @ d007050d
......@@ -103,16 +103,27 @@ class Request(Recipe):
key_file = self.options['key-file']
cert_file = self.options['cert-file']
key_content = self.options.get('key-content', None)
cert_content = self.options.get('cert-content', None)
request_needed = True
name = self.options['name']
hash_ = hashlib.sha512(name).hexdigest()
key = os.path.join(self.ca_private, hash_ + self.ca_key_ext)
certificate = os.path.join(self.ca_certs, hash_ + self.ca_crt_ext)
parser = ConfigParser.RawConfigParser()
parser.add_section('certificate')
parser.set('certificate', 'name', name)
parser.set('certificate', 'key_file', key)
parser.set('certificate', 'certificate_file', certificate)
parser.write(open(os.path.join(self.request_directory, hash_), 'w'))
# XXX Ugly hack to quickly provide custom certificate/key to everyone using the recipe
if key_content and cert_content:
open(key, 'w').write(key_content)
open(certificate, 'w').write(cert_content)
request_needed = False
else:
parser = ConfigParser.RawConfigParser()
parser.add_section('certificate')
parser.set('certificate', 'name', name)
parser.set('certificate', 'key_file', key)
parser.set('certificate', 'certificate_file', certificate)
parser.write(open(os.path.join(self.request_directory, hash_), 'w'))
for link in [key_file, cert_file]:
if os.path.islink(link):
......@@ -123,11 +134,14 @@ class Request(Recipe):
os.symlink(key, key_file)
os.symlink(certificate, cert_file)
wrapper = self.createPythonScript(
self.options['wrapper'],
'slapos.recipe.librecipe.execute.execute_wait',
[ [self.options['executable']],
[certificate, key] ],
)
path_list = [key_file, cert_file]
if request_needed:
wrapper = self.createPythonScript(
self.options['wrapper'],
'slapos.recipe.librecipe.execute.execute_wait',
[ [self.options['executable']],
[certificate, key] ],
)
path_list.append(wrapper)
return [key_file, cert_file, wrapper]
return path_list
slapos/recipe/check_url_available/__init__.py
View file @ d007050d
......@@ -37,6 +37,7 @@ class Recipe(GenericBaseRecipe):
'url': self.options['url'],
'shell_path': self.options['dash_path'],
'curl_path': self.options['curl_path'],
'check_secure': self.options.get('check-secure', 0)
}
# XXX-Cedric in this script, curl won't check certificate
......
slapos/recipe/check_url_available/template/check_url.in
View file @ d007050d
......@@ -31,6 +31,13 @@ if [ $CODE -eq 000 ]; then
exit 1
fi
if [ %(check_secure)s -eq 1 ]; then
if [ $CODE -eq 401 ]; then
echo "$URL is protected (returned $CODE)." >&2
exit 0
fi
fi
if ! [ $CODE -eq 200 ]; then
echo "$URL is not available (returned $CODE)." >&2
exit 2
......
slapos/recipe/dcron.py
View file @ d007050d
......@@ -26,7 +26,11 @@
##############################################################################
import os
from slapos.recipe.librecipe import GenericBaseRecipe
if __name__ == '__main__': # Hack to easily run test below.
GenericBaseRecipe = object
else:
from slapos.recipe.librecipe import GenericBaseRecipe
from zc.buildout import UserError
class Recipe(GenericBaseRecipe):
......@@ -51,18 +55,117 @@ class Recipe(GenericBaseRecipe):
return [script]
class Part(GenericBaseRecipe):
def install(self):
try:
periodicity = self.options['frequency']
except KeyError:
periodicity = self.options['time']
try:
periodicity = systemd_to_cron(periodicity)
except Exception:
raise UserError("Invalid systemd calendar spec %r" % periodicity)
cron_d = self.options['cron-entries']
name = self.options['name']
filename = os.path.join(cron_d, name)
with open(filename, 'w') as part:
part.write('%(frequency)s %(command)s\n' % {
'frequency': self.options['frequency'],
'command': self.options['command'],
})
part.write('%s %s\n' % (periodicity, self.options['command']))
return [filename]
day_of_week_dict = dict((name, dow) for dow, name in enumerate(
"sunday monday tuesday wednesday thursday friday saturday".split())
for name in (name, name[:3]))
symbolic_dict = dict(hourly = '0 * * * *',
daily = '0 0 * * *',
monthly = '0 0 1 * *',
weekly = '0 0 * * 0')
def systemd_to_cron(spec):
"""Convert from systemd.time(7) calendar spec to crontab spec"""
try:
return symbolic_dict[spec]
except KeyError:
pass
if not spec.strip():
raise ValueError
spec = spec.split(' ')
try:
dow = ','.join(sorted('-'.join(str(day_of_week_dict[x.lower()])
for x in x.split('-', 1))
for x in spec[0].split(',')
if x))
del spec[0]
except KeyError:
dow = '*'
day = spec.pop(0) if spec else '*-*'
if spec:
time, = spec
elif ':' in day:
time = day
day = '*-*'
else:
time = '0:0'
day = day.split('-')
time = time.split(':')
if (# years not supported
len(day) > 2 and day.pop(0) != '*' or
# some crons ignore day of month if day of week is given, and dcron
# treats day of month in a way that is not compatible with systemd
dow != '*' != day[1] or
# seconds not supported
len(time) > 2 and int(time.pop())):
raise ValueError
month, day = day
hour, minute = time
spec = minute, hour, day, month, dow
for x, (y, z) in zip(spec, ((0, 60), (0, 24), (1, 31), (1, 12))):
if x != '*':
for x in x.split(','):
x = map(int, x.split('/', 1))
x[0] -= y
if x[0] < 0 or len(x) > 1 and x[0] >= x[1] or z <= sum(x):
raise ValueError
return ' '.join(spec)
def test(self):
def _(systemd, cron):
self.assertEqual(systemd_to_cron(systemd), cron)
_("Sat,Mon-Thu,Sun", "0 0 * * 0,1-4,6")
_("mon,sun *-* 2,1:23", "23 2,1 * * 0,1")
_("Wed, 17:48", "48 17 * * 3")
_("Wed-Sat,Tue 10-* 1:2", "2 1 * 10 2,3-6")
_("*-*-7 0:0:0", "0 0 7 * *")
_("10-15", "0 0 15 10 *")
_("monday *-12-* 17:00", "00 17 * 12 1")
_("12,14,13,12:20,10,30", "20,10,30 12,14,13,12 * * *") # TODO: sort
_("*-1/2-1,3 *:30", "30 * 1,3 1/2 *")
_("03-05 08:05", "05 08 05 03 *")
_("08:05:00", "05 08 * * *")
_("05:40", "40 05 * * *")
_("Sat,Sun 12-* 08:05", "05 08 * 12 0,6")
_("Sat,Sun 08:05", "05 08 * * 0,6")
def _(systemd):
self.assertRaises(Exception, systemd_to_cron, systemd)
_("test")
_("")
_("7")
_("121212:1:2")
_("Wed *-1")
_("08:05:40")
_("2003-03-05")
_("0-1"); _("13-1"); _("6/4-1"); _("5/8-1")
_("1-0"); _("1-32"); _("1-4/3"); _("1-14/18")
_("24:0");_("9/9:0"); _("8/16:0")
_("0:60"); _("0:22/22"); _("0:15/45")
if __name__ == '__main__':
import unittest
unittest.TextTestRunner().run(type('', (unittest.TestCase,), {
'runTest': test})())
slapos/recipe/erp5_bootstrap/template/erp5_bootstrap.in
View file @ d007050d
......@@ -12,7 +12,8 @@ erp5_catalog_storage = 'erp5_mysql_innodb_catalog'
mysql_url = "%(sql_connection_string)s"
header_dict = {'Authorization': 'Basic %%s' %% \
base64.encodestring('%%s:%%s' %% (user, password)).strip()}
base64.encodestring('%%s:%%s' %% (user, password)).strip(),
'Referer':'http://%%s/manage_addProduct/ERP5/addERP5Site' %% host}
zope_connection = httplib.HTTPConnection(host)
# Check if an ERP5 site is already created, as ERP5 does support having
......
slapos/recipe/erp5_update/erp5.py
View file @ d007050d
......@@ -50,6 +50,7 @@ class ERP5Updater(object):
base64string = base64.encodestring(authentication_string).strip()
self.header_dict['Authorization'] = 'Basic %s' % base64string
self.header_dict['Referer'] = 'http://%s/manage_addProduct/ERP5/addERP5Site' % host
self.host = host
self.site_id = site_id
......
slapos/recipe/generatepassword.py
View file @ d007050d
......@@ -26,27 +26,69 @@
#
##############################################################################
import binascii
import errno
import os
import random
import string
from slapos.recipe.librecipe import GenericBaseRecipe
def generatePassword(length):
return ''.join(random.SystemRandom().sample(string.ascii_lowercase, length))
class Recipe(GenericBaseRecipe):
class Recipe(object):
"""Generate a password that is only composed of lowercase letters
This recipe only makes sure that ${:passwd} does not end up in `.installed`
file, which is world-readable by default. So be careful not to spread it
throughout the buildout configuration by referencing it directly: see
recipes like slapos.recipe.template:jinja2 to safely process the password.
Options:
- bytes: password length (default: 8 characters)
- storage-path: plain-text persistent storage for password,
that can only be accessed by the user
(default: ${buildout:parts-directory}/${:_buildout_section_name_})
"""
def __init__(self, buildout, name, options):
if os.path.exists(options['storage-path']):
open_file = open(options['storage-path'], 'r')
options['passwd'] = open_file.read()
open_file.close()
options_get = options.get
try:
self.storage_path = options['storage-path']
except KeyError:
self.storage_path = options['storage-path'] = os.path.join(
buildout['buildout']['parts-directory'], name)
try:
with open(self.storage_path) as f:
passwd = f.read()
except IOError, e:
if e.errno != errno.ENOENT:
raise
passwd = None
if not passwd:
passwd = self.generatePassword(int(options_get('bytes', '8')))
self.update = self.install
self.passwd = passwd
# Password must not go into .installed file, for 2 reasons:
# security of course but also to prevent buildout to always reinstall.
options.get = lambda option, *args, **kw: passwd \
if option == 'passwd' else options_get(option, *args, **kw)
if options.get('passwd', '') == '':
options['passwd'] = binascii.hexlify(os.urandom(
int(options.get('bytes', '24'))))
return GenericBaseRecipe.__init__(self, buildout, name, options)
generatePassword = staticmethod(generatePassword)
def install(self):
with open(self.options['storage-path'], 'w') as fout:
fout.write(self.options['passwd'])
return [self.options['storage-path']]
if self.storage_path:
try:
os.unlink(self.storage_path)
except OSError, e:
if e.errno != errno.ENOENT:
raise
fd = os.open(self.storage_path,
os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0600)
try:
os.write(fd, self.passwd)
finally:
os.close(fd)
return self.storage_path
def update(self):
return ()
slapos/recipe/kvm/__init__.py
View file @ d007050d
......@@ -41,39 +41,44 @@ class Recipe(GenericBaseRecipe):
'"virtio" value.'
self.options['disk-type'] = 'virtio'
config = dict(
tap_interface=self.options['tap'],
vnc_ip=self.options['vnc-ip'],
vnc_port=self.options['vnc-port'],
nbd_ip=self.options['nbd-host'],
nbd_port=self.options['nbd-port'],
nbd2_ip=self.options.get('nbd2-host', ''),
nbd2_port=self.options.get('nbd2-port', 1024),
disk_path=self.options['disk-path'],
disk_size=self.options['disk-size'],
disk_type=self.options['disk-type'],
mac_address=self.options['mac-address'],
smp_count=self.options['smp-count'],
ram_size=self.options['ram-size'],
socket_path=self.options['socket-path'],
pid_file_path=self.options['pid-path'],
python_path=sys.executable,
shell_path=self.options['shell-path'],
qemu_path=self.options['qemu-path'],
qemu_img_path=self.options['qemu-img-path'],
vnc_passwd=self.options['passwd']
)
self.options['python-path'] = sys.executable
path_list = []
if not self.isTrueValue(self.options.get('use-tap')):
# XXX This could be done using Jinja.
for item in self.options['nat-rules'].split():
ports = item.split(':')
if len(ports) == 1:
tunnel_port = int(ports[0]) + 10000
else:
tunnel_port = int(ports[1])
tunnel_path = self.createExecutable(
'%s-%s' % (self.options['6tunnel-wrapper-path'], tunnel_port),
self.substituteTemplate(
self.getTemplateFilename('6to4.in'),
{
'ipv6': self.options['ipv6'],
'ipv6_port': tunnel_port,
'ipv4': self.options['ipv4'],
'ipv4_port': tunnel_port,
'shell_path': self.options['shell-path'],
'6tunnel_path': self.options['6tunnel-path'],
},
),
)
path_list.append(tunnel_path)
# Runners
runner_path = self.createExecutable(
self.options['runner-path'],
self.substituteTemplate(self.getTemplateFilename('kvm_run.in'),
config))
self.options['runner-path'],
self.substituteTemplate(self.getTemplateFilename('kvm_run.in'),
self.options))
path_list.append(runner_path)
controller_path = self.createExecutable(
self.options['controller-path'],
self.substituteTemplate(self.getTemplateFilename('kvm_controller_run.in'),
config))
self.options))
return [runner_path, controller_path]
return path_list
slapos/recipe/kvm/template/6to4.in 0 → 100644
View file @ d007050d
#!%(shell_path)s
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
exec %(6tunnel_path)s -6 -4 -d -l %(ipv6)s %(ipv6_port)s %(ipv4)s %(ipv4_port)s
slapos/recipe/kvm/template/kvm_controller_run.in
View file @ d007050d
#!%(python_path)s
#!%(python-path)s
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
......@@ -6,12 +6,17 @@
import socket
import time
# XXX: to be factored with slapos.toolbox qemu qmp wrapper.
socket_path = '%(socket-path)s'
vnc_password = '%(vnc-passwd)s'
# Connect to KVM qmp socket
so = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
connected = False
while not connected:
try:
so.connect('%(socket_path)s')
so.connect(socket_path)
except socket.error:
time.sleep(1)
else:
......@@ -25,7 +30,7 @@ data = so.recv(1024)
# Set VNC password
so.send('{ "execute": "change", ' \
'"arguments": { "device": "vnc", "target": "password", ' \
' "arg": "%(vnc_passwd)s" } }')
' "arg": "' + vnc_password + '" } }')
data = so.recv(1024)
# Finish
......
slapos/recipe/kvm/template/kvm_run.in
View file @ d007050d
#!%(python_path)s
#!%(python-path)s
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
# Echo client program
import hashlib
import os
import socket
import subprocess
import urllib
# XXX: give all of this through parameter, don't use this as template, but as module
qemu_img_path = '%(qemu-img-path)s'
qemu_path = '%(qemu-path)s'
disk_size = '%(disk-size)s'
disk_type = '%(disk-type)s'
socket_path = '%(socket-path)s'
nbd_list = (('%(nbd-host)s', %(nbd-port)s), ('%(nbd2-host)s', %(nbd2-port)s))
default_disk_image = '%(default-disk-image)s'
disk_path = '%(disk-path)s'
virtual_hard_drive_url = '%(virtual-hard-drive-url)s'.strip()
virtual_hard_drive_md5sum = '%(virtual-hard-drive-md5sum)s'.strip()
nat_rules = '%(nat-rules)s'.strip()
use_tap = '%(use-tap)s'
tap_interface = '%(tap-interface)s'
listen_ip = '%(ipv4)s'
mac_address = '%(mac-address)s'
smp_count = '%(smp-count)s'
ram_size = '%(ram-size)s'
pid_file_path = '%(pid-file-path)s'
def md5Checksum(file_path):
with open(file_path, 'rb') as fh:
m = hashlib.md5()
while True:
data = fh.read(8192)
if not data:
break
m.update(data)
return m.hexdigest()
def getSocketStatus(host, port):
s = None
......@@ -26,27 +57,62 @@ def getSocketStatus(host, port):
break
return s
# create disk if doesn't exist
disk_path = '%(disk_path)s'
# Download existing hard drive if needed at first boot
if not os.path.exists(disk_path) and virtual_hard_drive_url != '':
print('Downloading virtual hard drive...')
try:
urllib.urlretrieve(virtual_hard_drive_url, disk_path)
except:
os.remove(disk_path)
raise
md5sum = virtual_hard_drive_md5sum.strip()
if md5sum:
print('Checking MD5 checksum...')
local_md5sum = md5Checksum(disk_path)
if local_md5sum != md5sum:
os.remove(disk_path)
raise Exception('MD5 mismatch. MD5 of local file is %%s, Specified MD5 is %%s.' %% (
local_md5sum, md5sum))
print('MD5sum check passed.')
else:
print('Warning: not checksum specified.')
# Create disk if doesn't exist
# XXX: move to Buildout profile
if not os.path.exists(disk_path):
subprocess.Popen(['%(qemu_img_path)s', 'create' ,'-f', 'qcow2',
disk_path, '%(disk_size)sG'])
kvm_argument_list = ['%(qemu_path)s',
'-enable-kvm', '-net', 'nic,macaddr=%(mac_address)s',
'-net', 'tap,ifname=%(tap_interface)s,script=no,downscript=no',
'-smp', '%(smp_count)s',
'-m', '%(ram_size)s',
'-drive', 'file=%(disk_path)s,if=%(disk_type)s',
'-vnc', '%(vnc_ip)s:1,ipv4,password',
print('Creating virtual hard drive...')
subprocess.Popen([qemu_img_path, 'create' ,'-f', 'qcow2',
disk_path, '%%sG' %% disk_size])
print('Done.')
# Generate network parameters
# XXX: use_tap should be a boolean
if use_tap == 'True':
qemu_network_parameter = 'tap,ifname=%%s,script=no,downscript=no' %% tap_interface
else:
qemu_network_parameter = 'user'
for item in nat_rules.split():
ports = item.split(':')
if len(ports) == 1:
ports.append(int(ports[0]) + 10000)
qemu_network_parameter += ',hostfwd=tcp:%%s:%%s-:%%s' %% (listen_ip, ports[1], ports[0])
kvm_argument_list = [qemu_path,
'-enable-kvm', '-net', 'nic,macaddr=%%s' %% mac_address,
'-net', qemu_network_parameter,
'-smp', smp_count,
'-m', ram_size,
'-drive', 'file=%%s,if=%%s' %% (disk_path, disk_type),
'-vnc', '%%s:1,ipv4,password' %% listen_ip,
'-boot', 'menu=on',
'-qmp', 'unix:%(socket_path)s,server',
'-pidfile', '%(pid_file_path)s',
'-qmp', 'unix:%%s,server' %% socket_path,
'-pidfile', pid_file_path,
]
# Try to connect to NBD server (and second nbd if defined)
for nbd_ip, nbd_port in (
('%(nbd_ip)s', %(nbd_port)s), ('%(nbd2_ip)s', %(nbd2_port)s)):
# Try to connect to NBD server (and second nbd if defined).
# If not available, don't even specify it in qemu command line parameters.
# Reason: if qemu starts with unavailable NBD drive, it will just crash.
for nbd_ip, nbd_port in nbd_list:
if nbd_ip and nbd_port:
s = getSocketStatus(nbd_ip, nbd_port)
if s is None:
......@@ -57,5 +123,10 @@ for nbd_ip, nbd_port in (
kvm_argument_list.extend([
'-drive',
'file=nbd:[%%s]:%%s,media=cdrom' %% (nbd_ip, nbd_port)])
# If no NBD is specified/available: use internal disk image
else:
kvm_argument_list.extend([
'-drive', 'file=%%s,media=cdrom' %% default_disk_image
])
os.execv('%(qemu_path)s', kvm_argument_list)
os.execv(qemu_path, kvm_argument_list)
slapos/recipe/librecipe/generic.py
View file @ d007050d
......@@ -33,6 +33,7 @@ import sys
import inspect
import re
import shutil
from textwrap import dedent
import urllib
import urlparse
......@@ -129,10 +130,14 @@ class GenericBaseRecipe(object):
return script
def createWrapper(self, name, command, parameters, comments=[],
parameters_extra=False, environment=None):
parameters_extra=False, environment=None,
pidfile=None
):
"""
Creates a very simple (one command) shell script for process replacement.
Takes care of quoting.
if pidfile parameter is specified, then it will make the wrapper a singleton,
accepting to run only if no other instance is running.
"""
lines = [ '#!/bin/sh' ]
......@@ -144,6 +149,21 @@ class GenericBaseRecipe(object):
for key in environment:
lines.append('export %s=%s' % (key, environment[key]))
if pidfile:
lines.append(dedent("""\
# Check for other instances
pidfile=%s
if [ -e $pidfile ]; then
pid=$(cat $pidfile)
if [[ ! -z $(ps -p "$pid" | grep $(basename %s)) ]]; then
echo "Already running with pid $pid."
exit 1
else
rm $pidfile
fi
fi
echo $$ > $pidfile""" % (pidfile, command)))
lines.append('exec %s' % shlex.quote(command))
for param in parameters:
......@@ -183,17 +203,13 @@ class GenericBaseRecipe(object):
'template/%s' % template_name)
def generatePassword(self, len_=32):
"""
The purpose of this method is to generate a password which doesn't change
from one execution to the next, so the generated password doesn't change
on each slapgrid-cp execution.
Currently, it returns a hardcoded password because no decision has been
taken on where a generated password should be kept (so it is generated
once only).
"""
# TODO: implement a real password generator which remember the last
# call.
# TODO: Consider having generate.password recipe inherit this class,
# so that it can be easily inheritable.
# In the long-term, it's probably better that passwords are provided
# by software requesters, to avoid keeping unhashed secrets in
# partitions when possible.
self.logger.warning("GenericBaseRecipe.generatePassword is deprecated."
" Use generate.password recipe instead.")
return "insecure"
def isTrueValue(self, value):
......@@ -247,7 +263,8 @@ class GenericBaseRecipe(object):
destination = self.location
if os.path.exists(destination):
# leftovers from a previous failed attempt, removing it.
log.warning('Removing already existing directory %s' % destination)
self.logger.warning('Removing already existing directory %s',
destination)
shutil.rmtree(destination)
os.mkdir(destination)
......
slapos/recipe/notifier.py
View file @ d007050d
......@@ -50,6 +50,9 @@ class Recipe(GenericBaseRecipe):
class Callback(GenericBaseRecipe):
def createCallback(self, notification_id, callback):
# XXX: hashing the name here and in
# slapos.toolbox/slapos/pubsub/__init__.py is completely messed up and
# prevent any debug.
callback_id = sha512(notification_id).hexdigest()
filepath = os.path.join(self.options['callbacks'], callback_id)
......@@ -64,7 +67,7 @@ class Callback(GenericBaseRecipe):
class Notify(GenericBaseRecipe):
def createNotifier(self, notifier_binary, wrapper, executable,
log, title, notification_url, feed_url):
log, title, notification_url, feed_url, pidfile=None):
if not os.path.exists(log):
# Just a touch
......@@ -82,6 +85,7 @@ class Notify(GenericBaseRecipe):
return self.createWrapper(name=wrapper,
command=notifier_binary,
parameters=parameters,
pidfile=pidfile,
comments=[
'',
'Call an executable and send notification(s).',
......@@ -101,6 +105,7 @@ class Notify(GenericBaseRecipe):
executable=options['executable'],
log=log,
title=options['title'],
pidfile=options['pidfile'],
notification_url=options['notify'],
feed_url=feed_url)
return [script]
slapos/recipe/pbs.py
View file @ d007050d
......@@ -25,43 +25,24 @@
#
##############################################################################
import hashlib
import json
import os
import signal
import subprocess
import sys
import textwrap
import urlparse
from slapos.recipe.librecipe import GenericSlapRecipe
from slapos.recipe.dropbear import KnownHostsFile
from slapos.recipe.notifier import Notify
from slapos.recipe.notifier import Callback
from slapos import slap as slapmodule
def promise(args):
def failed_ssh():
sys.stderr.write("SSH Connection failed\n")
partition = slap.registerComputerPartition(args['computer_id'],
args['partition_id'])
partition.bang("SSH Connection failed. rdiff-backup is unusable.")
def sigterm_handler(signum, frame):
failed_ssh()
signal.signal(signal.SIGTERM, sigterm_handler)
slap = slapmodule.slap()
slap.initializeConnection(args['server_url'],
key_file=args.get('key_file'),
cert_file=args.get('cert_file'))
ssh = subprocess.Popen([args['ssh_client'], '%(user)s@%(host)s/%(port)s' % args],
stdin=subprocess.PIPE,
stdout=open(os.devnull, 'w'),
stderr=open(os.devnull, 'w'))
ssh = subprocess.Popen(
[args['ssh_client'], '%(user)s@%(host)s/%(port)s' % args],
stdin=subprocess.PIPE, stdout=None, stderr=None
)
# Rdiff Backup protocol quit command
quitcommand = 'q' + chr(255) + chr(0) * 7
......@@ -74,7 +55,7 @@ def promise(args):
if ssh.poll() is None:
return 1
if ssh.returncode != 0:
failed_ssh()
sys.stderr.write("SSH Connection failed\n")
return ssh.returncode
......@@ -87,16 +68,19 @@ class Recipe(GenericSlapRecipe, Notify, Callback):
url = entry.get('url')
if not url:
raise ValueError('Missing URL parameter for PBS recipe')
parsed_url = urlparse.urlparse(url)
# We assume that thanks to sha512 there's no collisions
url_hash = hashlib.sha512(url).hexdigest()
name_hash = hashlib.sha512(entry['name']).hexdigest()
slave_type = entry['type']
if not slave_type in ['pull', 'push']:
raise ValueError('type parameter must be either pull or push.')
promise_path = os.path.join(self.options['promises-directory'],
url_hash)
parsed_url = urlparse.urlparse(url)
promise_dict = self.promise_base_dict.copy()
promise_dict.update(user=parsed_url.username,
slave_id = entry['notification-id']
print 'Processing PBS slave %s with type %s' % (slave_id, slave_type)
promise_path = os.path.join(self.options['promises-directory'], slave_id)
promise_dict = dict(ssh_client=self.options['sshclient-binary'],
user=parsed_url.username,
host=parsed_url.hostname,
port=parsed_url.port)
promise = self.createPythonScript(promise_path,
......@@ -104,72 +88,109 @@ class Recipe(GenericSlapRecipe, Notify, Callback):
promise_dict)
path_list.append(promise)
host = parsed_url.hostname
known_hosts_file[host] = entry['server-key']
# XXX use -y because the host might not yet be in the
# trusted hosts file until the next time slapgrid is run.
remote_schema = '%(ssh)s -y -p %%s %(user)s@%(host)s' % \
{
'ssh': self.options['sshclient-binary'],
'user': parsed_url.username,
'host': parsed_url.hostname,
}
notifier_wrapper_path = os.path.join(self.options['wrappers-directory'], slave_id)
rdiff_wrapper_path = notifier_wrapper_path + '_raw'
parameters = ['--remote-schema', remote_schema]
# Create the rdiff-backup wrapper
# It is useful to separate it from the notifier so that we can run it
# Manually.
rdiffbackup_parameter_list = []
# XXX use -y because the host might not yet be in the
# trusted hosts file until the next time slapgrid is run.
rdiffbackup_remote_schema = '%(ssh)s -y -p %%s %(user)s@%(host)s' % {
'ssh': self.options['sshclient-binary'],
'user': parsed_url.username,
'host': parsed_url.hostname,
}
remote_directory = '%(port)s::%(path)s' % {'port': parsed_url.port,
'path': parsed_url.path}
local_directory = self.createDirectory(self.options['directory'],
name_hash)
if entry['type'] == 'push':
parameters.extend(['--restore-as-of', 'now'])
parameters.append('--force')
parameters.extend([local_directory, remote_directory])
comments = ['','Push data to a PBS *-import instance.','']
else:
parameters.extend([remote_directory, local_directory])
comments = ['','Pull data from a PBS *-export instance.','']
wrapper_basepath = os.path.join(self.options['wrappers-directory'],
url_hash)
if 'notify' in entry:
wrapper_path = wrapper_basepath + '_raw'
else:
wrapper_path = wrapper_basepath
wrapper = self.createWrapper(name=wrapper_path,
command=self.options['rdiffbackup-binary'],
parameters=parameters,
comments = comments)
path_list.append(wrapper)
if 'notify' in entry:
feed_url = '%s/get/%s' % (self.options['notifier-url'],
entry['notification-id'])
wrapper = self.createNotifier(notifier_binary=self.options['notifier-binary'],
wrapper=wrapper_basepath,
executable=wrapper_path,
log=os.path.join(self.options['feeds'], entry['notification-id']),
title=entry.get('title', 'Untitled'),
notification_url=entry['notify'],
feed_url=feed_url,
)
path_list.append(wrapper)
#self.setConnectionDict(dict(feed_url=feed_url), entry['slave_reference'])
local_directory = self.createDirectory(self.options['directory'], entry['name'])
if slave_type == 'push':
# Create a simple rdiff-backup wrapper that will push
rdiffbackup_parameter_list.extend(['--remote-schema', rdiffbackup_remote_schema])
rdiffbackup_parameter_list.extend(['--restore-as-of', 'now'])
rdiffbackup_parameter_list.append('--force')
rdiffbackup_parameter_list.append(local_directory)
rdiffbackup_parameter_list.append(remote_directory)
comments = ['', 'Push data to a PBS *-import instance.', '']
rdiff_wrapper = self.createWrapper(
name=rdiff_wrapper_path,
command=self.options['rdiffbackup-binary'],
parameters=rdiffbackup_parameter_list,
comments=comments,
pidfile=os.path.join(self.options['run-directory'], '%s_raw.pid' % slave_id),
)
elif slave_type == 'pull':
# Wrap rdiff-backup call into a script that checks consistency of backup
# We need to manually escape the remote schema
rdiffbackup_parameter_list.extend(['--remote-schema', '"%s"' % rdiffbackup_remote_schema])
rdiffbackup_parameter_list.append(remote_directory)
rdiffbackup_parameter_list.append(local_directory)
comments = ['', 'Pull data from a PBS *-export instance.', '']
rdiff_wrapper_template = textwrap.dedent("""\
#!/bin/sh
# %(comment)s
RDIFF_BACKUP="%(rdiffbackup_binary)s"
$RDIFF_BACKUP %(rdiffbackup_parameter)s
if [ ! $? -eq 0 ]; then
# Check the backup, go to the last consistent backup, so that next
# run will be okay.
echo "Checking backup directory..."
$RDIFF_BACKUP --check-destination-dir %(local_directory)s
if [ ! $? -eq 0 ]; then
# Here, two possiblities:
# * The first backup failed. It is safe to remove it since there is nothing valuable there.
# * The backup has been complete, but is now in a really weird state. Not safe to remove it.
echo "Impossible to check backup: we move it to a safe place."
# XXX: bang
mv %(local_directory)s %(local_directory)s.$(date +%%s)
fi
else
# Everything's okay, cleaning up...
$RDIFF_BACKUP --remove-older-than %(remove_backup_older_than)s --force %(local_directory)s
fi
""")
rdiff_wrapper_content = rdiff_wrapper_template % {
'comment': comments,
'rdiffbackup_binary': self.options['rdiffbackup-binary'],
'local_directory': local_directory,
'rdiffbackup_parameter': ' \\\n '.join(rdiffbackup_parameter_list),
# XXX: only 10 increments is not enough by default.
'remove_backup_older_than': entry.get('remove-backup-older-than', '3B')
}
rdiff_wrapper = self.createFile(
name=rdiff_wrapper_path,
content=rdiff_wrapper_content,
mode=0700
)
path_list.append(rdiff_wrapper)
# Create notifier wrapper
notifier_wrapper = self.createNotifier(
notifier_binary=self.options['notifier-binary'],
wrapper=notifier_wrapper_path,
executable=rdiff_wrapper,
log=os.path.join(self.options['feeds'], entry['notification-id']),
title=entry.get('title', slave_id),
notification_url=entry['notify'],
feed_url='%s/get/%s' % (self.options['notifier-url'], entry['notification-id']),
pidfile=os.path.join(self.options['run-directory'], '%s.pid' % slave_id)
)
path_list.append(notifier_wrapper)
if 'on-notification' in entry:
path_list.append(self.createCallback(str(entry['on-notification']),
wrapper))
notifier_wrapper))
else:
cron_entry = os.path.join(self.options['cron-entries'], url_hash)
cron_entry = os.path.join(self.options['cron-entries'], slave_id)
with open(cron_entry, 'w') as cron_entry_file:
cron_entry_file.write('%s %s' % (entry['frequency'], wrapper))
cron_entry_file.write('%s %s' % (entry['frequency'], notifier_wrapper))
path_list.append(cron_entry)
return path_list
......@@ -181,20 +202,9 @@ class Recipe(GenericSlapRecipe, Notify, Callback):
if self.optionIsTrue('client', True):
self.logger.info("Client mode")
slap_connection = self.buildout['slap-connection']
self.promise_base_dict = {
'server_url': slap_connection['server-url'],
'computer_id': slap_connection['computer-id'],
'cert_file': slap_connection.get('cert-file'),
'key_file': slap_connection.get('key-file'),
'partition_id': slap_connection['partition-id'],
'ssh_client': self.options['sshclient-binary'],
}
slaves = json.loads(self.options['slave-instance-list'])
known_hosts = KnownHostsFile(self.options['known-hosts'])
with known_hosts:
# XXX this API could be cleaner
for slave in slaves:
path_list.extend(self.add_slave(slave, known_hosts))
else:
......@@ -209,4 +219,3 @@ class Recipe(GenericSlapRecipe, Notify, Callback):
path_list.append(wrapper)
return path_list
slapos/recipe/pwgen.py → slapos/recipe/readline.py
View file @ d007050d
# vim: set et sts=2:
##############################################################################
#
# Copyright (c) 2010 Vifib SARL and Contributors. All Rights Reserved.
# Copyright (c) 2013 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
......@@ -24,37 +25,39 @@
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
import subprocess
import os
from slapos.recipe.librecipe import GenericBaseRecipe
import errno
class Recipe(GenericBaseRecipe):
class Recipe(object):
"""Read the first line of a file.
def _options(self, options):
if not os.path.exists(self.options['file']):
password = subprocess.check_output([self.options['pwgen-binary'], '-1']).strip()
with open(self.options['file'], 'w') as password_file:
password_file.write(password)
else:
with open(self.options['file'], 'r') as password_file:
password = password_file.read()
options['password'] = password
As the result has to be provided as an options, it is mandatory that the
buildout profile fills the file content (if needed) before trying to read it.
def install(self):
os.chmod(self.options['file'], 0600)
return []
Options:
- storage-path: file to read
class StablePasswordGeneratorRecipe(GenericBaseRecipe):
Result set in options:
- readline: first line of the file
"""
The purpose of this class is to generate a password which doesn't change
from one execution to the next (hence "stable"), so the generated password
doesn't change on each slapgrid-cp execution.
See GenericBaseRecipe.generatePassword .
"""
def __init__(self, buildout, name, options):
storage_path = options['storage-path']
try:
with open(storage_path) as f:
readline = f.readline()
except IOError, e:
if e.errno != errno.ENOENT:
raise
readline = None
def _options(self, options):
options['password'] = self.generatePassword()
self.readline = readline
options['readline'] = readline
def install(self):
if self.readline is None:
raise ValueError('Unable to read the file content.')
return ()
update = install = lambda self: []
def update(self):
return ()
slapos/recipe/request.py
View file @ d007050d
......@@ -88,8 +88,15 @@ class Recipe(object):
installation of request section will fail.
Possible names depend on requested partition's software type.
state (optional)
Requested state, default value is the state of the requester.
Output:
See "return" input key.
"instance-state"
The current state of the instance.
"requested-state"
The requested state of the instance.
"""
failed = None
......@@ -112,6 +119,11 @@ class Recipe(object):
))
slave = options.get('slave', 'false').lower() in \
librecipe.GenericBaseRecipe.TRUE_VALUES
# By default XXXX Way of doing it is ugly and dangerous
requested_state = options.get('state', buildout['slap-connection'].get('requested','started'))
options['requested-state'] = requested_state
slap = slapmodule.slap()
slap.initializeConnection(
options['server-url'],
......@@ -125,21 +137,25 @@ class Recipe(object):
self._raise_request_exception = None
self._raise_request_exception_formatted = None
self.instance = None
# Try to do the request and fetch parameter dict...
try:
self.instance = request(software_url, software_type,
name, partition_parameter_kw=partition_parameter_kw,
filter_kw=filter_kw, shared=slave)
filter_kw=filter_kw, shared=slave, state=requested_state)
return_parameter_dict = self._getReturnParameterDict(self.instance,
return_parameters)
# Fetch the instance-guid and the instance-state
# Note: SlapOS Master does not support it for slave instances
if not slave:
try:
options['instance-guid'] = self.instance.getInstanceGuid()
# XXX: deprecated, to be removed
options['instance_guid'] = self.instance.getInstanceGuid()
options['instance-state'] = self.instance.getState()
except (slapmodule.ResourceNotReady, AttributeError):
# Backward compatibility. Old SlapOS master and core don't know this.
self.logger.warning("Impossible to fetch instance GUID.")
self.logger.warning("Impossible to fetch instance GUID nor state.")
except (slapmodule.NotFoundError, slapmodule.ServerError, slapmodule.ResourceNotReady) as exc:
self._raise_request_exception = exc
self._raise_request_exception_formatted = traceback.format_exc()
......
slapos/recipe/slapconfiguration.py
View file @ d007050d
......@@ -78,6 +78,8 @@ class Recipe(object):
Partition parameter whose name cannot be represented unambiguously in
buildout syntax are ignored. They cannot be accessed from buildout syntax
anyway, and are available through "configuration" output key.
instance-state
The instance state.
"""
# XXX: used to detect if a configuration key is a valid section key. This
......@@ -91,10 +93,12 @@ class Recipe(object):
options.get('key'),
options.get('cert'),
)
parameter_dict = slap.registerComputerPartition(
computer_partition = slap.registerComputerPartition(
options['computer'],
options['partition'],
).getInstanceParameterDict()
)
parameter_dict = computer_partition.getInstanceParameterDict()
options['instance-state'] = computer_partition.getState()
# XXX: those are not partition parameters, strictly speaking.
# Make them available as individual section keys.
for his_key in (
......@@ -129,9 +133,9 @@ class Recipe(object):
# also export single ip values for those recipes that don't support sets.
if ipv4_set:
options['ipv4-random'] = list(ipv4_set)[0]
options['ipv4-random'] = list(ipv4_set)[0].encode('UTF-8')
if ipv6_set:
options['ipv6-random'] = list(ipv6_set)[0]
options['ipv6-random'] = list(ipv6_set)[0].encode('UTF-8')
options['tap'] = tap_set
parameter_dict = self._expandParameterDict(options, parameter_dict)
......
slapos/recipe/slaprunner/__init__.py
View file @ d007050d
......@@ -40,8 +40,8 @@ class Recipe(GenericBaseRecipe):
self.partition_amount = options['partition-amount'].strip()
self.cloud9_url = options.get('cloud9-url', '').strip()
self.log_file = os.path.join(options['log_dir'].strip(), 'slaprunner.log')
# Set slaprunner access URL
options['access-url'] = 'http://[%s]:%s' % (self.ipv6, self.runner_port)
# Set slaprunner access URL, CLN Beware ipv6 access is made throught nginx
options['access-url'] = 'https://[%s]:%s' % (self.ipv6, self.runner_port)
def install(self):
path_list = []
......@@ -49,7 +49,7 @@ class Recipe(GenericBaseRecipe):
configuration = dict(
software_root=self.software_directory,
instance_root=self.instance_directory,
master_url='http://%s:%s/' % (self.ipv4, self.proxy_port),
master_url='http://%s:%s' % (self.ipv4, self.proxy_port),
computer_id='slaprunner',
partition_amount=self.partition_amount,
slapgrid_sr=self.options['slapgrid_sr'],
......@@ -62,7 +62,7 @@ class Recipe(GenericBaseRecipe):
etc_dir=self.options['etc_dir'],
run_dir=self.options['run_dir'],
log_dir=self.options['log_dir'],
runner_host=self.ipv6,
runner_host=self.ipv4,
runner_port=self.runner_port,
ipv4_address=self.ipv4,
ipv6_address=self.ipv6,
......@@ -132,7 +132,7 @@ class Test(GenericBaseRecipe):
etc_dir=self.options['etc_dir'],
run_dir=self.options['etc_dir'],
log_dir=self.workdir,
runner_host=self.ipv6,
runner_host=self.ipv4,
runner_port=self.runner_port,
ipv4_address=self.ipv4,
ipv6_address=self.ipv6,
......
slapos/recipe/slaprunner/backup.py
View file @ d007050d
......@@ -71,7 +71,7 @@ class ExportRecipe(GenericBaseRecipe):
done
}
sync_element %(srv-directory)s/runner %(backup-directory)s/runner/ instance project proxy.db softwareLink
sync_element %(etc-directory)s %(backup-directory)s/etc/ .rcode .project .users ssh
sync_element %(etc-directory)s %(backup-directory)s/etc/ .rcode .project .users .htpasswd ssh
if [ -d %(backup-directory)s/runner/software ]; then
rm %(backup-directory)s/runner/software/*
fi
......@@ -120,12 +120,12 @@ class ImportRecipe(GenericBaseRecipe):
done
}
restore_element %(backup-directory)s/runner/ %(srv-directory)s/runner instance project proxy.db softwareLink
restore_element %(backup-directory)s/etc/ %(etc-directory)s .rcode .project .users ssh
restore_element %(backup-directory)s/etc/ %(etc-directory)s .rcode .project .users .htpasswd ssh
ifs=$IFS IFS=';'
read user pass remaining < %(etc-directory)s/.users
IFS=$ifs
%(curl-binary)s -vg6L -F clogin="$user" -F cpwd="$pass" --dump-header login_cookie %(backend-url)s/doLogin;
%(curl-binary)s -vg6LX POST --cookie login_cookie --max-time 5 %(backend-url)s/runSoftwareProfile;
%(curl-binary)s --insecure -vg6L -F clogin="$user" -F cpwd="$pass" --dump-header login_cookie %(backend-url)s/doLogin;
%(curl-binary)s --insecure -vg6LX POST --cookie login_cookie --max-time 5 %(backend-url)s/runSoftwareProfile;
rm -f login_cookie
""" % self.options)
self.createExecutable(wrapper, content=content)
......
slapos/recipe/softwaretype.py
View file @ d007050d
......@@ -87,8 +87,6 @@ class Recipe:
computer_partition_id)
self.parameter_dict = self.computer_partition.getInstanceParameterDict()
software_type = self.parameter_dict['slap_software_type']
self.logger.info('Deploying instance with software type %s' % \
software_type)
# Raise if request software_type does not exist ...
if software_type not in self.options:
......
slapos/recipe/squid/__init__.py 0 → 100644
View file @ d007050d
##############################################################################
#
# Copyright (c) 2012 Vifib SARL and Contributors. All Rights Reserved.
#
# WARNING: This program as such is intended to be used by professional
# programmers who take the whole responsibility of assessing all potential
# consequences resulting from its eventual inadequacies and bugs
# End users who are looking for a ready-to-use solution with commercial
# guarantees and support are strongly adviced to contract a Free Software
# Service Company
#
# This program is Free Software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 3
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
##############################################################################
from slapos.recipe.librecipe import GenericBaseRecipe
class Recipe(GenericBaseRecipe):
"""
squid instance configuration.
wrapper-path -- location of the init script to generate
prepare-path -- location of the directory creation script to generate
binary-path -- location of the squid command
conf-path -- location of the configuration file
cache-path -- location of the cache directory
XXXX No good, specific...
open_port -- entrance port to the host and allowed to use cache
ip -- ip of the squid server
port -- port of the squid server
backend-ip -- ip of the service to cache
backend-port -- port of the service to cache
access-log-path -- location of the access log
cache-log-path -- location of the cache log
pid-filename-path -- location of the pid filename
"""
def install(self):
config = dict(
ip=self.options['ip'],
port=self.options['port'],
backend_ip=self.options['backend-ip'],
backend_port=self.options['backend-port'],
cache_path=self.options['cache-path'],
access_log_path=self.options['access-log-path'],
cache_log_path=self.options['cache-log-path'],
pid_filename_path=self.options['pid-filename-path'],
open_port=self.options['open-port'],
)
template_filename = self.getTemplateFilename('squid.conf.in')
configuration_path = self.createFile(
self.options['conf-path'],
self.substituteTemplate(template_filename, config))
# Prepare directories
prepare_path = self.createPythonScript(
self.options['prepare-path'],
'slapos.recipe.librecipe.execute.execute',
arguments=[self.options['binary-path'].strip(),
'-z',
'-f', configuration_path,
],)
# Create running wrapper
wrapper_path = self.createPythonScript(
self.options['wrapper-path'],
'slapos.recipe.librecipe.execute.execute',
arguments=[self.options['binary-path'].strip(),
'-N',
'-f', configuration_path,
],)
return [configuration_path, wrapper_path, prepare_path]
slapos/recipe/squid/template/squid.conf.in 0 → 100644
View file @ d007050d
refresh_pattern . 0 20%% 4320 max-stale=604800
# Dissallow cachemgr access
http_access deny manager
# Squid service configuration
http_port %(ip)s:%(port)s accel defaultsite=%(ip)s
cache_peer %(backend_ip)s parent %(backend_port)s 0 no-query originserver name=backend
acl our_sites port %(open_port)s
http_access allow our_sites
cache_peer_access backend allow our_sites
cache_peer_access backend deny all
# Drop squid headers
# via off
# reply_header_access X-Cache-Lookup deny all
# reply_header_access X-Squid-Error deny all
# reply_header_access X-Cache deny all
header_replace X-Forwarded-For
follow_x_forwarded_for allow all
forwarded_for on
# Use 1Go of RAM
cache_mem 1024 MB
# But do not keep big object in RAM
maximum_object_size_in_memory 2048 KB
# Log
access_log %(access_log_path)s
cache_log %(cache_log_path)s
pid_filename %(pid_filename_path)s
slapos/recipe/sshkeys_authority.py
View file @ d007050d
......@@ -113,11 +113,11 @@ class Request(GenericBaseRecipe):
hashlib.sha256(options['name']).hexdigest())
self.public_key = self.private_key + '.pub'
options['public-key-value'] = ''
if os.path.exists(self.public_key):
with open(self.public_key) as key:
options['public-key-value'] = key.read()
else:
options['public-key-value'] = ''
key_content = open(self.public_key).read()
if key_content:
options['public-key-value'] = key_content
def install(self):
requests_directory = self.options['request-directory']
......
software/apache-frontend/README.apache_frontend.txt
View file @ d007050d
......@@ -8,6 +8,31 @@ apache_frontend works using the master instance / slave instance design.
It means that a single main instance of Apache will be used to act as frontend
for many slaves.
Software type
=============
Apache frontend is available in 3 software types:
* default : The standard way to use the apache frontend configuring everything with a few given parameters
* custom-personal : This software type allow each slave to edit its apache configuration file
* custom-group : This software type use a template given as a parameter on master instance to generate apache configuration for all slaves
* replicate : This software type is set to replicate any kind of apache
About replicate frontend
========================
Slaves of the root instance (type "replicate") are sent as a parameter to requested frontends which will process them. The only difference is that they will then return the « would-be published information » to the root instance instead of publishing it. The root instance will then do a synthesis and publish the information to its slaves. The replicate instance only use 3 type of parameters for itself and will transmit the rest to requested frontends.
These parameters are :
* "-frontend-type" : the type to deploy frontends with. (default to 2)
* "-frontend-quantity" : The quantity of frontends to request (default to "default")
* "-sla-i-foo" : where "i" is the number of the concerned frontend (between 1 and "-frontend-quantity") and "foo" a sla parameter.
ex:
<parameter id="-frontend-quantity">3</parameter>
<parameter id="-frontend-type">custom-personal</parameter>
<parameter id="-sla-3-computer_guid">COMP-1234</parameter>
will request the third frontend on COMP-1234. All frontends will be of software type "custom-personal".
Note: the way slaves are transformed to a parameter avoid modifying more than 3 lines in the frontend logic.
Important NOTE: The way you ask for slave to a replicate frontend is the same as the one you would use for the software given in "-frontend-quantity". Do not forget to use "replicate" for software type. XXXXX So far it is not possible to do a simple request on a replicate frontend if you do not know the software_guid or other sla-parameter of the master instance. In fact we do not know yet the software type of the "requested" frontends. TO BE IMPLEMENTED
How to deploy a frontend server
===============================
......@@ -43,24 +68,28 @@ all slave instances.
Finally, the slave instance will be accessible from:
https://someidentifier.moulefrite.org.
How to have custom configuration in frontend server
===================================================
In your instance directory, you, as sysadmin, can directly edit two
configuration files that won't be overwritten by SlapOS to customize your
instance:
* $PARTITION_PATH/srv/srv/apache-conf.d/apache_frontend.custom.conf
* $PARTITION_PATH/srv/srv/apache-conf.d/apache_frontend.virtualhost.custom.conf
The first one is included in the end of the main apache configuration file.
The second one is included in the virtualhost of the main apache configuration file.
SlapOS will jsut create those two files for you, then completely forget them.
Note: make sure that the UNIX user of the instance has read access to those
files if you edit them.
About SSL
=========
Default and custom-personal software type can handle specific ssl for one slave instance.
IMPORTANT: One apache can not serve more than One specific SSL VirtualHost and be compatible with obsolete browser (i.e.: IE8). See http://wiki.apache.org/httpd/NameBasedSSLVHostsWithSNI
#How to have custom configuration in frontend server
#===================================================
#
#In your instance directory, you, as sysadmin, can directly edit two
#configuration files that won't be overwritten by SlapOS to customize your
#instance:
#
# * $PARTITION_PATH/srv/srv/apache-conf.d/apache_frontend.custom.conf
# * $PARTITION_PATH/srv/srv/apache-conf.d/apache_frontend.virtualhost.custom.conf
#
#The first one is included in the end of the main apache configuration file.
#The second one is included in the virtualhost of the main apache configuration file.
#
#SlapOS will jsut create those two files for you, then completely forget them.
#
#Note: make sure that the UNIX user of the instance has read access to those
#files if you edit them.
Instance Parameters
===================
......@@ -78,6 +107,10 @@ for the subdomains of the chosen domain like::
Using the IP given by the Master Instance.
"domain" is a mandatory Parameter.
public-ipv4
~~~~~~~~~~~
Public ipv4 of the frontend (the one Apache will be indirectly listening to)
port
~~~~
Port used by Apache. Optional parameter, defaults to 4443.
......@@ -87,8 +120,17 @@ plain_http_port
Port used by apache to serve plain http (only used to redirect to https).
Optional parameter, defaults to 8080.
Slave Instance Parameters
-------------------------
apache_custom_http (custom-group)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jinja template for apache virtualhost http configuration. It will be used by all slaves
apache_custom_https (custom-group)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jinja template for apache virtualhost https configuration. It will be used by all slaves
Slave Instance Parameters (default)
-----------------------------------
url
~~~
......@@ -98,7 +140,7 @@ Example: http://mybackend.com/myresource
enable_cache
~~~~~
Specify if slave instance should use a varnish / stunnel to connect to backend.
Specify if slave instance should use a squid to connect to backend.
Possible values: "true", "false".
"enable_cache" is an optional parameter. Defaults to "false".
Example: true
......@@ -111,10 +153,10 @@ Possible values: "zope", "default".
"type" is an optional parameter. Defaults to "default".
Example: zope
custom_domain
~~~~~~~~~~~~~
domain (former custom_domain)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Domain name to use as frontend. The frontend will be accessible from this domain.
"custom_domain" is an optional parameter. Defaults to
"domain" is an optional parameter. Defaults to
[instancereference].[masterdomain].
Example: www.mycustomdomain.com
......@@ -123,7 +165,8 @@ https-only
Specify if website should be accessed using https only. If so, the frontend
will redirect the user to https if accessed from http.
Possible values: "true", "false".
This is an optional parameter. Defaults to false.
"https-only" is an optional parameter. Defaults to "false".
Example: true
path
~~~~
......@@ -135,14 +178,75 @@ VirtualHostMonster.
"path" is an optional parameter, ignored if not specified.
Example of value: "/erp5/web_site_module/hosting/"
Slave Instance Parameters (custom-personal)
-------------------------------------------
apache_custom_https
~~~~~~~~~~~~~~~~~~~
Raw apache configuration in python template format (i.e. write "%%" for one "%") for the slave listening to the https port. Its content will be templatified in order to access functionalities such as cache access, ssl certificates... The list is available above.
NOTE: If you want to use the cache, use the apache option "ProxyPreserveHost On"
apache_custom_http
~~~~~~~~~~~~~~~~~~
Raw apache configuration in python template format (i.e. write "%%" for one "%") for the slave listening to the http port. Its content will be templatified in order to access functionalities such as cache access, ssl certificates... The list is available above
NOTE: If you want to use the cache, use the apache option "ProxyPreserveHost On"
url
~~~
Necesarry to activate cache. url of backend to use.
"url" is an optional parameter.
Example: http://mybackend.com/myresource
domain
~~~~~~
Necesarry to activate cache. The frontend will be accessible from this domain.
"domain" is an optional parameter.
Example: www.mycustomdomain.com
enable_cache
~~~~~~~~~~~~
Necesarry to activate cache.
"enable_cache" is an optional parameter.
ssl_key, ssl_crt, ssl_ca_crt, ssl_crs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SSL certificates of the slave.
They are optional.
Functionalities for apache configuration:
In the slave apache configuration you can use parameters that will be replaced during instanciation. They should be entered as python templates parameters ex:" %(parameter)s"
* cache_access : url of the cache. Should replace backend url in configuration to use the cache
* error_log : path of the slave error log in order to log in a deferenciated file.
* error_log : path of the slave access log in order to log in a deferenciated file.
* ssl_key, ssl_crt, ssl_ca_crt, ssl_crs : path of the certificates given in slave instance parameters
Slave Instance Parameters (custom-group)
----------------------------------------
url
~~~
Necesarry to activate cache. url of backend to use.
"url" is an optional parameter.
Example: http://mybackend.com/myresource
domain
~~~~~~
Domain name to use as frontend. The frontend will be accessible from this domain.
"domain" is an optional parameter necessary to activate cache. Defaults to
[instancereference].[masterdomain].
Example: www.mycustomdomain.com
The rest of the parameters are defined by templates given to the master and accessible by the slave_parameter dict in it.
Examples
========
Here are some example of how to make your SlapOS service available through
an already deployed frontend.
Simple Example
--------------
Simple Example (default)
------------------------
Request slave frontend instance so that https://[1:2:3:4:5:6:7:8]:1234 will be
redirected and accessible from the proxy::
......@@ -157,8 +261,8 @@ redirected and accessible from the proxy::
)
Zope Example
------------
Zope Example (default)
----------------------
Request slave frontend instance using a Zope backend so that
https://[1:2:3:4:5:6:7:8]:1234 will be redirected and accessible from the
......@@ -175,8 +279,8 @@ proxy::
)
Advanced example
----------------
Advanced example (default)
--------------------------
Request slave frontend instance using a Zope backend, with Varnish activated,
listening to a custom domain and redirecting to /erp5/ so that
......@@ -192,7 +296,192 @@ the proxy::
"enable_cache":"true",
"type":"zope",
"path":"/erp5",
"custom_domain":"mycustomdomain.com",
"domain":"mycustomdomain.com",
}
)
Simple Example (custom-personal)
--------------------------------
Request slave frontend instance so that https://[1:2:3:4:5:6:7:8]:1234 will be
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
software_type="custom-personal",
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"apache_custom_https":'
ServerName www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLEngine on
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
RewriteRule ^/(.*) https://[1:2:3:4:5:6:7:8]:1234/$1 [L,P]',
"apache_custom_http":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteRule ^/(.*) https://[1:2:3:4:5:6:7:8]:1234/$1 [L,P],
}
)
Simple Cache Example (custom-personal)
--------------------------------
Request slave frontend instance so that https://[1:2:3:4:5:6:7:8]:1234 will be
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
software_type="custom-personal",
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"domain": "www.example.org",
"enable_cache": "True",
"apache_custom_https":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLEngine on
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
RewriteRule ^/(.*) %(cache_access)s/$1 [L,P]',
"apache_custom_http":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteRule ^/(.*) %(cache_access)s/$1 [L,P],
}
)
Advanced example (custom-personal)
----------------------------------
Request slave frontend instance using custom apache configuration, willing to use cache and ssl certificates.
listening to a custom domain and redirecting to /erp5/ so that
https://[1:2:3:4:5:6:7:8]:1234/erp5/ will be redirected and accessible from
the proxy::
instance = request(
software_release=apache_frontend,
software_type="RootSoftwareInstance",
partition_reference='my frontend',
shared=True,
software_type="custom-personal",
partition_parameter_kw={
"url":"https://[1:2:3:4:5:6:7:8]:1234",
"enable_cache":"true",
"type":"zope",
"path":"/erp5",
"domain":"example.org",
"apache_custom_https":'
ServerName www.example.org
ServerAlias www.example.org
ServerAdmin example.org
SSLEngine on
SSLProxyEngine on
SSLProtocol -ALL +SSLv3 +TLSv1
SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!ADH
# Use personal ssl certificates
SSLCertificateFile %(ssl_crt)s
SSLCertificateKeyFile %(ssl_key)s
SSLCACertificateFile %(ssl_ca_crt)s
SSLCertificateChainFile %(ssl_ca_crt)s
# Configure personal logs
ErrorLog "%(error_log)s"
LogLevel info
LogFormat "%%h %%l %%{REMOTE_USER}i %%t \"%%r\" %%>s %%b \"%%{Referer}i\" \"%%{User-Agent}i\" %%D" combined
CustomLog "%(access_log)s" combined
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Redirect / to /index.html
RewriteRule ^/$ /index.html [R=302,L]
# Use cache
RewriteRule ^/(.*) %(cache_access)s/VirtualHostBase/https/www.example.org:443/erp5/VirtualHostRoot/$1 [L,P]',
"apache_custom_http":'
ServerName www.example.org
ServerAlias www.example.org
ServerAlias example.org
ServerAdmin geronimo@example.org
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Configure personal logs
ErrorLog "%(error_log)s"
LogLevel info
LogFormat "%%h %%l %%{REMOTE_USER}i %%t \"%%r\" %%>s %%b \"%%{Referer}i\" \"%%{User-Agent}i\" %%D" combined
CustomLog "%(access_log)s" combined
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteRule ^/(.*)$ https://%%{SERVER_NAME}%%{REQUEST_URI}',
"ssl_key":"-----BEGIN RSA PRIVATE KEY-----
XXXXXXX..........XXXXXXXXXXXXXXX
-----END RSA PRIVATE KEY-----",
"ssl_crt":'-----BEGIN CERTIFICATE-----
XXXXXXXXXXX.............XXXXXXXXXXXXXXXXXXX
-----END CERTIFICATE-----',
"ssl_ca_crt":'-----BEGIN CERTIFICATE-----
XXXXXXXXX...........XXXXXXXXXXXXXXXXX
-----END CERTIFICATE-----',
"ssl_csr":'-----BEGIN CERTIFICATE REQUEST-----
XXXXXXXXXXXXXXX.............XXXXXXXXXXXXXXXXXX
-----END CERTIFICATE REQUEST-----',
}
)
......
software/apache-frontend/TODO.txt 0 → 100644
View file @ d007050d
Apache:
=======
- set a redirection of / in option
- Implement support of multiple ip (if possible) for multiple ssl (other solution is to ask master instance but quid to much apache process?)
Squid:
======
- Only cache in ram. Problems with to much squid on the computer or too many slave?
SlapOS:
=======
- Implement intelligent apache graceful -> Should check slave configuration and return error to slave if there is a problem (important but difficult)
- useless srv/squid_cache directory so far
software/apache-frontend/common.cfg
View file @ d007050d
[buildout]
extends =
# dev Stuff
../../component/git/buildout.cfg
../../stack/slapos.cfg
../../component/binutils/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/apache/buildout.cfg
../../component/gzip/buildout.cfg
../../component/stunnel/buildout.cfg
../../component/varnish/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
../../component/rdiff-backup/buildout.cfg
../../stack/slapos.cfg
../../component/squid/buildout.cfg
parts =
parts +=
slapos-cookbook
slapos-toolbox
template
template-apache-frontend
template-apache-replicate
binutils
apache-2.2
apache-antiloris-apache-2.2
stunnel
varnish-2.1
dcron
logrotate
rdiff-backup
squid
# Buildoutish
eggs
instance-recipe-egg
[instance-recipe]
# Note: In case if specific instantiation recipe is used this is the place to
# put its name
egg = slapos.cookbook
module = apache.frontend
[instance-recipe-egg]
[slapos-toolbox]
recipe = zc.recipe.egg
eggs = ${instance-recipe:egg}
[eggs]
recipe = z3c.recipe.scripts
eggs =
${lxml-python:egg}
slapos.toolbox
scripts =
killpidfromfile
onetimedownload
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
[template]
# Default template for apache instance.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = e7b9f57da7eb1450fc15789e239388d4
md5sum = 9c6346c8eaf484748e6be0b62b65cf2e
output = ${buildout:directory}/template.cfg
mode = 0644
[template-apache-frontend]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-apache-frontend.cfg
md5sum = 7e7e7599ec41cf1eb6e8e725d855c345
output = ${buildout:directory}/template-apache-frontend.cfg
mode = 0644
[template-apache-replicate]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/instance-apache-replicate.cfg.in
md5sum = 2c96799f1429d0541c04c0875d864777
mode = 0644
[template-slave-list]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/apache-custom-slave-list.cfg.in
md5sum = f5eef006211809669b12422240c6f436
mode = 640
[template-slave-configuration]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/slave-virtualhost.conf.in
md5sum = a7ad2e83b7f919fc45a7ef1e64344dcb
mode = 640
[template-replicate-publish-slave-information]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/replicate-publish-slave-information.cfg.in
mode = 640
[template-apache-frontend-configuration]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/apache.conf.in
md5sum = c141b9e78c7e80d75bb40493910294e5
mode = 640
[template-apache-cached-configuration]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/apache_cached.conf.in
md5sum = 0c4393db80670daf18b432b7f07383e9
mode = 640
[template-rewrite-cached]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/apache_cached_rewrite.txt.in
md5sum = 2f30af4f9da340c2b0618599da03ed4b
mode = 640
[template-custom-slave-list]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/apache-default-slave-list.cfg.in
md5sum = 9362384cd80727987b34c7746a6de196
mode = 640
[template-not-found-html]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/notfound.html
filename = notfound.html
md5sum = f20d6c3d2d94fb685f8d26dfca1e822b
mode = 640
[template-default-virtualhost]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/000.conf.in
md5sum = c2bbf029e6adc432de0884fb5cf5d2ab
mode = 640
[template-default-slave-virtualhost]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/default-virtualhost.conf.in
md5sum = ac845c0fa3835832307a0e7323cb339d
mode = 640
[template-empty]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/templates/empty.in
md5sum = c2314c3a9c3412a38d14b312d3df83c1
mode = 640
\ No newline at end of file
software/apache-frontend/instance-apache-frontend.cfg 0 → 100644
View file @ d007050d
[buildout]
parts =
directory
configtest
logrotate
cron
cron-entry-logrotate
ca-frontend
certificate-authority
squid-cache
logrotate-entry-apache
logrotate-entry-apache-cached
logrotate-entry-squid
apache-frontend
apache-cached
switch-apache-softwaretype
frontend-apache-graceful
cached-apache-graceful
squid-reload
dynamic-template-default-vh
not-found-html
promise-apache-frontend-v4-https
promise-apache-frontend-v4-http
promise-apache-frontend-v6-https
promise-apache-frontend-v6-http
promise-apache-cached
promise-squid
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
# Create all needed directories
[directory]
recipe = slapos.cookbook:mkdirectory
bin = $${buildout:directory}/bin/
etc = $${buildout:directory}/etc/
srv = $${buildout:directory}/srv/
var = $${buildout:directory}/var/
template = $${buildout:directory}/template/
backup = $${:srv}/backup
log = $${:var}/log
run = $${:var}/run
service = $${:etc}/service
etc-run = $${:etc}/run
promise = $${:etc}/promise
logrotate-backup = $${:backup}/logrotate
logrotate-entries = $${:etc}/logrotate.d
cron-entries = $${:etc}/cron.d
crontabs = $${:etc}/crontabs
cronstamps = $${:etc}/cronstamps
ca-dir = $${:srv}/ssl
squid-cache = $${:srv}/squid_cache
[switch-apache-softwaretype]
recipe = slapos.cookbook:softwaretype
default = $${dynamic-default-template-slave-list:rendered}
custom-personal = $${dynamic-custom-personal-template-slave-list:rendered}
custom-group = $${dynamic-custom-group-template-slave-list:rendered}
[instance-parameter]
# Fetches parameters defined in SlapOS Master for this instance.
# Always the same.
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
# Define default parameter(s) that will be used later, in case user didn't
# specify it
# All parameters are available through the configuration.XX syntax.
# All possible parameters should have a default.
configuration.domain = example.org
configuration.public-ipv4 =
configuration.port = 4443
configuration.plain_http_port = 8080
configuration.server-admin = admin@example.com
configuration.apache_custom_https = ""
configuration.apache_custom_http = ""
configuration.apache-key =
configuration.apache-certificate =
configuration.open-port = 80 443
configuration.extra_slave_instance_list =
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type instance-parameter:slap-software-type
key slapparameter_dict instance-parameter:configuration
$${:extra-context}
[dynamic-template-default-vh]
< = jinja2-template-base
template = ${template-default-virtualhost:target}
rendered = $${apache-directory:slave-configuration}/000.conf
extensions = jinja2.ext.do
extra-context =
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
[dynamic-custom-personal-template-slave-list]
< = jinja2-template-base
template = ${template-slave-list:target}
filename = custom-personal-instance-slave-list.cfg
extensions = jinja2.ext.do
extra-context =
key apache_configuration_directory apache-directory:slave-configuration
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key public_ipv4 instance-parameter:configuration.public-ipv4
key slave_instance_list instance-parameter:slave-instance-list
key extra_slave_instance_list instance-parameter:configuration.extra_slave_instance_list
key rewrite_cached_configuration apache-configuration:cached-rewrite-file
key custom_ssl_directory apache-directory:vh-ssl
key apache_log_directory apache-directory:slave-log
key local_ipv4 instance-parameter:ipv4-random
key cache_port apache-configuration:cache-port
raw empty_template ${template-empty:target}
raw template_slave_configuration ${template-slave-configuration:target}
raw template_rewrite_cached ${template-rewrite-cached:target}
raw software_type custom-personal
[dynamic-custom-group-template-slave-list]
< = jinja2-template-base
template = ${template-custom-slave-list:target}
filename = custom-group-instance-slave-list.cfg
extensions = jinja2.ext.do
extra-context =
key apache_configuration_directory apache-directory:slave-configuration
key domain instance-parameter:configuration.domain
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key public_ipv4 instance-parameter:configuration.public-ipv4
key slave_instance_list instance-parameter:slave-instance-list
key extra_slave_instance_list instance-parameter:configuration.extra_slave_instance_list
key rewrite_cached_configuration apache-configuration:cached-rewrite-file
key custom_ssl_directory apache-directory:vh-ssl
key template_slave_configuration dynamic-virtualhost-template-slave:rendered
key apache_log_directory apache-directory:slave-log
key local_ipv4 instance-parameter:ipv4-random
key cache_port apache-configuration:cache-port
raw empty_template ${template-empty:target}
raw template_rewrite_cached ${template-rewrite-cached:target}
raw software_type custom-group
[dynamic-default-template-slave-list]
< = jinja2-template-base
template = ${template-custom-slave-list:target}
filename = default-instance-slave-list.cfg
extensions = jinja2.ext.do
extra-context =
key apache_configuration_directory apache-directory:slave-configuration
key domain instance-parameter:configuration.domain
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key public_ipv4 instance-parameter:configuration.public-ipv4
key slave_instance_list instance-parameter:slave-instance-list
key extra_slave_instance_list instance-parameter:configuration.extra_slave_instance_list
key rewrite_cached_configuration apache-configuration:cached-rewrite-file
key custom_ssl_directory apache-directory:vh-ssl
key apache_log_directory apache-directory:slave-log
key local_ipv4 instance-parameter:ipv4-random
key cache_port apache-configuration:cache-port
raw template_slave_configuration ${template-default-slave-virtualhost:target}
raw empty_template ${template-empty:target}
raw template_rewrite_cached ${template-rewrite-cached:target}
raw software_type default-RootSoftwareInstance
# XXXX Hack to allow two software types
[dynamic-virtualhost-template-slave]
<= jinja2-template-base
template = ${template-slave-configuration:target}
rendered = $${directory:template}/slave-virtualhost.conf.in
extensions = jinja2.ext.do
extra-context =
key https_port instance-parameter:configuration.port
key http_port instance-parameter:configuration.plain_http_port
key apache_custom_https instance-parameter:configuration.apache_custom_https
key apache_custom_http instance-parameter:configuration.apache_custom_http
# Deploy Apache Frontend (new way, no recipe, jinja power)
[dynamic-apache-frontend-template]
< = jinja2-template-base
template = ${template-apache-frontend-configuration:target}
rendered = $${apache-configuration:frontend-configuration}
extra-context =
raw httpd_home ${apache-2.2:location}
key httpd_mod_ssl_cache_directory apache-directory:mod-ssl
key domain instance-parameter:configuration.domain
key document_root apache-directory:document-root
key instance_home buildout:directory
key ipv4_addr instance-parameter:ipv4-random
key ipv6_addr instance-parameter:ipv6-random
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key server_admin instance-parameter:configuration.server-admin
key protected_path apache-configuration:protected-path
key access_control_string apache-configuration:access-control-string
key login_certificate ca-frontend:cert-file
key login_key ca-frontend:key-file
key ca_dir certificate-authority:ca-dir
key ca_crl certificate-authority:ca-crl
key access_log apache-configuration:access-log
key error_log apache-configuration:error-log
key pid_file apache-configuration:pid-file
key slave_configuration_directory apache-directory:slave-configuration
[apache-frontend]
recipe = slapos.cookbook:wrapper
command-line = ${apache-2.2:location}/bin/httpd -f $${dynamic-apache-frontend-template:rendered} -DFOREGROUND
wrapper-path = $${directory:service}/frontend_apache
wait-for-files =
$${ca-frontend:cert-file}
$${ca-frontend:key-file}
# Deploy Apache for cached website
[dynamic-apache-cached-template]
< = jinja2-template-base
template = ${template-apache-cached-configuration:target}
rendered = $${apache-configuration:cached-configuration}
extra-context =
raw httpd_home ${apache-2.2:location}
key httpd_mod_ssl_cache_directory apache-directory:mod-ssl
key domain instance-parameter:configuration.domain
key document_root apache-directory:document-root
key instance_home buildout:directory
key ipv4_addr instance-parameter:ipv4-random
key cached_port apache-configuration:cache-through-port
key server_admin instance-parameter:configuration.server-admin
key protected_path apache-configuration:protected-path
key access_control_string apache-configuration:access-control-string
key login_certificate ca-frontend:cert-file
key login_key ca-frontend:key-file
key ca_dir certificate-authority:ca-dir
key ca_crl certificate-authority:ca-crl
key access_log apache-configuration:cache-access-log
key error_log apache-configuration:cache-error-log
key pid_file apache-configuration:cache-pid-file
key apachecachedmap_path apache-configuration:cached-rewrite-file
[apache-cached]
recipe = slapos.cookbook:wrapper
command-line = ${apache-2.2:location}/bin/httpd -f $${dynamic-apache-cached-template:rendered} -DFOREGROUND
wrapper-path = $${directory:service}/frontend_cached_apache
wait-for-files =
$${ca-frontend:cert-file}
$${ca-frontend:key-file}
[not-found-html]
recipe = slapos.cookbook:symbolic.link
target-directory = $${apache-directory:document-root}
link-binary =
${template-not-found-html:target}
[apache-directory]
recipe = slapos.cookbook:mkdirectory
document-root = $${directory:srv}/htdocs
slave-configuration = $${directory:etc}/apache-slave-conf.d/
cache = $${directory:var}/cache
mod-ssl = $${:cache}/httpd_mod_ssl
vh-ssl = $${:slave-configuration}/ssl
slave-log = $${directory:log}/httpd
[apache-configuration]
frontend-configuration = $${directory:etc}/apache_frontend.conf
cached-configuration = $${directory:etc}/apache_frontend_cached.conf
access-log = $${directory:log}/frontend-apache-access.log
error-log = $${directory:log}/frontend-apache-error.log
pid-file = $${directory:run}/httpd.pid
protected-path = /
access-control-string = none
cached-rewrite-file = $${directory:etc}/apache_rewrite_cached.txt
# Apache for cache configuration
cache-access-log = $${directory:log}/frontend-apache-access-cached.log
cache-error-log = $${directory:log}/frontend-apache-error-cached.log
cache-pid-file = $${directory:run}/httpd-cached.pid
# Comunication with squid
cache-port = 26010
cache-through-port = 26011
# Create wrapper for "apachectl conftest" in bin
[configtest]
recipe = slapos.cookbook:wrapper
command-line = ${apache-2.2:location}/bin/httpd -f $${directory:etc}/apache_frontend.conf -t
wrapper-path = $${directory:bin}/apache-configtest
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = ${openssl:location}/bin/openssl
ca-dir = $${directory:ca-dir}
requests-directory = $${cadirectory:requests}
wrapper = $${directory:service}/certificate_authority
ca-private = $${cadirectory:private}
ca-certs = $${cadirectory:certs}
ca-newcerts = $${cadirectory:newcerts}
ca-crl = $${cadirectory:crl}
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:ca-dir}/requests/
private = $${directory:ca-dir}/private/
certs = $${directory:ca-dir}/certs/
newcerts = $${directory:ca-dir}/newcerts/
crl = $${directory:ca-dir}/crl/
[ca-frontend]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${cadirectory:certs}/apache_frontend.key
cert-file = $${cadirectory:certs}/apache_frontend.crt
executable = $${directory:service}/frontend_apache
wrapper = $${directory:service}/frontend_apache
key-content = $${instance-parameter:configuration.apache-key}
cert-content = $${instance-parameter:configuration.apache-certificate}
# Put domain name
name = $${instance-parameter:configuration.domain}
[cron]
recipe = slapos.cookbook:cron
dcrond-binary = ${dcron:location}/sbin/crond
cron-entries = $${directory:cron-entries}
crontabs = $${directory:crontabs}
cronstamps = $${directory:cronstamps}
catcher = $${cron-simplelogger:wrapper}
binary = $${directory:service}/crond
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
wrapper = $${directory:bin}/cron_simplelogger
log = $${directory:log}/cron.log
[cron-entry-logrotate]
<= cron
recipe = slapos.cookbook:cron.d
name = logrotate
frequency = 0 0 * * *
command = $${logrotate:wrapper}
# Deploy Logrotate
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = ${logrotate:location}/usr/sbin/logrotate
gzip-binary = ${gzip:location}/bin/gzip
gunzip-binary = ${gzip:location}/bin/gunzip
# Directories
wrapper = $${directory:bin}/logrotate
conf = $${directory:etc}/logrotate.conf
logrotate-entries = $${directory:logrotate-entries}
backup = $${directory:logrotate-backup}
state-file = $${directory:srv}/logrotate.status
[logrotate-entry-apache]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = apache
log = $${apache-directory:slave-log}/*_log $${apache-configuration:error-log} $${apache-configuration:access-log}
frequency = daily
rotatep-num = 30
post = ${buildout:bin-directory}/killpidfromfile $${apache-configuration:pid-file} SIGUSR1
sharedscripts = true
notifempty = true
create = true
[logrotate-entry-apache-cached]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = apache-cached
log = $${apache-configuration:cache-error-log} $${apache-configuration:cache-access-log}
frequency = daily
rotatep-num = 30
post = ${buildout:bin-directory}/killpidfromfile $${apache-configuration:cache-pid-file} SIGUSR1
sharedscripts = true
notifempty = true
create = true
[logrotate-entry-squid]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = squid
log = $${squid-cache:cache-log-path} $${squid-cache:access-log-path}
frequency = daily
rotatep-num = 30
post = ${buildout:bin-directory}/killpidfromfile $${apache-configuration:pid-file} SIGHUP
sharedscripts = true
notifempty = true
create = true
[squid-cache]
recipe = slapos.cookbook:squid
prepare-path = $${directory:etc-run}/squid-prepare
wrapper-path = $${directory:service}/squid
binary-path = ${squid:location}/sbin/squid
conf-path = $${directory:etc}/squid.cfg
cache-path = $${directory:squid-cache}
ip = $${instance-parameter:ipv4-random}
port = $${apache-configuration:cache-port}
backend-ip = $${instance-parameter:ipv4-random}
backend-port = $${apache-configuration:cache-through-port}
open-port = $${instance-parameter:configuration.open-port}
access-log-path = $${directory:log}/squid-access.log
cache-log-path = $${directory:log}/squid-cache.log
pid-filename-path = $${directory:run}/squid.pid
[squid-reload]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/killpidfromfile $${squid-cache:pid-filename-path} SIGHUP
wrapper-path = $${directory:etc-run}/squid-reload
[frontend-apache-graceful]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/killpidfromfile $${apache-configuration:pid-file} SIGUSR1
wrapper-path = $${directory:etc-run}/frontend-apache-graceful
[cached-apache-graceful]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/killpidfromfile $${apache-configuration:cache-pid-file} SIGUSR1
wrapper-path = $${directory:etc-run}/cached-apache-graceful
[promise-apache-frontend-v4-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv4_https
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.port}
[promise-apache-frontend-v4-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv4_http
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.plain_http_port}
[promise-apache-frontend-v6-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv6_https
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.port}
[promise-apache-frontend-v6-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_frontend_ipv6_http
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.plain_http_port}
[promise-apache-cached]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/apache_cached
hostname = $${instance-parameter:ipv4-random}
port = $${apache-configuration:cache-through-port}
[promise-squid]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/squid
hostname = $${instance-parameter:ipv4-random}
port = $${apache-configuration:cache-port}
[slap_connection]
# Kept for backward compatiblity
computer_id = $${slap-connection:computer-id}
partition_id = $${slap-connection:partition-id}
server_url = $${slap-connection:server-url}
software_release_url = $${slap-connection:software-release-url}
key_file = $${slap-connection:key-file}
cert_file = $${slap-connection:cert-file}
software/apache-frontend/instance-apache-replicate.cfg.in 0 → 100644
View file @ d007050d
{% if slap_software_type.startswith(software_type) -%}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = ${buildout:directory}/${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-parameter:slap_software_type
key slave_instance_list slap-parameter:slave_instance_list
${:extra-context}
{% set part_list = [] -%}
{% set type_key = 'replicate-' %}
{% set type_key_length = type_key | length %}
{% if slap_software_type.startswith(type_key) %}
{% set frontend_type = slap_software_type[type_key_length:] -%}
{% else -%}
{% set frontend_type = slapparameter_dict.pop('-frontend-type', 'default') -%}
{% endif -%}
{% set frontend_quantity = slapparameter_dict.pop('-frontend-quantity', '2') | int -%}
{% set slave_list_name = 'extra_slave_instance_list' -%}
{% set frontend_list = [] %}
{% set frontend_section_list = [] %}
{% set namebase = 'apache-frontend' -%}
# Here we request individualy each frontend.
# The presence of sla parameters is checked and added if found
{% for i in range(1, frontend_quantity + 1) -%}
{% set frontend_name = "%s-%s" % (namebase, i) -%}
{% set request_section_title = 'request-%s' % frontend_name -%}
{% set sla_key = "-sla-%s-" % i -%}
{% set sla_key_length = sla_key | length %}
{% set sla_parameters = [] %}
{% for key in slapparameter_dict.keys() %}
{% if key.startswith(sla_key) %}
{% do sla_parameters.append(key[sla_key_length:]) %}
{% endif -%}
{% endfor -%}
{% do frontend_list.append(frontend_name) -%}
{% do frontend_section_list.append(request_section_title) -%}
{% do part_list.append(request_section_title) -%}
[{{request_section_title}}]
<= replicate
name = {{frontend_name}}
{% if sla_parameters %}
sla = {{ ' '.join(sla_parameters) }}
{% for parameter in sla_parameters -%}
sla-{{ parameter }} = {{ slapparameter_dict.pop( sla_key + parameter ) }}
{% endfor -%}
{% endif -%}
{% endfor -%}
[replicate]
<= slap-connection
recipe = slapos.cookbook:request
software-url = ${slap-connection:software-release-url}
software-type = {{frontend_type}}
return = private-ipv4 public-ipv4 slave-instance-information-list
config = {{ ' '.join(slapparameter_dict.keys()) + ' ' + slave_list_name }}
{% for parameter, value in slapparameter_dict.iteritems() -%}
config-{{parameter}} = {{ value }}
{% endfor -%}
config-{{ slave_list_name }} = {{ json_module.dumps(slave_instance_list) }}
[publish-information]
recipe = slapos.cookbook:publish
domain = {{ slapparameter_dict.get('domain') }}
slave-amount = {{ slave_instance_list | length }}
{% for frontend in frontend_list -%}
#{{frontend}}-private-ipv4 = ${request-{{frontend}}:private-ipv4}
{% endfor -%}
#----------------------------
#--
#-- Publish slave information
[publish-slave-information]
recipe = slapos.cookbook:softwaretype
default = ${dynamic-publish-slave-information:rendered}
replicate = ${dynamic-publish-slave-information:rendered}
[slave-information]
{% for frontend_section in frontend_section_list -%}
{{ frontend_section }} = {{ "${%s:connection-slave-instance-information-list}" % frontend_section }}
{% endfor -%}
[dynamic-publish-slave-information]
< = jinja2-template-base
template = {{ template_publish_slave_information }}
filename = dynamic-publish-slave-information.cfg
extensions = jinja2.ext.do
extra-context =
section slave_information slave-information
[buildout]
parts =
publish-slave-information
publish-information
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
# publish-information
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[slap_connection]
# Kept for backward compatiblity
computer_id = ${slap-connection:computer-id}
partition_id = ${slap-connection:partition-id}
server_url = ${slap-connection:server-url}
software_release_url = ${slap-connection:software-release-url}
key_file = ${slap-connection:key-file}
cert_file = ${slap-connection:cert-file}
[slap-parameter]
slave_instance_list =
-frontend-quantity = 2
-frontend-type = default
{%- endif %}
software/apache-frontend/instance.cfg
View file @ d007050d
[buildout]
parts =
directory
apache
configtest
logrotate
logrotate-entry-apache
dynamic-template-apache-replicate
switch-softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
# Create all needed directories
[directory]
recipe = slapos.cookbook:mkdirectory
bin = $${buildout:directory}/bin/
etc = $${buildout:directory}/etc/
srv = $${buildout:directory}/srv/
var = $${buildout:directory}/var/
backup = $${:srv}/backup
log = $${:var}/log
run = $${:var}/run
service = $${:etc}/service
logrotate-backup = $${:backup}/logrotate
logrotate-entries = $${:etc}/logrotate.d
# Deploy Apache (old way, with monolithic recipe)
[apache]
recipe = ${instance-recipe:egg}:${instance-recipe:module}
httpd_home = ${apache-2.2:location}
httpd_binary = ${apache-2.2:location}/bin/httpd
logrotate_binary = ${logrotate:location}/usr/sbin/logrotate
openssl_binary = ${openssl:location}/bin/openssl
dcrond_binary = ${dcron:location}/sbin/crond
varnishd_binary = ${varnish-2.1:location}/sbin/varnishd
stunnel_binary = ${stunnel:location}/bin/stunnel
rdiff_backup_binary = ${buildout:bin-directory}/rdiff-backup
gcc_binary = gcc
binutils_directory = ${binutils:location}/bin/
access-log = $${directory:log}/frontend-apache-access.log
error-log = $${directory:log}/frontend-apache-error.log
pid-file = $${directory:run}/httpd.pid
# Create wrapper for "apachectl conftest" in bin
[configtest]
recipe = slapos.cookbook:wrapper
command-line = $${apache:httpd_binary} -f $${directory:etc}/apache_frontend.conf -t
wrapper-path = $${directory:bin}/apache-configtest
# Deploy Logrotate
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = ${logrotate:location}/usr/sbin/logrotate
gzip-binary = ${gzip:location}/bin/gzip
gunzip-binary = ${gzip:location}/bin/gunzip
# Directories
wrapper = $${directory:bin}/logrotate
conf = $${directory:etc}/logrotate.conf
logrotate-entries = $${directory:logrotate-entries}
backup = $${directory:logrotate-backup}
state-file = $${directory:srv}/logrotate.status
[logrotate-entry-apache]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = apache
log = $${apache:error-log} $${apache:access-log}
frequency = daily
rotate-num = 30
post = ${buildout:bin-directory}/killpidfromfile $${apache:pid-file} SIGUSR1
sharedscripts = true
notifempty = true
create = true
offline = true
[slap-parameters]
recipe = slapos.cookbook:slapconfiguration
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-parameters:slap-software-type
key slapparameter_dict slap-parameters:configuration
key slave_instance_list slap-parameters:slave-instance-list
$${:extra-context}
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${template-apache-frontend:output}
custom-personal = ${template-apache-frontend:output}
custom-group = ${template-apache-frontend:output}
replicate-default = $${dynamic-template-apache-replicate:rendered}
replicate-custom-personal = $${dynamic-template-apache-replicate:rendered}
replicate-custom-group = $${dynamic-template-apache-replicate:rendered}
replicate = $${dynamic-template-apache-replicate:rendered}
[dynamic-template-apache-replicate]
< = jinja2-template-base
template = ${template-apache-replicate:target}
filename = instance-apache-replicate.cfg
extensions = jinja2.ext.do
extra-context =
raw template_publish_slave_information ${template-replicate-publish-slave-information:target}
# Must match the key id in [switch-softwaretype] which uses this section.
raw software_type replicate
software/apache-frontend/software.cfg
View file @ d007050d
......@@ -18,6 +18,15 @@ slapos.recipe.template = 2.4.2
slapos.toolbox = 0.34.0
smmap = 0.8.2
z3c.recipe.scripts = 1.0.1
cliff = 1.4.4
cmd2 = 0.6.5.1
prettytable = 0.7.2
requests = 1.2.3
slapos.cookbook = 0.82
# Required by:
# slapos.cookbook==0.82
lock-file = 2.0
# Required by:
# slapos.core==0.35.1
......@@ -37,17 +46,17 @@ atomize = 0.1.1
feedparser = 5.1.3
# Required by:
# slapos.cookbook==0.77.1
# slapos.cookbook==0.82
inotifyx = 0.2.0
# Required by:
# slapos.cookbook==0.77.1
# slapos.cookbook==0.82
# slapos.core==0.35.1
# xml-marshaller==0.9.7
lxml = 3.1.2
# Required by:
# slapos.cookbook==0.77.1
# slapos.cookbook==0.82
netaddr = 0.7.10
# Required by:
......@@ -67,11 +76,11 @@ psutil = 0.7.0
pyflakes = 0.7
# Required by:
# slapos.cookbook==0.77.1
# slapos.cookbook==0.82
pytz = 2013b
# Required by:
# slapos.cookbook==0.77.1
# slapos.cookbook==0.82
# slapos.core==0.35.1
# slapos.toolbox==0.34.0
# zc.buildout==1.6.0-dev-SlapOS-010
......@@ -79,7 +88,7 @@ pytz = 2013b
setuptools = 0.6c12dev-r88846
# Required by:
# slapos.cookbook==0.77.1
# slapos.cookbook==0.82
# slapos.toolbox==0.34.0
slapos.core = 0.35.1
......@@ -92,7 +101,7 @@ supervisor = 3.0b1
unittest2 = 0.5.1
# Required by:
# slapos.cookbook==0.77.1
# slapos.cookbook==0.82
# slapos.toolbox==0.34.0
xml-marshaller = 0.9.7
......@@ -102,10 +111,54 @@ zope.interface = 4.0.5
[networkcache]
# signature certificates of the following uploaders.
# Cedric de Saint Martin
# Romain Courteaud
# Test Agent
# Sebastien Robin
# Kazuhiko Shiozaki
# Cedric de Saint Martin
# Yingjie Xu
# Gabriel Monnerat
# Łukasz Nowak
# Test Agent (Automatic update from tests)
signature-certificate-list =
-----BEGIN CERTIFICATE-----
MIIB4DCCAUkCADANBgkqhkiG9w0BAQsFADA5MQswCQYDVQQGEwJGUjEZMBcGA1UE
CBMQRGVmYXVsdCBQcm92aW5jZTEPMA0GA1UEChMGTmV4ZWRpMB4XDTExMDkxNTA5
MDAwMloXDTEyMDkxNTA5MDAwMlowOTELMAkGA1UEBhMCRlIxGTAXBgNVBAgTEERl
ZmF1bHQgUHJvdmluY2UxDzANBgNVBAoTBk5leGVkaTCBnzANBgkqhkiG9w0BAQEF
AAOBjQAwgYkCgYEApYZv6OstoqNzxG1KI6iE5U4Ts2Xx9lgLeUGAMyfJLyMmRLhw
boKOyJ9Xke4dncoBAyNPokUR6iWOcnPHtMvNOsBFZ2f7VA28em3+E1JRYdeNUEtX
Z0s3HjcouaNAnPfjFTXHYj4um1wOw2cURSPuU5dpzKBbV+/QCb5DLheynisCAwEA
ATANBgkqhkiG9w0BAQsFAAOBgQBCZLbTVdrw3RZlVVMFezSHrhBYKAukTwZrNmJX
mHqi2tN8tNo6FX+wmxUUAf3e8R2Ymbdbn2bfbPpcKQ2fG7PuKGvhwMG3BlF9paEC
q7jdfWO18Zp/BG7tagz0jmmC4y/8akzHsVlruo2+2du2freE8dK746uoMlXlP93g
QUUGLQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB8jCCAVugAwIBAgIJAPu2zchZ2BxoMA0GCSqGSIb3DQEBBQUAMBIxEDAOBgNV
BAMMB3RzeGRldjMwHhcNMTExMDE0MTIxNjIzWhcNMTIxMDEzMTIxNjIzWjASMRAw
DgYDVQQDDAd0c3hkZXYzMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCrPbh+
YGmo6mWmhVb1vTqX0BbeU0jCTB8TK3i6ep3tzSw2rkUGSx3niXn9LNTFNcIn3MZN
XHqbb4AS2Zxyk/2tr3939qqOrS4YRCtXBwTCuFY6r+a7pZsjiTNddPsEhuj4lEnR
L8Ax5mmzoi9nE+hiPSwqjRwWRU1+182rzXmN4QIDAQABo1AwTjAdBgNVHQ4EFgQU
/4XXREzqBbBNJvX5gU8tLWxZaeQwHwYDVR0jBBgwFoAU/4XXREzqBbBNJvX5gU8t
LWxZaeQwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOBgQA07q/rKoE7fAda
FED57/SR00OvY9wLlFEF2QJ5OLu+O33YUXDDbGpfUSF9R8l0g9dix1JbWK9nQ6Yd
R/KCo6D0sw0ZgeQv1aUXbl/xJ9k4jlTxmWbPeiiPZEqU1W9wN5lkGuLxV4CEGTKU
hJA/yXa1wbwIPGvX3tVKdOEWPRXZLg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB7jCCAVegAwIBAgIJAJWA0jQ4o9DGMA0GCSqGSIb3DQEBBQUAMA8xDTALBgNV
BAMMBHg2MXMwIBcNMTExMTI0MTAyNDQzWhgPMjExMTEwMzExMDI0NDNaMA8xDTAL
BgNVBAMMBHg2MXMwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANdJNiFsRlkH
vq2kHP2zdxEyzPAWZH3CQ3Myb3F8hERXTIFSUqntPXDKXDb7Y/laqjMXdj+vptKk
3Q36J+8VnJbSwjGwmEG6tym9qMSGIPPNw1JXY1R29eF3o4aj21o7DHAkhuNc5Tso
67fUSKgvyVnyH4G6ShQUAtghPaAwS0KvAgMBAAGjUDBOMB0GA1UdDgQWBBSjxFUE
RfnTvABRLAa34Ytkhz5vPzAfBgNVHSMEGDAWgBSjxFUERfnTvABRLAa34Ytkhz5v
PzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4GBAFLDS7zNhlrQYSQO5KIj
z2RJe3fj4rLPklo3TmP5KLvendG+LErE2cbKPqnhQ2oVoj6u9tWVwo/g03PMrrnL
KrDm39slYD/1KoE5kB4l/p6KVOdeJ4I6xcgu9rnkqqHzDwI4v7e8/D3WZbpiFUsY
vaZhjNYKWQf79l6zXfOvphzJ
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAO4V/jiMoICoMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMjMyMCAXDTEyMDIxNjExMTAyM1oYDzIxMTIwMTIzMTExMDIzWjAT
......@@ -120,17 +173,43 @@ signature-certificate-list =
If1a2ZoqHRxoNo2yTmm7TSYRORWVS+vvfjY=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB4DCCAUkCADANBgkqhkiG9w0BAQsFADA5MQswCQYDVQQGEwJGUjEZMBcGA1UE
CBMQRGVmYXVsdCBQcm92aW5jZTEPMA0GA1UEChMGTmV4ZWRpMB4XDTExMDkxNTA5
MDAwMloXDTEyMDkxNTA5MDAwMlowOTELMAkGA1UEBhMCRlIxGTAXBgNVBAgTEERl
ZmF1bHQgUHJvdmluY2UxDzANBgNVBAoTBk5leGVkaTCBnzANBgkqhkiG9w0BAQEF
AAOBjQAwgYkCgYEApYZv6OstoqNzxG1KI6iE5U4Ts2Xx9lgLeUGAMyfJLyMmRLhw
boKOyJ9Xke4dncoBAyNPokUR6iWOcnPHtMvNOsBFZ2f7VA28em3+E1JRYdeNUEtX
Z0s3HjcouaNAnPfjFTXHYj4um1wOw2cURSPuU5dpzKBbV+/QCb5DLheynisCAwEA
ATANBgkqhkiG9w0BAQsFAAOBgQBCZLbTVdrw3RZlVVMFezSHrhBYKAukTwZrNmJX
mHqi2tN8tNo6FX+wmxUUAf3e8R2Ymbdbn2bfbPpcKQ2fG7PuKGvhwMG3BlF9paEC
q7jdfWO18Zp/BG7tagz0jmmC4y/8akzHsVlruo2+2du2freE8dK746uoMlXlP93g
QUUGLQ==
MIIB9jCCAV+gAwIBAgIJAIlBksrZVkK8MA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMzU3MCAXDTEyMDEyNjEwNTUyOFoYDzIxMTIwMTAyMTA1NTI4WjAT
MREwDwYDVQQDDAhDT01QLTM1NzCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
ts+iGUwi44vtIfwXR8DCnLtHV4ydl0YTK2joJflj0/Ws7mz5BYkxIU4fea/6+VF3
i11nwBgYgxQyjNztgc9u9O71k1W5tU95yO7U7bFdYd5uxYA9/22fjObaTQoC4Nc9
mTu6r/VHyJ1yRsunBZXvnk/XaKp7gGE9vNEyJvPn2bkCAwEAAaNQME4wHQYDVR0O
BBYEFKuGIYu8+6aEkTVg62BRYaD11PILMB8GA1UdIwQYMBaAFKuGIYu8+6aEkTVg
62BRYaD11PILMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAMoTRpBxK
YLEZJbofF7gSrRIcrlUJYXfTfw1QUBOKkGFFDsiJpEg4y5pUk1s5Jq9K3SDzNq/W
it1oYjOhuGg3al8OOeKFrU6nvNTF1BAvJCl0tr3POai5yXyN5jlK/zPfypmQYxE+
TaqQSGBJPVXYt6lrq/PRD9ciZgKLOwEqK8w=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAPHoWu90gbsgMA0GCSqGSIb3DQEBBQUAMBQxEjAQBgNV
BAMMCXZpZmlibm9kZTAeFw0xMjAzMTkyMzIwNTVaFw0xMzAzMTkyMzIwNTVaMBQx
EjAQBgNVBAMMCXZpZmlibm9kZTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
ozBijpO8PS5RTeKTzA90vi9ezvv4vVjNaguqT4UwP9+O1+i6yq1Y2W5zZxw/Klbn
oudyNzie3/wqs9VfPmcyU9ajFzBv/Tobm3obmOqBN0GSYs5fyGw+O9G3//6ZEhf0
NinwdKmrRX+d0P5bHewadZWIvlmOupcnVJmkks852BECAwEAAaNQME4wHQYDVR0O
BBYEFF9EtgfZZs8L2ZxBJxSiY6eTsTEwMB8GA1UdIwQYMBaAFF9EtgfZZs8L2ZxB
JxSiY6eTsTEwMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAc43YTfc6
baSemaMAc/jz8LNLhRE5dLfLOcRSoHda8y0lOrfe4lHT6yP5l8uyWAzLW+g6s3DA
Yme/bhX0g51BmI6gjKJo5DoPtiXk/Y9lxwD3p7PWi+RhN+AZQ5rpo8UfwnnN059n
yDuimQfvJjBFMVrdn9iP6SfMjxKaGk6gVmI=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAMNZBmoIOXPBMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMTMyMCAXDTEyMDUwMjEyMDQyNloYDzIxMTIwNDA4MTIwNDI2WjAT
MREwDwYDVQQDDAhDT01QLTEzMjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
6peZQt1sAmMAmSG9BVxxcXm8x15kE9iAplmANYNQ7z2YO57c10jDtlYlwVfi/rct
xNUOKQtc8UQtV/fJWP0QT0GITdRz5X/TkWiojiFgkopza9/b1hXs5rltYByUGLhg
7JZ9dZGBihzPfn6U8ESAKiJzQP8Hyz/o81FPfuHCftsCAwEAAaNQME4wHQYDVR0O
BBYEFNuxsc77Z6/JSKPoyloHNm9zF9yqMB8GA1UdIwQYMBaAFNuxsc77Z6/JSKPo
yloHNm9zF9yqMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAl4hBaJy1
cgiNV2+Z5oNTrHgmzWvSY4duECOTBxeuIOnhql3vLlaQmo0p8Z4c13kTZq2s3nhd
Loe5mIHsjRVKvzB6SvIaFUYq/EzmHnqNdpIGkT/Mj7r/iUs61btTcGUCLsUiUeci
Vd0Ozh79JSRpkrdI8R/NRQ2XPHAo+29TT70=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAKRvzcy7OH0UMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
......
software/apache-frontend/templates/000.conf.in 0 → 100644
View file @ d007050d
<VirtualHost *:{{ https_port }}>
ServerName www.example.org
SSLEngine on
SSLProxyEngine on
SSLProtocol -ALL +SSLv3 +TLSv1
SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!ADH
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
ErrorDocument 404 /notfound.html
</VirtualHost>
<VirtualHost *:{{ http_port }}>
ServerName www.example.org
ErrorDocument 404 /notfound.html
</VirtualHost>
\ No newline at end of file
software/apache-frontend/templates/apache-custom-slave-list.cfg.in 0 → 100644
View file @ d007050d
{% if software_type == slap_software_type -%}
{% set cached_server_dict = {} -%}
{% set part_list = [] -%}
{% set cache_access = "http://%s:%s" % (local_ipv4, cache_port) -%}
{% set generic_instance_parameter_dict = {'cache_access': cache_access,} -%}
{% if extra_slave_instance_list -%}
{% set slave_instance_information_list = []-%}
{% set slave_instance_list = slave_instance_list + json_module.loads(extra_slave_instance_list) -%}
{% endif -%}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = {{ apache_configuration_directory }}/${:filename}
extra-context =
context =
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
${:extra-context}
# Loop trhought slave list to set up slaves
{% for slave_instance in slave_instance_list -%}
{% set slave_reference = slave_instance.get('slave_reference') -%}
{% set slave_section_title = 'dynamic-template-slave-instance-%s' % slave_reference -%}
{% set slave_parameter_dict = generic_instance_parameter_dict.copy() -%}
{% do part_list.append(slave_section_title) -%}
# Set Up log files
{% do slave_parameter_dict.__setitem__('access_log', '/'.join([apache_log_directory, '%s_access_log' % slave_reference])) -%}
{% do slave_parameter_dict.__setitem__('error_log', '/'.join([apache_log_directory, '%s_error_log' % slave_reference])) -%}
# Set up apache configuration file for slave
[{{ slave_section_title }}]
< = jinja2-template-base
template = {{ template_slave_configuration }}
filename = {{ '%s.conf' % slave_reference }}
extra-context =
key apache_custom_https {{ 'slave-instance-%s-configuration:apache_custom_https' % slave_reference }}
key apache_custom_http {{ 'slave-instance-%s-configuration:apache_custom_http' % slave_reference }}
raw https_port {{ https_port }}
raw http_port {{ http_port }}
{{ '\n' }}
# Set ssl certificates for each slave
{% for cert_name in ('ssl_key', 'ssl_crt', 'ssl_ca_crt', 'ssl_csr')-%}
{% if cert_name in slave_instance -%}
{% set cert_title = '%s-%s' % (slave_reference, cert_name.replace('ssl_', '')) -%}
{% set cert_file = '/'.join([custom_ssl_directory, cert_title.replace('-','.')]) -%}
{% do part_list.append(cert_title) -%}
{% do slave_parameter_dict.__setitem__(cert_name, cert_file) -%}
# Store certificates on fs
[{{ cert_title }}]
< = jinja2-template-base
template = {{ empty_template }}
rendered = {{ cert_file }}
extra-context =
key content {{ cert_title + '-config:value' }}
# Store certificate in config
[{{ cert_title + '-config' }}]
value = {{ dumps(slave_instance.get(cert_name)) }}
{% endif -%}
{% endfor -%}
# Set apache configuration value for slave
[{{ ('slave-instance-%s-configuration' % slave_reference) }}]
{% set apache_custom_http = ((slave_instance.get('apache_custom_http', '')) % slave_parameter_dict) -%}
{% set apache_custom_https = ((slave_instance.get('apache_custom_https', '')) % slave_parameter_dict) -%}
apache_custom_http = {{ dumps(apache_custom_http) }}
apache_custom_https = {{ dumps(apache_custom_https) }}
{{ '\n' }}
# The slave use cache
{% if 'enable_cache' in slave_instance and 'url' in slave_instance and 'domain' in slave_instance -%}
{% do cached_server_dict.__setitem__(slave_instance.get('domain'), slave_instance.get('url')) -%}
{% endif -%}
# Publish slave information
{% if not extra_slave_instance_list -%}
{% set publish_section_title = 'publish-%s-connection-information' % slave_instance.get('slave_reference') -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
public-ipv4 = {{ public_ipv4 }}
-slave-reference = {{ slave_instance.get('slave_reference') }}
{% else -%}
{% do slave_instance_information_list.append({'slave-reference':slave_instance.get('slave_reference'), 'public-ipv4':public_ipv4}) -%}
{% endif -%}
{% endfor -%}
# Publish information for the instance
{% set publish_section_title = 'publish-apache-information' -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
public-ipv4 = {{ public_ipv4 }}
private-ipv4 = {{ local_ipv4 }}
{% if extra_slave_instance_list -%}
slave-instance-information-list = {{ json_module.dumps(slave_instance_information_list) }}
{% endif -%}
{% do part_list.append('cached-rewrite-rules') -%}
[cached-rewrite-rules]
< = jinja2-template-base
template = {{ template_rewrite_cached }}
rendered = {{ rewrite_cached_configuration }}
extra-context =
import json_module json
key server_dict rewrite-rules:rules
[rewrite-rules]
rules = {{ dumps(cached_server_dict) }}
[buildout]
parts +=
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
cache-access = {{ cache_access }}
{% endif -%}
software/apache-frontend/templates/apache-default-slave-list.cfg.in 0 → 100644
View file @ d007050d
{% if slap_software_type in software_type -%}
{% set cached_server_dict = {} -%}
{% set part_list = [] -%}
{% set cache_access = "http://%s:%s" % (local_ipv4, cache_port) -%}
{% set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
{% set generic_instance_parameter_dict = {'cache_access': cache_access,} -%}
{% if extra_slave_instance_list -%}
{% set slave_instance_information_list = []-%}
{% set slave_instance_list = slave_instance_list + json_module.loads(extra_slave_instance_list) -%}
{% endif -%}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = {{ apache_configuration_directory }}/${:filename}
extra-context =
context =
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
${:extra-context}
# Go throught slave list to set their configuration
{% for slave_instance in slave_instance_list -%}
{% set slave_reference = slave_instance.get('slave_reference') -%}
{% set slave_section_title = 'dynamic-template-slave-instance-%s' % slave_reference -%}
{% set slave_parameter_dict = generic_instance_parameter_dict.copy() -%}
# Set slave domain if none was defined
{% if slave_instance.get('domain', None) == None -%}
# Backward compatibility
{% if slave_instance.get('custom_domain', None) != None -%}
{% do slave_instance.__setitem__('domain', slave_instance.get('custom_domain') )-%}
{% else -%}
{% do slave_instance.__setitem__('domain', "%s.%s" % (slave_instance.get('slave_reference').replace("-", "").lower(), domain)) -%}
{% endif -%}
{% endif -%}
# Set personal log, two per slave
{% set access_log = '/'.join([apache_log_directory, '%s_access_log' % slave_reference]) -%}
{% set error_log = '/'.join([apache_log_directory, '%s_error_log' % slave_reference]) -%}
# The slave use cache
# Next line is forbidden and people who copy it will be hanged short
{% set enable_cache = ('' ~ slave_instance.get('enable_cache', '')).lower() in TRUE_VALUES -%}
{% if enable_cache -%}
{% do cached_server_dict.__setitem__(slave_instance.get('domain'), slave_instance.get('url')) -%}
{% do slave_instance.__setitem__('url', cache_access) -%}
{% endif -%}
{% do part_list.append(slave_section_title) -%}
# Set up slave configuration file
[{{ slave_section_title }}]
< = jinja2-template-base
template = {{ template_slave_configuration }}
filename = {{ '%s.conf' % slave_reference }}
extensions = jinja2.ext.do
extra-context =
section slave_parameter {{ 'slave-instance-%s-configuration' % slave_reference }}
raw domain {{ domain }}
raw https_port {{ https_port }}
raw http_port {{ http_port }}
raw access_log {{ access_log }}
raw error_log {{ error_log }}
{{ '\n' }}
# Set ssl certificates for each slave
{% for cert_name in ('ssl_key', 'ssl_crt', 'ssl_ca_crt', 'ssl_csr')-%}
{% if cert_name in slave_instance -%}
{% set cert_title = '%s-%s' % (slave_reference, cert_name.replace('ssl_', '')) -%}
{% set cert_file = '/'.join([custom_ssl_directory, cert_title.replace('-','.')]) -%}
{% do part_list.append(cert_title) -%}
{% do slave_instance.__setitem__('path_to_' ~ cert_name, cert_file) -%}
# Store certificates on fs
[{{ cert_title }}]
< = jinja2-template-base
template = {{ empty_template }}
rendered = {{ cert_file }}
extra-context =
key content {{ cert_title + '-config:value' }}
# Store certificate in config
[{{ cert_title + '-config' }}]
value = {{ dumps(slave_instance.get(cert_name)) }}
{% endif -%}
{% endfor -%}
# Set apache configuration for slave
[{{ ('slave-instance-%s-configuration' % slave_reference) }}]
{% for key, value in slave_instance.iteritems() -%}
{{ key }} = {{ dumps(value) }}
{% endfor %}
# Publish slave information
{% if not extra_slave_instance_list -%}
{% set publish_section_title = 'publish-%s-connection-information' % slave_instance.get('slave_reference') -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
-slave-reference = {{ slave_instance.get('slave_reference') }}
public-ipv4 = {{ public_ipv4 }}
domain = {{ slave_instance.get('domain') }}
url = http://{{ slave_instance.get('domain') }}
# Backward compatibility
site_url = ${:url}
{% else -%}
{% do slave_instance_information_list.append({'slave-reference':slave_instance.get('slave_reference'), 'public-ipv4':public_ipv4, 'domain':slave_instance.get('domain'), 'url':"http://%s" % slave_instance.get('domain'), 'site_url':"http://%s" % slave_instance.get('domain')}) -%}
{% endif -%}
{% endfor -%}
# Publish information for the instance
{% set publish_section_title = 'publish-apache-information' -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
public-ipv4 = {{ public_ipv4 }}
private-ipv4 = {{ local_ipv4 }}
domain = {{ domain }}
{% if extra_slave_instance_list -%}
slave-instance-information-list = {{ json_module.dumps(slave_instance_information_list) }}
{% endif -%}
{% do part_list.append('cached-rewrite-rules') -%}
# Set rewrite rules for second apache
[cached-rewrite-rules]
< = jinja2-template-base
template = {{ template_rewrite_cached }}
rendered = {{ rewrite_cached_configuration }}
extra-context =
import json_module json
key server_dict rewrite-rules:rules
# Store Rewrite rules for second apache
[rewrite-rules]
rules = {{ dumps(cached_server_dict) }}
# Add parts generated by template
[buildout]
parts +=
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
cache-access = {{ cache_access }}
{% endif -%}
software/apache-frontend/templates/apache.conf.in 0 → 100644
View file @ d007050d
# Apache configuration file for Zope
# Automatically generated
# Basic server configuration
PidFile "{{ pid_file }}"
ServerName {{ domain }}
DocumentRoot {{ document_root }}
ServerRoot {{ instance_home }}
{% for ip in (ipv4_addr, "[%s]" % ipv6_addr) -%}
{% for port in (http_port, https_port) -%}
{{ "Listen %s:%s" % (ip, port) }}
{% endfor -%}
{% endfor -%}
ServerAdmin {{ server_admin }}
DefaultType text/plain
TypesConfig {{ httpd_home }}/conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
# As backend is trusting REMOTE_USER header unset it always
RequestHeader unset REMOTE_USER
ServerTokens Prod
# Log configuration
ErrorLog "{{ error_log }}"
LogLevel info
# LogFormat "%h %{REMOTE_USER}i %{Host}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
# LogFormat "%h %{REMOTE_USER}i %{Host}i %l %u %t \"%r\" %>s %b" common
# CustomLog "{{ access_log }}" common
LogFormat "%h %l %{REMOTE_USER}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog "{{ access_log }}" combined
<Directory {{ protected_path }}>
Order Deny,Allow
Allow from {{ access_control_string }}
</Directory>
<Directory {{ document_root }}>
Order Allow,Deny
Allow from All
</Directory>
# List of modules
#LoadModule unixd_module modules/mod_unixd.so
#LoadModule access_compat_module modules/mod_access_compat.so
#LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_host_module {{ httpd_home }}/modules/mod_authz_host.so
LoadModule log_config_module {{ httpd_home }}/modules/mod_log_config.so
LoadModule deflate_module {{ httpd_home }}/modules/mod_deflate.so
LoadModule setenvif_module {{ httpd_home }}/modules/mod_setenvif.so
LoadModule version_module {{ httpd_home }}/modules/mod_version.so
LoadModule proxy_module {{ httpd_home }}/modules/mod_proxy.so
LoadModule proxy_http_module {{ httpd_home }}/modules/mod_proxy_http.so
LoadModule ssl_module {{ httpd_home }}/modules/mod_ssl.so
LoadModule mime_module {{ httpd_home }}/modules/mod_mime.so
LoadModule dav_module {{ httpd_home }}/modules/mod_dav.so
LoadModule dav_fs_module {{ httpd_home }}/modules/mod_dav_fs.so
LoadModule negotiation_module {{ httpd_home }}/modules/mod_negotiation.so
LoadModule rewrite_module {{ httpd_home }}/modules/mod_rewrite.so
LoadModule headers_module {{ httpd_home }}/modules/mod_headers.so
LoadModule cache_module {{ httpd_home }}/modules/mod_cache.so
LoadModule mem_cache_module {{ httpd_home }}/modules/mod_mem_cache.so
LoadModule antiloris_module {{ httpd_home }}/modules/mod_antiloris.so
# The following directives modify normal HTTP response behavior to
# handle known problems with browser implementations.
BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch ".*MSIE.*" nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4\.0" force-response-1.0
BrowserMatch "Java/1\.0" force-response-1.0
BrowserMatch "JDK/1\.0" force-response-1.0
# The following directive disables redirects on non-GET requests for
# a directory that does not include the trailing slash. This fixes a
# problem with Microsoft WebFolders which does not appropriately handle
# redirects for folders with DAV methods.
# Same deal with Apple's DAV filesystem and Gnome VFS support for DAV.
BrowserMatch "Microsoft Data Access Internet Publishing Provider" redirect-carefully
BrowserMatch "MS FrontPage" redirect-carefully
BrowserMatch "^WebDrive" redirect-carefully
BrowserMatch "^WebDAVFS/1.[0123]" redirect-carefully
BrowserMatch "^gnome-vfs" redirect-carefully
BrowserMatch "^XML Spy" redirect-carefully
BrowserMatch "^Dreamweaver-WebDAV-SCM1" redirect-carefully
# Cache directives
CacheEnable mem /
CacheDefaultExpire 3600
MCacheSize 8192
MCacheMaxObjectCount 1000
MCacheMaxObjectSize 8192
MCacheRemovalAlgorithm LRU
# Deflate
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript application/x-javascript application/javascript
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
# SSL Configuration
SSLCertificateFile {{ login_certificate }}
SSLCertificateKeyFile {{ login_key }}
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLSessionCache shmcb:/{{ httpd_mod_ssl_cache_directory }}/ssl_scache(512000)
SSLSessionCacheTimeout 300
SSLRandomSeed startup /dev/urandom 256
SSLRandomSeed connect builtin
SSLProtocol -ALL +SSLv3 +TLSv1
SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!ADH
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
# Accept proxy to sites using self-signed SSL certificates
SSLProxyCheckPeerCN off
SSLProxyCheckPeerExpire off
NameVirtualHost *:{{ http_port }}
NameVirtualHost *:{{ https_port }}
include {{ slave_configuration_directory }}/*.conf
\ No newline at end of file
slapos/recipe/apache_frontend/template/apache.conf.in → software/apache-frontend/templates/apache_cached.conf.in
View file @ d007050d
......@@ -2,16 +2,16 @@
# Automatically generated
# Basic server configuration
PidFile "%(pid_file)s"
ServerName %(server_name)s
DocumentRoot %(document_root)s
ServerRoot %(instance_home)s
PidFile "{{ pid_file }}"
ServerName {{ domain }}
DocumentRoot {{ document_root }}
ServerRoot {{ instance_home }}
%(listen)s
{{ "Listen %s:%s" % (ipv4_addr, cached_port) }}
ServerAdmin %(server_admin)s
ServerAdmin {{ server_admin }}
DefaultType text/plain
TypesConfig %(httpd_home)s/conf/mime.types
TypesConfig {{ httpd_home }}/conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
......@@ -21,35 +21,45 @@ RequestHeader unset REMOTE_USER
ServerTokens Prod
# Log configuration
ErrorLog "%(error_log)s"
ErrorLog "{{ error_log }}"
LogLevel info
LogFormat "%%h %%{REMOTE_USER}i %%{Host}i %%l %%u %%t \"%%r\" %%>s %%b \"%%{Referer}i\" \"%%{User-Agent}i\"" combined
LogFormat "%%h %%{REMOTE_USER}i %%{Host}i %%l %%u %%t \"%%r\" %%>s %%b" common
CustomLog "%(access_log)s" common
%(path_enable)s
# LogFormat "%h %{REMOTE_USER}i %{Host}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
# LogFormat "%h %{REMOTE_USER}i %{Host}i %l %u %t \"%r\" %>s %b" common
# CustomLog "{{ access_log }}" common
LogFormat "%h %l %{REMOTE_USER}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog "{{ access_log }}" combined
<Directory {{ protected_path }}>
Order Deny,Allow
Allow from {{ access_control_string }}
</Directory>
<Directory {{ document_root }}>
Order Allow,Deny
Allow from All
</Directory>
# List of modules
#LoadModule unixd_module modules/mod_unixd.so
#LoadModule access_compat_module modules/mod_access_compat.so
#LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_host_module %(httpd_home)s/modules/mod_authz_host.so
LoadModule log_config_module %(httpd_home)s/modules/mod_log_config.so
LoadModule deflate_module %(httpd_home)s/modules/mod_deflate.so
LoadModule setenvif_module %(httpd_home)s/modules/mod_setenvif.so
LoadModule version_module %(httpd_home)s/modules/mod_version.so
LoadModule proxy_module %(httpd_home)s/modules/mod_proxy.so
LoadModule proxy_http_module %(httpd_home)s/modules/mod_proxy_http.so
LoadModule ssl_module %(httpd_home)s/modules/mod_ssl.so
LoadModule mime_module %(httpd_home)s/modules/mod_mime.so
LoadModule dav_module %(httpd_home)s/modules/mod_dav.so
LoadModule dav_fs_module %(httpd_home)s/modules/mod_dav_fs.so
LoadModule negotiation_module %(httpd_home)s/modules/mod_negotiation.so
LoadModule rewrite_module %(httpd_home)s/modules/mod_rewrite.so
LoadModule headers_module %(httpd_home)s/modules/mod_headers.so
LoadModule cache_module %(httpd_home)s/modules/mod_cache.so
LoadModule mem_cache_module %(httpd_home)s/modules/mod_mem_cache.so
LoadModule antiloris_module %(httpd_home)s/modules/mod_antiloris.so
LoadModule authz_host_module {{ httpd_home }}/modules/mod_authz_host.so
LoadModule log_config_module {{ httpd_home }}/modules/mod_log_config.so
LoadModule deflate_module {{ httpd_home }}/modules/mod_deflate.so
LoadModule setenvif_module {{ httpd_home }}/modules/mod_setenvif.so
LoadModule version_module {{ httpd_home }}/modules/mod_version.so
LoadModule proxy_module {{ httpd_home }}/modules/mod_proxy.so
LoadModule proxy_http_module {{ httpd_home }}/modules/mod_proxy_http.so
LoadModule ssl_module {{ httpd_home }}/modules/mod_ssl.so
LoadModule mime_module {{ httpd_home }}/modules/mod_mime.so
LoadModule dav_module {{ httpd_home }}/modules/mod_dav.so
LoadModule dav_fs_module {{ httpd_home }}/modules/mod_dav_fs.so
LoadModule negotiation_module {{ httpd_home }}/modules/mod_negotiation.so
LoadModule rewrite_module {{ httpd_home }}/modules/mod_rewrite.so
LoadModule headers_module {{ httpd_home }}/modules/mod_headers.so
LoadModule cache_module {{ httpd_home }}/modules/mod_cache.so
LoadModule mem_cache_module {{ httpd_home }}/modules/mod_mem_cache.so
LoadModule antiloris_module {{ httpd_home }}/modules/mod_antiloris.so
# The following directives modify normal HTTP response behavior to
# handle known problems with browser implementations.
......@@ -85,14 +95,28 @@ AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javasc
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent
# SSL Configuration
%(ssl_snippet)s
SSLCertificateFile {{ login_certificate }}
SSLCertificateKeyFile {{ login_key }}
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLSessionCache shmcb:/{{ httpd_mod_ssl_cache_directory }}/ssl_scache(512000)
SSLSessionCacheTimeout 300
SSLRandomSeed startup /dev/urandom 256
SSLRandomSeed connect builtin
SSLProtocol -ALL +SSLv3 +TLSv1
SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!ADH
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
# Accept proxy to sites using self-signed SSL certificates
SSLProxyCheckPeerCN off
SSLProxyCheckPeerExpire off
<VirtualHost *:%(https_port)s>
SSLEngine on
# Only accept generic (i.e not Zope) backends on http
<VirtualHost *:{{ cached_port }}>
SSLProxyEngine on
# Rewrite part
ProxyVia On
......@@ -100,68 +124,10 @@ Header append Vary User-Agent
ProxyTimeout 600
RewriteEngine On
# Include configuration file not operated by slapos. This file won't be erased
# or changed when slapgrid is ran. It can be freely customized by node admin.
Include %(custom_apache_virtualhost_conf)s
# Define the 3 RewriteMaps (key -> value store): one for Zope, one generic,
# one generic https only,
# containing: rewritten URL -> original URL (a.k.a VirtualHostBase in Zope)
RewriteMap apachemapzope txt:%(apachemapzope_path)s
RewriteMap apachemapgeneric txt:%(apachemap_path)s
RewriteMap apachemapgenerichttpsonly txt:%(apachemap_httpsonly_path)s
# Define another RewriteMap for Zope, containing:
# rewritten URL -> VirtualHostRoot
RewriteMap apachemapzopepath txt:%(apachemapzopepath_path)s
# First, we check if we have a zope backend server
# If so, let's use Virtual Host Daemon rewrite
RewriteCond ${apachemapzope:%%{SERVER_NAME}} >""
# We suppose that Apache listens to 443 (even indirectly thanks to things like iptables)
RewriteRule ^/(.*)$ ${apachemapzope:%%{SERVER_NAME}}/VirtualHostBase/https/%%{SERVER_NAME}:443/${apachemapzopepath:%%{SERVER_NAME}}/VirtualHostRoot/$1 [L,P]
# If we have generic backend server, let's rewrite without virtual host daemon
RewriteCond ${apachemapgeneric:%%{SERVER_NAME}} >""
# We suppose that Apache listens to 443 (even indirectly thanks to things like iptables)
RewriteRule ^/(.*)$ ${apachemapgeneric:%%{SERVER_NAME}}/$1 [L,P]
# Same for https only server
RewriteCond ${apachemapgenerichttpsonly:%%{SERVER_NAME}} >""
# We suppose that Apache listens to 443 (even indirectly thanks to things like iptables)
RewriteRule ^/(.*)$ ${apachemapgenerichttpsonly:%%{SERVER_NAME}}/$1 [L,P]
RewriteMap apachemapcached txt:{{ apachecachedmap_path }}
RewriteCond ${apachemapcached:%{SERVER_NAME}} >""
RewriteRule ^/(.*)$ ${apachemapcached:%{SERVER_NAME}}/$1 [L,P]
# If nothing exist : put a nice error
ErrorDocument 404 /notfound.html
</VirtualHost>
# Only accept generic (i.e not Zope) backends on http
<VirtualHost *:%(plain_http_port)s>
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Include configuration file not operated by slapos. This file won't be erased
# or changed when slapgrid is ran. It can be freely customized by node admin.
Include %(custom_apache_virtualhost_conf)s
RewriteMap apachemapgeneric txt:%(apachemap_path)s
RewriteCond ${apachemapgeneric:%%{SERVER_NAME}} >""
RewriteRule ^/(.*)$ ${apachemapgeneric:%%{SERVER_NAME}}/$1 [L,P]
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteRule ^/(.*)$ https://%%{SERVER_NAME}%%{REQUEST_URI}
</VirtualHost>
# Include configuration file not operated by slapos. This file won't be erased
# or changed when slapgrid is ran. It can be freely customized by node admin.
Include %(custom_apache_conf)s
software/apache-frontend/templates/apache_cached_rewrite.txt.in 0 → 100644
View file @ d007050d
{% for server_tuple in server_dict.items() -%}
{{ "%s %s" % server_tuple }}
{% endfor -%}
software/apache-frontend/templates/default-virtualhost.conf.in 0 → 100644
View file @ d007050d
{% set TRUE_VALUES = ['y', 'yes', '1', 'true'] -%}
<VirtualHost *:{{ https_port }}>
ServerName {{ slave_parameter.get('domain') }}
ServerAlias {{ slave_parameter.get('domain') }}
SSLEngine on
SSLProxyEngine on
SSLProtocol -ALL +SSLv3 +TLSv1
SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!ADH
{% set ssl_configuration_list = [('SSLCertificateFile', 'path_to_ssl_crt'),
('SSLCertificateKeyFile', 'path_to_ssl_key'),
('SSLCACertificateFile', 'path_to_ssl_ca_crt'),
('SSLCertificateChainFile', 'path_to_ssl_ca_crt')] -%}
{% for key, value in ssl_configuration_list -%}
{% if value in slave_parameter -%}
{{ ' %s' % key }} {{ slave_parameter.get(value) }}
{% endif -%}
{% endfor -%}
# One Slave two logs
ErrorLog "{{ error_log }}"
LogLevel info
LogFormat "%h %l %{REMOTE_USER}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog "{{ access_log }}" combined
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
{% if slave_parameter.get('type', '') == 'zope' -%}
# First, we check if we have a zope backend server
# If so, let's use Virtual Host Daemon rewrite
# We suppose that Apache listens to 443 (even indirectly thanks to things like iptables)
RewriteRule ^/(.*)$ {{ slave_parameter.get('url', '') }}/VirtualHostBase/https/{{ slave_parameter.get('domain', '') }}:443/{{ slave_parameter.get('path', '') }}/VirtualHostRoot/$1 [L,P]
{% else -%}
RewriteRule ^/(.*)$ {{ slave_parameter.get('url', '') }}/$1 [L,P]
{% endif -%}
</VirtualHost>
<VirtualHost *:{{ http_port }}>
ServerName {{ slave_parameter.get('domain') }}
ServerAlias {{ slave_parameter.get('domain') }}
SSLProxyEngine on
# Rewrite part
ProxyVia On
ProxyPreserveHost On
ProxyTimeout 600
RewriteEngine On
# One Slave two logs
ErrorLog "{{ error_log }}"
LogLevel info
LogFormat "%h %l %{REMOTE_USER}i %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog "{{ access_log }}" combined
# Remove "Secure" from cookies, as backend may be https
Header edit Set-Cookie "(?i)^(.+);secure$" "$1"
# Next line is forbidden and people who copy it will be hanged short
{% set https_only = ('' ~ slave_parameter.get('https-only', '')).lower() in TRUE_VALUES -%}
{% if https_only -%}
# Not using HTTPS? Ask that guy over there.
# Dummy redirection to https. Note: will work only if https listens
# on standard port (443).
RewriteCond %{SERVER_PORT} !^{{ https_port }}$
RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [NC,R,L]
{% elif slave_parameter.get('type', '') == 'zope' -%}
# First, we check if we have a zope backend server
# If so, let's use Virtual Host Daemon rewrite
# We suppose that Apache listens to 80 (even indirectly thanks to things like iptables)
RewriteRule ^/(.*)$ {{ slave_parameter.get('url', '') }}/VirtualHostBase/http/{{ slave_parameter.get('domain', '') }}:80/{{ slave_parameter.get('path', '') }}/VirtualHostRoot/$1 [L,P]
{% else -%}
RewriteRule ^/(.*)$ {{ slave_parameter.get('url', '') }}/$1 [L,P]
{% endif -%}
# If nothing exist : put a nice error
# ErrorDocument 404 /notfound.html
# Dadiboom
</VirtualHost>
software/apache-frontend/templates/empty.in 0 → 100644
View file @ d007050d
{{ content }}
\ No newline at end of file
software/apache-frontend/templates/replicate-publish-slave-information.cfg.in 0 → 100644
View file @ d007050d
{% set part_list = [] -%}
{% set slave_information_dict = {} -%}
# regroup slave information from all frontends
{%- for frontend, slave_list_raw in slave_information.iteritems() -%}
{% set slave_list = json_module.loads(slave_list_raw) -%}
{% for slave_dict in slave_list -%}
{% set slave_reference = slave_dict.pop('slave-reference') %}
{% set current_slave_dict = slave_information_dict.get(slave_reference, {}) %}
{% do current_slave_dict.update(slave_dict) -%}
{% do current_slave_dict.__setitem__(
'replication_number',
current_slave_dict.get('replication_number', 0) + 1
) -%}
{% do slave_information_dict.__setitem__(slave_reference, current_slave_dict) -%}
{% endfor -%}
{% endfor %}
# Publish information for each slave
{% for slave_reference, slave_information in slave_information_dict.iteritems() %}
{% set publish_section_title = 'publish-%s' % slave_reference -%}
{% do part_list.append(publish_section_title) -%}
[{{ publish_section_title }}]
recipe = slapos.cookbook:publish
-slave-reference = {{ slave_reference }}
{% for key, value in slave_information.iteritems() -%}
{{ key }} = {{ value }}
{% endfor -%}
{% endfor %}
[buildout]
parts =
{% for part in part_list %}
{{ ' %s' % part }}
{% endfor %}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
\ No newline at end of file
software/apache-frontend/templates/slave-virtualhost.conf.in 0 → 100644
View file @ d007050d
<VirtualHost *:{{ https_port }}>
{{ apache_custom_https }}
</VirtualHost>
<VirtualHost *:{{ http_port }}>
{{ apache_custom_http }}
</VirtualHost>
software/backupserver/instance-pullrdiffbackup.cfg.in
View file @ d007050d
......@@ -12,7 +12,9 @@ etc = $${buildout:directory}/etc
bin = $${buildout:directory}/bin
srv = $${buildout:directory}/srv
var = $${buildout:directory}/var
run = $${:var}/run
log = $${:var}/log
varnginx = $${:var}/nginx
# scripts = $${:etc}/run
services = $${:etc}/service
cron-entries = $${:etc}/cron.d
......@@ -20,8 +22,11 @@ cron-lines = $${:etc}/cron.lines
crontabs = $${:etc}/crontabs
cronstamps = $${:etc}/cronstamps
backup = $${:srv}/backup
status = $${:srv}/status
backupscript = $${:etc}/backup
www = $${:srv}/www
home = $${:etc}/home
ssl = $${:etc}/ssl
ssh = $${:home}/.ssh
#################################
......@@ -67,36 +72,30 @@ directory = $${directory:backup}/$${:_buildout_section_name_}
[{{ slave_reference }}-backup-private_key]
recipe = plone.recipe.command
stop-on-error = true
stop-on-error = false
command = ${dropbear-output:keygen} -t $${:type} -s 2048 -f $${:key}
key = $${directory:ssh}/$${:_buildout_section_name_}
location = $${:key}
type = rsa
[{{ slave_reference }}-backup-public_key]
recipe = plone.recipe.command
stop-on-error = true
update-command = $${:command}
command = ${coreutils-output:rm} -f $${:key} && ${dropbear-output:keygen} -y -f {{ '$${' ~ slave_reference }}-backup-private_key:key} | ${grep-output:grep} {{ '$${' ~ slave_reference }}-backup-private_key:type} > $${:key}
key = {{ '$${' ~ slave_reference }}-backup-private_key:key}.pub
location = $${:key}
[{{ slave_reference }}-backup-check-public_key]
recipe = plone.recipe.command
stop-on-error = true
update-command = $${:command}
command = grep ssh-{{ '$${' ~ slave_reference }}-backup-private_key:type} {{ '$${' ~ slave_reference }}-backup-public_key:key}
# Insert as a beginning part, to ensure that all public keys are generated before trying to publish. This will reduce the number of slapgrid-cp run.
{% do part_list.insert(0, "%s-backup-public_key" % slave_reference) -%}
[{{ slave_reference }}-backup-read-public_key]
recipe = slapos.cookbook:generate.password
recipe = slapos.cookbook:readline
storage-path = {{ '$${' ~ slave_reference }}-backup-public_key:key}
bytes = 8
# Publish slave {{ slave_reference }} information
[{{ slave_reference }}-backup-publish]
recipe = slapos.cookbook:publish
-slave-reference = {{ slave_reference }}
authorized_key = {{ '$${' ~ slave_reference }}-backup-read-public_key:passwd}
authorized_key = {{ '$${' ~ slave_reference }}-backup-read-public_key:readline}
rss = https://[$${nginx-configuration:ip}]:$${nginx-configuration:port}/{{ '$${' ~ slave_reference }}-backup-script:status_name}.rss
{% do part_list.append("%s-backup-publish" % slave_reference) -%}
[{{ slave_reference }}-backup-script]
......@@ -111,6 +110,8 @@ hostname = {{ hostname }}
include = {{ include_string }}
exclude_string = {{ exclude_string }}
remote_schema = {{ remote_schema }}
status_name = {{ slave_reference }}_status.txt
status_log = $${directory:status}/$${:status_name}
[{{ slave_reference }}-backup-crontab-line]
recipe = slapos.recipe.template
......@@ -127,21 +128,64 @@ frequency = {{ frequency }}
#################################
# Generate crontab file
#################################
[update-rss-script]
recipe = slapos.recipe.template
url = ${template-update-rss-script:output}
output = $${directory:etc}/$${:_buildout_section_name_}
mode = 0700
global_rss = $${slap-connection:computer-id}-$${slap-connection:partition-id}.rss
[update-rss-crontab-line]
recipe = slapos.recipe.template
url = ${template-crontab-line:output}
output = $${directory:cron-lines}/$${:_buildout_section_name_}
mode = 0600
script = $${update-rss-script:output}
frequency = */5 * * * *
{% do crontab_line_list.append("$${update-rss-crontab-line:output}") -%}
[publish-global-rss]
recipe = slapos.cookbook:publish
rss = https://[$${nginx-configuration:ip}]:$${nginx-configuration:port}/$${update-rss-script:global_rss}
{% set crontab_line_list_string = " ".join(crontab_line_list) -%}
[activate-crontab-file]
# XXX File is never removed
recipe = plone.recipe.command
stop-on-error = true
update-command = $${:command}
command = ${coreutils-output:cat} ${template-crontab:output} {{ crontab_line_list_string }} | ${dcron-output:crontab} -c $${directory:crontabs} -
#################################
# Nginx service
#################################
[nginx-service]
recipe = slapos.recipe.template
url = ${template-nginx-service:output}
output = $${directory:services}/nginx
mode = 0700
virtual-depends =
$${nginx-configuration:ip}
[nginx-configuration]
recipe = slapos.recipe.template
url = ${template-nginx-configuration:output}
output = $${directory:etc}/nginx.cfg
mode = 0600
access_log = $${directory:log}/nginx-access.log
error_log = $${directory:log}/nginx-error.log
ip = $${slap-network-information:global-ipv6}
port = 9443
ssl_key = $${directory:ssl}/nginx.key
ssl_csr = $${directory:ssl}/nginx.csr
ssl_crt = $${directory:ssl}/nginx.crt
# Add parts generated by template
[buildout]
parts =
dcron-service
nginx-service
activate-crontab-file
publish-global-rss
{% for part in part_list -%}
{{ ' %s' % part }}
{% endfor -%}
......
software/backupserver/software.cfg
View file @ d007050d
......@@ -3,7 +3,8 @@ extends =
../../component/dash/buildout.cfg
../../component/dcron/buildout.cfg
../../component/logrotate/buildout.cfg
# ../../component/nginx/buildout.cfg
../../component/openssl/buildout.cfg
../../component/nginx/buildout.cfg
../../component/rdiff-backup/buildout.cfg
# ../../component/duplicity/buildout.cfg
# ../../component/git/buildout.cfg
......@@ -15,11 +16,13 @@ extends =
../../stack/slapos.cfg
parts =
extra-eggs
rdiff-backup
# duplicity
dcron
logrotate
# nginx
nginx
openssl
# git
# subversion
rsync
......@@ -30,6 +33,12 @@ parts =
template-crontab-line
slapos-cookbook
[extra-eggs]
recipe = zc.recipe.egg
interpreter = pythonforrssgen
eggs =
PyRSS2Gen
[networkcache]
# signature certificates of the following uploaders.
# Romain Courteaud
......@@ -163,6 +172,20 @@ signature-certificate-list =
##########################################################
# Service startup scripts and configuration files
##########################################################
[template-nginx-service]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/template-nginx-service.sh.in
md5sum = 5c94d952305552dcbeaeaeb27dd28f3b
output = ${buildout:directory}/template-nginx-service.sh.in
mode = 0644
[template-nginx-configuration]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/template-nginx.cfg.in
md5sum = c54d36f55ba71c897505ed61213e104a
output = ${buildout:directory}/template-nginx.cfg.in
mode = 0644
[template-dcron-service]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/template-dcron-service.sh.in
......@@ -173,7 +196,7 @@ mode = 0644
[template-backup-script]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/template-backup-script.sh.in
md5sum = 19c3a72913cc285bf9745ab2f869018e
md5sum = 8a076962fc4df7f154572543899328e3
output = ${buildout:directory}/template-backup-script.sh.in
mode = 0644
......@@ -191,13 +214,27 @@ md5sum = 072be0fd04896880c931d44d8eabde37
output = ${buildout:directory}/template-crontab.in
mode = 0644
[status2rss]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/status2rss.py
md5sum = 138c96e0836f2b06414b98ba2643f21c
output = ${buildout:directory}/status2rss.py
mode = 0644
[template-update-rss-script]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/template-update-rss.sh.in
md5sum = 529058c54e873ab26f7920c868b23c50
output = ${buildout:directory}/template-update-rss.sh.in
mode = 0644
##########################################################
# Buildout instance.cfg templates
##########################################################
[template-pullrdiffbackup]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-pullrdiffbackup.cfg.in
md5sum = 62c236773dadecac11eb9a47dbca9351
md5sum = 9bf3a34fa41ae6fe57b183293b3ff377
output = ${buildout:directory}/template-pullrdiffbackup.cfg
mode = 0644
......@@ -209,6 +246,7 @@ output = ${buildout:directory}/template.cfg
mode = 0644
[versions]
zc.buildout = 1.6.0-dev-SlapOS-010
rdiff-backup = 1.0.5
Jinja2 = 2.7
MarkupSafe = 0.18
......@@ -218,11 +256,12 @@ gunicorn = 17.5
itsdangerous = 0.22
meld3 = 0.6.10
plone.recipe.command = 1.1
slapos.cookbook = 0.78.3
slapos.cookbook = 0.80
slapos.recipe.build = 0.11.6
slapos.recipe.cmmi = 0.1.1
slapos.recipe.template = 2.4.2
zc.recipe.egg = 1.3.2
PyRSS2Gen = 1.1
# Required by:
# slapos.core==0.35.1
......
software/backupserver/status2rss.py 0 → 100644
View file @ d007050d
import datetime
import uuid
import PyRSS2Gen
import sys
from email.utils import parsedate_tz, mktime_tz
import base64
# Based on http://thehelpfulhacker.net/2011/03/27/a-rss-feed-for-your-crontabs/
# ### Defaults
TITLE = sys.argv[1]
LINK = sys.argv[2]
DESCRIPTION = TITLE
items = []
while 1:
try:
line = sys.stdin.readline()
except KeyboardInterrupt:
break
if not line:
break
time, desc = line.split(',', 1)
rss_item = PyRSS2Gen.RSSItem(
title = desc,
description = "%s, %s" % (time, desc),
link = LINK,
pubDate = datetime.datetime.fromtimestamp(mktime_tz(parsedate_tz(time))),
guid = PyRSS2Gen.Guid(base64.b64encode("%s, %s" % (time, desc)))
)
items.append(rss_item)
### Build the rss feed
rss_feed = PyRSS2Gen.RSS2 (
title = TITLE,
link = LINK,
description = DESCRIPTION,
lastBuildDate = datetime.datetime.utcnow(),
items = items
)
print rss_feed.to_xml()
software/backupserver/template-backup-script.sh.in
View file @ d007050d
......@@ -4,7 +4,13 @@
export HOME=$${directory:home}
set -e
# Clean status file (no history needed)
${coreutils-output:rm} -f $${:status_log}
# Inform about beginning of backup
${coreutils-output:echo} "`${coreutils-output:date} -u`, $${:hostname} backup running" >> $${:status_log}
# set -e
cd $${:datadirectory}
${rdiff-backup-output:rdiff-backup} \
$${:exclude_string} \
......@@ -13,5 +19,16 @@ ${rdiff-backup-output:rdiff-backup} \
--remote-schema '${dropbear-output:ssh} -T -y -i $${:sshkey} %s $${:remote_schema}' \
$${:connection}::/ ./
RESULT=$?
# Inform about backup status
${coreutils-output:rm} -f $${:status_log}
if [ $RESULT -eq 0 ]
then
${coreutils-output:echo} "`${coreutils-output:date} -u`, $${:hostname} backup success" >> $${:status_log}
else
${coreutils-output:echo} "`${coreutils-output:date} -u`, $${:hostname} backup failed" >> $${:status_log}
fi
# python scripts/verify_with_sudo.py ./ $${:connection}:/
software/backupserver/template-nginx-service.sh.in 0 → 100644
View file @ d007050d
#!${dash-output:dash}
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
if [ ! -e $${nginx-configuration:ssl_crt} ]
then
${openssl-output:openssl} genrsa -out $${nginx-configuration:ssl_key} 2048
${openssl-output:openssl} req -new \
-subj "/C=AA/ST=Denial/L=Nowhere/O=Dis/CN=$${nginx-configuration:ip}" \
-key $${nginx-configuration:ssl_key} -out $${nginx-configuration:ssl_csr}
${openssl-output:openssl} x509 -req -days 365 \
-in $${nginx-configuration:ssl_csr} \
-signkey $${nginx-configuration:ssl_key} \
-out $${nginx-configuration:ssl_crt}
fi
exec ${nginx-output:nginx} \
-c $${nginx-configuration:output}
software/backupserver/template-nginx.cfg.in 0 → 100644
View file @ d007050d
daemon off; # run in the foreground so supervisord can look after it
worker_processes 4;
pid $${directory:run}/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
error_log $${nginx-configuration:error_log};
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
default_type application/octet-stream;
include ${nginx-output:mime};
##
# Logging Settings
##
access_log $${nginx-configuration:access_log};
error_log $${nginx-configuration:error_log};
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
server {
listen [$${nginx-configuration:ip}]:$${nginx-configuration:port};
ssl on;
ssl_certificate $${nginx-configuration:ssl_crt};
ssl_certificate_key $${nginx-configuration:ssl_key};
fastcgi_temp_path $${directory:varnginx} 1 2;
uwsgi_temp_path $${directory:varnginx} 1 2;
scgi_temp_path $${directory:varnginx} 1 2;
client_body_temp_path $${directory:varnginx} 1 2;
proxy_temp_path $${directory:varnginx} 1 2;
## Only allow GET and HEAD request methods
if ($request_method !~ ^(GET|HEAD)$ ) {
return 444;
}
## Serve an error 204 (No Content) for favicon.ico
location = /favicon.ico {
return 204;
}
location /
{
root $${directory:www};
# index index.html;
}
}
}
software/backupserver/template-update-rss.sh.in 0 → 100644
View file @ d007050d
#!${dash-output:dash}
STATUS_DIR=$${directory:status}
RSS_DIR=$${directory:www}
CAT=${coreutils-output:cat}
PYTHON=${buildout:directory}/bin/${extra-eggs:interpreter}
STATUS2RSS=${status2rss:output}
BASENAME=${coreutils-output:basename}
for status in $STATUS_DIR/*
do
NAME=`$BASENAME $status`
$CAT $status | $PYTHON $STATUS2RSS "Backup status $NAME" "https://[$${nginx-configuration:ip}]:$${nginx-configuration:port}/$NAME.rss" > $RSS_DIR/$NAME.rss
done
$CAT $STATUS_DIR/* | $PYTHON $STATUS2RSS "Full backup status $${:global_rss}" "https://[$${nginx-configuration:ip}]:$${nginx-configuration:port}/$${:global_rss}" > $RSS_DIR/$${:global_rss}
software/erp5-util-testing/instance.cfg
View file @ d007050d
......@@ -39,7 +39,7 @@ environment = environment
[environment]
CPPFLAGS = -I${python2.7:location}/include/python2.7 -I${libxml2:location}/include -I${libxslt:location}/include
LDFLAGS = -L${python2.7:location}/lib -L${libxml2:location}/lib -L${libxslt:location}/lib -L${zlib:location}/lib
PYTHONPATH = ${python-setuptools:location}
PYTHONPATH = ${buildout:eggs-directory}/setuptools-${versions:setuptools}-py2.7.egg
LD_LIBRARY_PATH = ${libxslt:location}/lib:${libxml2:location}/lib:${zlib:location}/lib
[sh-environment]
......
software/erp5-util-testing/software.cfg
View file @ d007050d
......@@ -45,7 +45,7 @@ branch = master
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = 6b919dd280d6972afe0f50d389ba5fe6
md5sum = e8b87313667c944567029c61c9c0bd2e
output = ${buildout:directory}/template.cfg
mode = 640
......@@ -94,7 +94,7 @@ netifaces = 0.8
# slapos.core==0.28.5
# zc.buildout==1.6.0-dev-SlapOS-006
# zc.recipe.egg==1.3.2
setuptools = 0.6c12dev-r88846
setuptools = 0.9.8
# Required by:
# slapos.cookbook==0.65
......
software/erp5testnode/instance-default.cfg
View file @ d007050d
......@@ -7,7 +7,6 @@ offline = true
parts =
connection-dict
testnode
pwgen
shell
shellinabox
certificate-authority
......@@ -16,12 +15,11 @@ parts =
[connection-dict]
recipe = slapos.cookbook:publish
url = http://[$${shellinabox:ipv6}]:$${shellinabox:port}/
password = $${pwgen:password}
password = $${pwgen:passwd}
[pwgen]
recipe = slapos.cookbook:pwgen
file = $${buildout:directory}/.password
pwgen-binary = ${pwgen:location}/bin/pwgen
recipe = slapos.cookbook:generate.password
storage-path = $${buildout:directory}/.password
[testnode]
recipe = slapos.cookbook:erp5testnode
......@@ -82,7 +80,7 @@ port = 8080
shell = $${shell:wrapper}
wrapper = $${rootdirectory:bin}/shellinaboxd
shellinabox-binary = ${shellinabox:location}/bin/shellinaboxd
password = $${pwgen:password}
password = $${pwgen:passwd}
directory = $${buildout:directory}/
login-shell = $${rootdirectory:bin}/login
certificate-directory = $${directory:shellinabox}
......
software/erp5testnode/software.cfg
View file @ d007050d
......@@ -20,7 +20,6 @@ extends =
../../component/zip/buildout.cfg
../../component/busybox/buildout.cfg
../../component/shellinabox/buildout.cfg
../../component/pwgen/buildout.cfg
# Local development
develop =
......
software/gitrepo/instance-gitrepo.cfg
View file @ d007050d
......@@ -13,14 +13,13 @@ parts =
gitdaemon
git-http-backend-cgi
htpasswd
pwgen
git-repos
[publish]
recipe = slapos.cookbook:publish
url = http://[$${slap-network-information:global-ipv6}]:$${httpd-conf:port}/
user = $${pwgen:user}
password = $${pwgen:password}
password = $${pwgen:passwd}
[httpd]
recipe = slapos.cookbook:wrapper
......@@ -79,14 +78,12 @@ output = $${basedirectory:services}/git-daemon
recipe = collective.recipe.cmd
output = $${rootdirectory:etc}/httpd.htpasswd
on_install = true
on_udptae = true
on_update = true
cmds =
${apache:location}/bin/htpasswd -cb $${:output} $${pwgen:user} $${pwgen:password}
${apache:location}/bin/htpasswd -cb $${:output} $${pwgen:user} $${pwgen:passwd}
[pwgen]
recipe = slapos.cookbook:pwgen
file = $${buildout:directory}/.password
pwgen-binary = ${pwgen:location}/bin/pwgen
recipe = slapos.cookbook:generate.password
user = slapos
[rootdirectory]
......
software/gitrepo/software.cfg
View file @ d007050d
......@@ -4,7 +4,6 @@ extends =
../../component/apache/buildout.cfg
../../component/perl/buildout.cfg
../../component/git/buildout.cfg
../../component/pwgen/buildout.cfg
../../stack/slapos.cfg
parts =
......
software/helloworld/instance.cfg.in
View file @ d007050d
......@@ -45,7 +45,9 @@ configuration.name = John Doe
# Create all needed directories, depending on your needs
[directory]
recipe = slapos.cookbook:mkdirectory
etc = $${buildout:directory}/etc
home = $${buildout:directory}
etc = $${:home}/etc
var = $${:home}/var
# Executables put here will be started but not monitored (for startup scripts)
script = $${:etc}/run/
# Executables put here will be started and monitored (for daemons)
......@@ -53,7 +55,8 @@ service = $${:etc}/service
# Executables put here will be launched after buildout has completed to see
# if instance is running
promise = $${:etc}/promise/
# Path of the log directory used by our service (see [hello-world])
log = $${:var}/log
# Create a simple shell script that will only output your name if you
# specified it as instance parameter.
......@@ -62,9 +65,11 @@ promise = $${:etc}/promise/
# This recipe will try to "exec" the command-line after separating parameters.
recipe = slapos.cookbook:wrapper
# Notice that there is only one $ at ${dash:location}, it is because it comes from the Software Release buildout profile.
command-line = ${dash:location}/bin/dash -c 'echo "Hello $${instance-parameter:configuration.name}!"; sleep 100000;'
command-line = ${dash:location}/bin/dash -c 'echo "Hello $${instance-parameter:configuration.name}, it is $(date)." > $${directory:log}/log.log; sleep 1000000;'
# Put this shell script in the "etc/service" directory. Every executable of this
# repository will be started and monitored by supervisord
# repository will be started and monitored by supervisord. If one service
# exits/crashes, it will trigger a "bang" and cause run of slapgrid for the
# instance.
wrapper-path = $${directory:service}/hello-world
......
software/helloworld/software.cfg
View file @ d007050d
......@@ -16,15 +16,15 @@ parts =
slapos-cookbook
# Call creation of instance.cfg file that will be called for deployment of
# instance
template
instance-profile
# Download instance.cfg.in (buildout profile used to deployment of instance),
# replace all ${foo:bar} parameters by real values, and change $${foo:bar} to
# ${foo:bar}
[template]
[instance-profile]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg.in
output = ${buildout:directory}/instance.cfg
# MD5 checksum can be skipped for development (easier to develop), but must be filled for production
md5sum = 1fc461c00e86485bee77a942f39e3c43
md5sum = ed94ac99ae1e596c0da5350da6ab6f52
mode = 0644
software/kvm/README.txt
View file @ d007050d
......@@ -16,7 +16,7 @@ The following examples list how to request different possible instances of KVM
Software Release from slap console or command line.
KVM instance (1GB of RAM, 10GB of SSD, one core)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note that the KVM instance will try to request a frontend slave instance in order
to be accessible from IPv4.
......@@ -32,22 +32,51 @@ to be accessible from IPv4.
See the instance-kvm-input-schema.json file for more instance parameters (cpu-count, ram-size, disk-size, etc).
NBD instance
KVM instance (1GB of RAM, 10GB of SSD, one core)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This type of instance will allow to host a disk image that will be used by
any KVM instance.
Note that the KVM instance will try to request a frontend slave instance in order
to be accessible from IPv4.
::
mynbd = request(
myawesomekvm = request(
software_release=kvm,
partition_reference="mynbd",
software_type="nbd",
partition_reference="My awesome KVM",
partition_parameter_kw={
"nbd-host":"ubuntu-1204.nbd.vifib.net",
}
)
See the instance-kvm-input-schema.json file for more instance parameters (cpu-count, ram-size, disk-size, etc).
Resilient KVM instance
~~~~~~~~~~~~~~~~~~~~~
Like KVM instance, but backed-up (with history) in two places.
::
kvm = 'http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/tags/slapos-0.188:/software/kvm/software.cfg'
myresilientkvm = request(
software_release=kvm,
partition_reference="My resilient KVM",
software_type="kvm-resilient",
partition_parameter_kw={
"-sla-0-computer_guid": "COMP-1000", # Location of the main instance (KVM)
"-sla-1-computer_guid": "COMP-1001", # Location of the first clone
"-sla-2-computer_guid": "COMP-1002", # Location of the second clone
}
)
See the instance-kvm-input-schema.json AND instance-kvm-resilient-input-schema.json AND /stack/resilient/README.txt
files for more instance parameters (cpu-count, ram-size, disk-size, specific location of clones, etc).
Then, if you want one of the two clones to takeover, you need to login into
the hosting machine, go to the partition of the clone, and invoke bin/takeover.
KVM Frontend Master Instance (will host all frontend Slave Instances)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This type of instance will allow to host any frontend slave instance requested
by KVM instances. Slave instances (and thus KVM instance) will be accessible
......
software/kvm/common.cfg
View file @ d007050d
[buildout]
extends =
../../component/6tunnel/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/dcron/buildout.cfg
......@@ -9,13 +10,19 @@ extends =
../../component/logrotate/buildout.cfg
../../component/noVNC/buildout.cfg
../../component/openssl/buildout.cfg
../../component/dcron/buildout.cfg
../../stack/nodejs.cfg
../../stack/resilient/buildout.cfg
../../stack/slapos.cfg
parts =
template
eggs
# XXX: we have to manually add this for resilience
rdiff-backup
collective.recipe.template-egg
#XXX-Cedric : Currently, one can only access to KVM using noVNC.
# Ideally one should be able to access KVM by using either NoVNC or VNC.
# Problem is : no native crypto support in web browsers. So we have to disable ssl
......@@ -36,6 +43,7 @@ eggs =
websockify
slapos.cookbook
slapos.toolbox
erp5.util
[http-proxy]
# https://github.com/nodejitsu/node-http-proxy
......@@ -67,13 +75,70 @@ command =
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install pkginfo@0.2.3
# Create all templates that will be used to deploy instances
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg.in
md5sum = bc5a986c7208d02d3284a897ea90b39d
output = ${buildout:directory}/template.cfg
mode = 0644
[template-kvm]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/instance-kvm.cfg.jinja2
mode = 644
md5sum = e16c15f72fdeb92ce1854bc25daf5ad7
download-only = true
on-update = true
[template-kvm-resilient]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/instance-kvm-resilient.cfg.jinja2
mode = 644
md5sum = a07c96b53fe9145278cd64a3b27a459a
download-only = true
on-update = true
[template-kvm-resilient-test]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/instance-kvm-resilient-test.cfg.jinja2
md5sum = b4894680283d3912df4e9740f3e7848b
mode = 0644
download-only = true
on-update = true
[template-kvm-import]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-kvm.cfg.in
md5sum = 87197471aa93863c310204e8865b5ac1
output = ${buildout:directory}/template-kvm.cfg
url = ${:_profile_base_location_}/instance-kvm-import.cfg.in
md5sum = 7b36d6c61154b7ec3113a1bfaa25a904
output = ${buildout:directory}/template-kvm-import.cfg
mode = 0644
[template-kvm-import-script]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/template/kvm-import.sh.in
filename = kvm-import.sh.in
md5sum = e03ed049cddd8d157228b09e1ebc071a
download-only = true
mode = 0755
[template-kvm-export]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/instance-kvm-export.cfg.jinja2
mode = 644
md5sum = 900f416956903fa4858e67e93b5169a1
download-only = true
on-update = true
[template-kvm-export-script]
recipe = hexagonit.recipe.download
url = ${:_profile_base_location_}/template/kvm-export.sh.in
filename = kvm-export.sh.in
md5sum = 95fde96f35cbf90d677c44d18b60fafb
download-only = true
mode = 0755
[template-nbd]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-nbd.cfg.in
......@@ -87,10 +152,3 @@ url = ${:_profile_base_location_}/instance-frontend.cfg.in
md5sum = cdb690495e9eb007d2b7d2f8e12f5c59
output = ${buildout:directory}/template-frontend.cfg
mode = 0644
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg.in
md5sum = 0a98e34aaec7097a84066c0665e3a49a
output = ${buildout:directory}/template.cfg
mode = 0644
software/kvm/development.cfg
View file @ d007050d
......@@ -6,11 +6,13 @@ extends =
parts +=
slapos.cookbook-repository
slapos.toolbox-repository
erp5.util-repository
check-recipe
develop =
${:parts-directory}/slapos.cookbook-repository
${:parts-directory}/slapos.toolbox-repository
${:parts-directory}/erp5.util-repository
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
......@@ -21,7 +23,13 @@ git-executable = ${git:location}/bin/git
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.toolbox.git
branch = master
branch = kvmresiliency
git-executable = ${git:location}/bin/git
[erp5.util-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/erp5.git
branch = scalability-master2
git-executable = ${git:location}/bin/git
[check-recipe]
......@@ -30,4 +38,6 @@ stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link &&
grep parts ${buildout:develop-eggs-directory}/erp5.util.egg-link
software/kvm/instance-kvm-export.cfg.jinja2 0 → 100644
View file @ d007050d
[buildout]
extends =
{{ kvm_template }}
{{ pbsready_export_template }}
parts +=
cron-entry-backup
certificate-authority
publish-connection-information
kvm-vnc-promise
kvm-disk-image-corruption-promise
websockify-sighandler
novnc-promise
cron
frontend-promise
# Create the exporter executable, which is a simple shell script
[exporter]
recipe = slapos.recipe.template
url = {{ template_kvm_export }}
output = ${directory:bin}/${slap-parameter:namebase}-exporter
mode = 0755
backup-disk-path = ${directory:backup}/virtual.qcow2
# Resilient stack wants a "wrapper" parameter
wrapper = ${:output}
# Extends publish section with resilient parameters
[publish-connection-information]
<= resilient-publish-connection-parameter
software/kvm/instance-kvm-import.cfg.in 0 → 100644
View file @ d007050d
[buildout]
# Here, we don't need KVM to run to import data, so we don't
# even extend the kvm instance profile.
extends = ${pbsready-import:output}
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[directory]
recipe = slapos.cookbook:mkdirectory
etc = $${buildout:directory}/etc
bin = $${buildout:directory}/bin
srv = $${buildout:directory}/srv
var = $${buildout:directory}/var
log = $${:var}/log
scripts = $${:etc}/run
services = $${:etc}/service
promises = $${:etc}/promise
novnc-conf = $${:etc}/novnc
run = $${:var}/run
ca-dir = $${:srv}/ssl
cron-entries = $${:etc}/cron.d
crontabs = $${:etc}/crontabs
cronstamps = $${:etc}/cronstamps
[importer]
recipe = slapos.recipe.template
url = ${template-kvm-import-script:location}/${template-kvm-import-script:filename}
output = $${directory:bin}/$${slap-parameter:namebase}-importer
mode = 0755
backup-disk-path = $${directory:backup}/virtual.qcow2
disk-path = $${directory:srv}/virtual.qcow2
# Resilient stack wants a "wrapper" parameter
wrapper = $${:output}
backup-disk-path = $${directory:backup}/virtual.qcow2
software/kvm/instance-kvm-input-schema.json
View file @ d007050d
{
"name": "Input Parameters",
"type": "object",
"$schema": "http://json-schema.org/draft-04/schema",
"title": "Input Parameters",
"properties": {
"ram-size": {
"title": "RAM size",
......@@ -7,7 +10,7 @@
"type": "integer",
"default": 1024,
"minimum": 128,
"divisibleBy": 128,
"multipleOf": 128,
"maximum": 16384
},
"disk-size": {
......@@ -34,7 +37,6 @@
"maximum": 8
},
"nbd-host": {
"title": "NBD hostname",
"description": "hostname (or IP) of the NBD server containing the boot image.",
......@@ -65,6 +67,30 @@
"maximum": 65535
},
"virtual-hard-drive-url": {
"title": "Existing disk image URL",
"description": "If specified, will download an existing disk image (qcow2, raw, ...), and will use it as main virtual hard drive. Can be used to download and use an already installed and customized virtual hard drive.",
"format": "uri",
"type": "string",
},
"virtual-hard-drive-md5sum": {
"title": "Checksum of virtual hard drive",
"description": "MD5 checksum of virtual hard drive, used if virtual-hard-drive-url is specified.",
"type": "string",
},
"use-tap": {
"title": "Use QEMU TAP network interface",
"description": "Use QEMU TAP network interface, requires a bridge on SlapOS Node. If false, use user-mode network stack (NAT).",
"type": "boolean",
"default": false
},
"nat-rules": {
"title": "List of rules for NAT of QEMU user mode network stack.",
"description": "List of rules for NAT of QEMU user mode network stack, as comma-separated list of ports. For each port specified, it will redirect port x of the VM (example: 80) to the port x + 10000 of the public IPv6 (example: 10080). Defaults to \"22 80 443\". Ignored if \"use-tap\" parameter is enabled.",
"type": "string",
},
"frontend-instance-guid": {
"title": "Frontend Instance ID",
......
software/kvm/instance-kvm-output-schema.json
View file @ d007050d
......@@ -13,13 +13,6 @@
"description": "URL used to connect to the service.",
"type": "uri",
"required": false
},
"password": {
"title": "Password",
"description": "Password used to authenticate in the service webpage.",
"type": "uri",
"required": true
}
}
}
software/kvm/instance-kvm-resilient-input-schema.json 0 → 100644
View file @ d007050d
{
"type": "object",
"$schema": "http://json-schema.org/draft-04/schema",
"items": {
"allOf": [
{
"$ref": "instance-kvm-input-schema.json"
}
],
"title": "Input Parameters",
"properties": {
"-sla-0-computer_guid": {
"title": "Target computer for main instance",
"description": "Target computer GUID for main instance.",
"type": "string"
},
"-sla-1-computer_guid": {
"title": "Target computer for first clone",
"description": "Target computer for first clone and PBS.",
"type": "string"
},
"-sla-2-computer_guid": {
"title": "Target computer for second clone",
"description": "Target computer for second clone and PBS.",
"type": "string"
},
"resiliency-backup-periodicity": {
"title": "Periodicity of backup",
"description": "Periodicity of backup, in cron format.",
"type": "string"
},
"remove-backup-older-than": {
"title": "Remove backups older than...",
"description": "Remove all the backups in PBS that are older than specified value. It should be rdiff-backup-compatible.",
"type": "string",
"default": "3B"
}
}
}
software/kvm/instance-kvm-resilient-test.cfg.jinja2 0 → 100644
View file @ d007050d
[buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
parts =
deploy-resiliency-test
request-resilient-kvm
deploy-standalone-resiliency-test
[directory]
recipe = slapos.cookbook:mkdirectory
home = ${buildout:directory}
etc = ${:home}/etc/
var = ${:home}/var/
srv = ${:home}/srv/
bin = ${:home}/bin/
tmp = ${:home}/tmp/
log = ${:var}/log/
services = ${:etc}/service/
scripts = ${:etc}/run/
[deploy-resiliency-test]
recipe = slapos.cookbook:wrapper
wrapper-path = ${directory:scripts}/runKVMResiliencyTestSuite
testnode-parameters = --test-result-path={{ slapparameter_dict.get('test-result-path') }} --revision={{ slapparameter_dict.get('test-suite-revision') }} --node-title={{ slapparameter_dict.get('scalability-launcher-title') }} --test-suite={{ slapparameter_dict.get('test-suite') }} --test-suite-master-url={{ slapparameter_dict.get('test-suite-master-url') }} --log-path=${directory:log}
kvm-test-parameters = server_url=${slap-connection:server-url} key_file=${slap-connection:key-file} cert_file=${slap-connection:cert-file} computer_id=${slap-connection:computer-id} partition_id=${slap-connection:partition-id} software=${slap-connection:software-release-url} namebase=kvm kvm_rootinstance_name='${request-resilient-kvm:name}'
command-line = {{ bin_directory }}/runResiliencyTest ${:testnode-parameters} ${:kvm-test-parameters}
[deploy-standalone-resiliency-test]
# Used to manually run the KVM test if we don't have a running testnode.
recipe = slapos.cookbook:wrapper
wrapper-path = ${directory:bin}/runStandaloneResiliencyTestSuite
command-line = {{ bin_directory }}/runStandaloneResiliencyTest --test-suite-title=kvm ${deploy-resiliency-test:kvm-test-parameters}
[request-resilient-kvm]
<= slap-connection
recipe = slapos.cookbook:request
software-url = ${slap-connection:software-release-url}
software-type = kvm-resilient
name = Resilient KVM (Root Instance)
{% set cluster_parameter_dict = slapparameter_dict.get('cluster', {}) -%}
config = virtual-hard-drive-url virtual-hard-drive-md5sum resiliency-backup-periodicity {{ cluster_parameter_dict.keys() | join(' ') }}
{% for key, value in cluster_parameter_dict.items() -%}
config-{{ key }} = {{ dumps(value) }}
{% endfor -%}
config-virtual-hard-drive-url = ${slap-parameter:virtual-hard-drive-url}
config-virtual-hard-drive-md5sum = ${slap-parameter:virtual-hard-drive-md5sum}
config-resiliency-backup-periodicity = */5 * * * *
# XXX What to do?
sla = computer_guid
sla-computer_guid = ${slap-connection:computer-id}
[slap-parameter]
virtual-hard-drive-url = https://softinst43236.host.vifib.net/data/public/fbd4ad.php?dl=true
virtual-hard-drive-md5sum = 465e1024447997e7b86ee2e5151e031b
software/kvm/instance-kvm-resilient.cfg.jinja2 0 → 100644
View file @ d007050d
# vim: set ft=cfg:
{% import 'parts' as parts %}
{% import 'replicated' as replicated with context %}
[buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
# += because we need to take up parts (like instance-custom, slapmonitor etc) from the profile we extended
parts +=
{{ parts.replicate("kvm", "3") }}
publish-connection-informations
kvm-frontend-url-promise
kvm-backend-url-promise
{{ replicated.replicate("kvm", "3", "kvm-export", "kvm-import", slapparameter_dict=slapparameter_dict) }}
[directory]
recipe = slapos.cookbook:mkdirectory
etc = ${buildout:directory}/etc
promises = ${:etc}/promise
# Bubble down the parameters of the requested instance to the user
[request-kvm]
# Note: += doesn't work.
return =
# Resilient related parameters
url ssh-public-key ssh-url notification-id ip
# KVM related parameters
# XXX: return ALL parameters (like nat rules), through jinja
backend-url url
[publish-connection-informations]
recipe = slapos.cookbook:publish
backend-url = ${request-kvm:connection-backend-url}
url = ${request-kvm:connection-url}
[kvm-frontend-url-promise]
# Check that url parameter is complete
recipe = collective.recipe.template
input = inline:#!/bin/sh
URL="${request-kvm:connection-url}"
if [[ ! "$URL" == https://* ]]; then
exit 1
fi
output = ${resilient-directory:promise}/kvm-frontend-url
mode = 700
[kvm-backend-url-promise]
# Check that backend url is reachable
recipe = slapos.cookbook:check_url_available
path = ${directory:promises}/frontend_promise
url = ${publish-connection-informations:url}
dash_path = /bin/sh
curl_path = {{ curl_executable_location }}
software/kvm/instance-kvm.cfg.jinja2 0 → 100644
View file @ d007050d
#############################
#
# Instanciate kvm
#
#############################
[buildout]
parts =
certificate-authority
publish-connection-information
kvm-vnc-promise
kvm-disk-image-corruption-promise
websockify-sighandler
novnc-promise
# kvm-monitor
cron
# cron-entry-monitor
frontend-promise
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[directory]
recipe = slapos.cookbook:mkdirectory
etc = ${buildout:directory}/etc
bin = ${buildout:directory}/bin
srv = ${buildout:directory}/srv
var = ${buildout:directory}/var
log = ${:var}/log
scripts = ${:etc}/run
services = ${:etc}/service
promises = ${:etc}/promise
novnc-conf = ${:etc}/novnc
run = ${:var}/run
ca-dir = ${:srv}/ssl
cron-entries = ${:etc}/cron.d
crontabs = ${:etc}/crontabs
cronstamps = ${:etc}/cronstamps
[create-mac]
recipe = slapos.cookbook:generate.mac
storage-path = ${directory:srv}/mac
[gen-passwd]
recipe = slapos.cookbook:generate.password
storage-path = ${directory:srv}/passwd
bytes = 8
[kvm-instance]
# XXX-Cedric: change "KVM" recipe to simple "create wrappers". No need for this
# Specific code. It needs Jinja.
recipe = slapos.cookbook:kvm
vnc-passwd = ${gen-passwd:passwd}
ipv4 = ${slap-network-information:local-ipv4}
ipv6 = ${slap-network-information:global-ipv6}
vnc-ip = ${:ipv4}
vnc-port = 5901
# XXX-Cedric: should be named "default-cdrom-iso"
default-disk-image = {{ debian_amd64_netinst_location }}
nbd-host = ${slap-parameter:nbd-host}
nbd-port = ${slap-parameter:nbd-port}
nbd2-host = ${slap-parameter:nbd2-host}
nbd2-port = ${slap-parameter:nbd2-port}
tap-interface = ${slap-network-information:network-interface}
disk-path = ${directory:srv}/virtual.qcow2
disk-size = ${slap-parameter:disk-size}
disk-type = ${slap-parameter:disk-type}
socket-path = ${directory:var}/qmp_socket
pid-file-path = ${directory:run}/pid_file
smp-count = ${slap-parameter:cpu-count}
ram-size = ${slap-parameter:ram-size}
mac-address = ${create-mac:mac-address}
# XXX-Cedric: should be named runner-wrapper-path and controller-wrapper-path
runner-path = ${directory:services}/kvm
controller-path = ${directory:scripts}/kvm_controller
use-tap = ${slap-parameter:use-tap}
nat-rules = ${slap-parameter:nat-rules}
6tunnel-wrapper-path = ${directory:services}/6tunnel
virtual-hard-drive-url = ${slap-parameter:virtual-hard-drive-url}
virtual-hard-drive-md5sum = ${slap-parameter:virtual-hard-drive-md5sum}
shell-path = {{ dash_executable_location }}
qemu-path = {{ qemu_executable_location }}
qemu-img-path = {{ qemu_img_executable_location }}
6tunnel-path = {{ sixtunnel_executable_location }}
[kvm-vnc-promise]
recipe = slapos.cookbook:check_port_listening
path = ${directory:promises}/vnc_promise
hostname = ${kvm-instance:vnc-ip}
port = ${kvm-instance:vnc-port}
[kvm-disk-image-corruption-promise]
# Check that disk image is not corrupted
recipe = collective.recipe.template
input = inline:#!/bin/sh
${kvm-instance:qemu-img-path} check ${kvm-instance:disk-path}
output = ${directory:promises}/kvm-disk-image-corruption
mode = 700
[novnc-instance]
recipe = slapos.cookbook:novnc
path = ${ca-novnc:executable}
ip = ${slap-network-information:global-ipv6}
port = 6080
vnc-ip = ${kvm-instance:vnc-ip}
vnc-port = ${kvm-instance:vnc-port}
novnc-location = {{ novnc_location }}
websockify-path = {{ websockify_executable_location }}
ssl-key-path = ${ca-novnc:key-file}
ssl-cert-path = ${ca-novnc:cert-file}
[websockify-sighandler]
recipe = slapos.cookbook:signalwrapper
wrapper-path = ${directory:services}/websockify
wrapped-path = ${novnc-instance:path}
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = {{ openssl_executable_location }}
ca-dir = ${directory:ca-dir}
requests-directory = ${cadirectory:requests}
wrapper = ${directory:services}/certificate_authority
ca-private = ${cadirectory:private}
ca-certs = ${cadirectory:certs}
ca-newcerts = ${cadirectory:newcerts}
ca-crl = ${cadirectory:crl}
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = ${directory:ca-dir}/requests/
private = ${directory:ca-dir}/private/
certs = ${directory:ca-dir}/certs/
newcerts = ${directory:ca-dir}/newcerts/
crl = ${directory:ca-dir}/crl/
[ca-novnc]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = ${directory:novnc-conf}/novnc.key
cert-file = ${directory:novnc-conf}/novnc.crt
executable = ${directory:bin}/novnc
wrapper = ${directory:bin}/websockify
[novnc-promise]
recipe = slapos.cookbook:check_port_listening
path = ${directory:promises}/novnc_promise
hostname = ${novnc-instance:ip}
port = ${novnc-instance:port}
#----------------
#--
#-- Deploy cron.
[cron]
recipe = slapos.cookbook:cron
dcrond-binary = {{ dcron_executable_location }}
cron-entries = ${directory:cron-entries}
crontabs = ${directory:crontabs}
cronstamps = ${directory:cronstamps}
catcher = ${cron-simplelogger:wrapper}
binary = ${directory:services}/crond
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
wrapper = ${directory:bin}/cron_simplelogger
log = ${directory:log}/crond.log
#----------------
#--
#-- Deploy frontend.
[request-slave-frontend]
recipe = slapos.cookbook:requestoptional
software-url = ${slap-parameter:frontend-software-url}
server-url = ${slap-connection:server-url}
key-file = ${slap-connection:key-file}
cert-file = ${slap-connection:cert-file}
computer-id = ${slap-connection:computer-id}
partition-id = ${slap-connection:partition-id}
name = VNC Frontend
software-type = ${slap-parameter:frontend-software-type}
slave = true
config = host port
config-host = ${novnc-instance:ip}
config-port = ${novnc-instance:port}
return = url resource port domainname
sla = instance_guid
sla-instance_guid = ${slap-parameter:frontend-instance-guid}
[frontend-promise]
recipe = slapos.cookbook:check_url_available
path = ${directory:promises}/frontend_promise
url = ${publish-connection-information:url}
dash_path = {{ dash_executable_location }}
curl_path = {{ curl_executable_location }}
[publish-connection-information]
recipe = slapos.cookbook:publish
backend-url = https://[${novnc-instance:ip}]:${novnc-instance:port}/vnc_auto.html?host=[${novnc-instance:ip}]&port=${novnc-instance:port}&encrypt=1&password=${kvm-instance:vnc-passwd}
url = ${request-slave-frontend:connection-url}/vnc_auto.html?host=${request-slave-frontend:connection-domainname}&port=${request-slave-frontend:connection-port}&encrypt=1&path=${request-slave-frontend:connection-resource}&password=${kvm-instance:vnc-passwd}
# Publish NAT port mapping status
# XXX: hardcoded value from [slap-parameter]
{% set nat_rule_list = slapparameter_dict.get('nat-rules', '22 80 443') %}
{% for port in nat_rule_list.split(' ') -%}
{% set external_port = 10000 + port|int() -%}
nat-rule-port-{{port}} = ${slap-network-information:global-ipv6} : {{external_port}}
{% endfor -%}
[slap-parameter]
# Default values if not specified
frontend-software-type = frontend
frontend-software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/tags/slapos-0.92:/software/kvm/software.cfg
frontend-instance-guid =
nbd-port = 1024
nbd-host =
nbd2-port = 1024
nbd2-host =
ram-size = 1024
disk-size = 10
disk-type = virtio
cpu-count = 1
nat-rules = 22 80 443
use-tap = False
virtual-hard-drive-url =
virtual-hard-drive-md5sum =
software/kvm/instance.cfg.in
View file @ d007050d
......@@ -4,20 +4,93 @@ parts =
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${template-kvm:output}
kvm = ${template-kvm:output}
default = $${:kvm}
kvm = $${dynamic-template-kvm:rendered}
nbd = ${template-nbd:output}
frontend = ${template-frontend:output}
[slap-connection]
# part to migrate to new - separated words
computer-id = $${slap_connection:computer_id}
partition-id = $${slap_connection:partition_id}
server-url = $${slap_connection:server_url}
software-release-url = $${slap_connection:software_release_url}
key-file = $${slap_connection:key_file}
cert-file = $${slap_connection:cert_file}
kvm-resilient = $${dynamic-template-kvm-resilient:rendered}
kvm-import = ${template-kvm-import:output}
kvm-export = $${dynamic-template-kvm-export:rendered}
# Used for the test of resiliency. The system wants a "test" software_type.
test = $${dynamic-template-kvm-resilient-test:rendered}
frozen = ${instance-frozen:output}
pull-backup = ${template-pull-backup:output}
[slap-configuration]
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
[dynamic-template-kvm]
recipe = slapos.recipe.template:jinja2
template = ${template-kvm:location}/instance-kvm.cfg.jinja2
rendered = $${buildout:directory}/template-kvm.cfg
extensions = jinja2.ext.do
context =
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
raw curl_executable_location ${curl:location}/bin/curl
raw dash_executable_location ${dash:location}/bin/dash
raw dcron_executable_location ${dcron:location}/sbin/crond
raw debian_amd64_netinst_location ${debian-amd64-netinst.iso:location}/${debian-amd64-netinst.iso:filename}
raw novnc_location ${noVNC:location}
raw openssl_executable_location ${openssl:location}/bin/openssl
raw qemu_executable_location ${kvm:location}/bin/qemu-system-x86_64
raw qemu_img_executable_location ${kvm:location}/bin/qemu-img
raw sixtunnel_executable_location ${6tunnel:location}/bin/6tunnel
raw websockify_executable_location ${buildout:directory}/bin/websockify
template-parts-destination = ${template-parts:destination}
template-replicated-destination = ${template-replicated:destination}
import-list = file parts :template-parts-destination
file replicated :template-replicated-destination
mode = 0644
[dynamic-template-kvm-resilient]
recipe = slapos.recipe.template:jinja2
template = ${template-kvm-resilient:location}/instance-kvm-resilient.cfg.jinja2
rendered = $${buildout:directory}/template-kvm-resilient.cfg
extensions = jinja2.ext.do
context =
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
raw curl_executable_location ${curl:location}/bin/curl
template-parts-destination = ${template-parts:destination}
template-replicated-destination = ${template-replicated:destination}
import-list = file parts :template-parts-destination
file replicated :template-replicated-destination
mode = 0644
[dynamic-template-kvm-export]
recipe = slapos.recipe.template:jinja2
template = ${template-kvm-export:location}/instance-kvm-export.cfg.jinja2
rendered = $${buildout:directory}/template-kvm-export.cfg
extensions = jinja2.ext.do
context =
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
raw kvm_template $${dynamic-template-kvm:rendered}
raw template_kvm_export ${template-kvm-export-script:location}/${template-kvm-export-script:filename}
raw pbsready_export_template ${pbsready-export:output}
mode = 0644
[dynamic-template-kvm-resilient-test]
recipe = slapos.recipe.template:jinja2
template = ${template-kvm-resilient-test:location}/instance-kvm-resilient-test.cfg.jinja2
rendered = $${buildout:directory}/template-kvm-resilient-test.cfg
context =
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
raw bin_directory ${buildout:bin-directory}
mode = 0644
software/kvm/software.cfg
View file @ d007050d
......@@ -4,8 +4,13 @@ extends = common.cfg
[networkcache]
# signature certificates of the following uploaders.
# Romain Courteaud
# Sebastien Robin
# Kazuhiko Shiozaki
# Cedric de Saint Martin
# Test Agent
# Yingjie Xu
# Gabriel Monnerat
# Test Agent (Automatic update from tests)
# Aurélien Calonne
signature-certificate-list =
-----BEGIN CERTIFICATE-----
MIIB4DCCAUkCADANBgkqhkiG9w0BAQsFADA5MQswCQYDVQQGEwJGUjEZMBcGA1UE
......@@ -21,6 +26,32 @@ signature-certificate-list =
QUUGLQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB8jCCAVugAwIBAgIJAPu2zchZ2BxoMA0GCSqGSIb3DQEBBQUAMBIxEDAOBgNV
BAMMB3RzeGRldjMwHhcNMTExMDE0MTIxNjIzWhcNMTIxMDEzMTIxNjIzWjASMRAw
DgYDVQQDDAd0c3hkZXYzMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCrPbh+
YGmo6mWmhVb1vTqX0BbeU0jCTB8TK3i6ep3tzSw2rkUGSx3niXn9LNTFNcIn3MZN
XHqbb4AS2Zxyk/2tr3939qqOrS4YRCtXBwTCuFY6r+a7pZsjiTNddPsEhuj4lEnR
L8Ax5mmzoi9nE+hiPSwqjRwWRU1+182rzXmN4QIDAQABo1AwTjAdBgNVHQ4EFgQU
/4XXREzqBbBNJvX5gU8tLWxZaeQwHwYDVR0jBBgwFoAU/4XXREzqBbBNJvX5gU8t
LWxZaeQwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOBgQA07q/rKoE7fAda
FED57/SR00OvY9wLlFEF2QJ5OLu+O33YUXDDbGpfUSF9R8l0g9dix1JbWK9nQ6Yd
R/KCo6D0sw0ZgeQv1aUXbl/xJ9k4jlTxmWbPeiiPZEqU1W9wN5lkGuLxV4CEGTKU
hJA/yXa1wbwIPGvX3tVKdOEWPRXZLg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB7jCCAVegAwIBAgIJAJWA0jQ4o9DGMA0GCSqGSIb3DQEBBQUAMA8xDTALBgNV
BAMMBHg2MXMwIBcNMTExMTI0MTAyNDQzWhgPMjExMTEwMzExMDI0NDNaMA8xDTAL
BgNVBAMMBHg2MXMwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANdJNiFsRlkH
vq2kHP2zdxEyzPAWZH3CQ3Myb3F8hERXTIFSUqntPXDKXDb7Y/laqjMXdj+vptKk
3Q36J+8VnJbSwjGwmEG6tym9qMSGIPPNw1JXY1R29eF3o4aj21o7DHAkhuNc5Tso
67fUSKgvyVnyH4G6ShQUAtghPaAwS0KvAgMBAAGjUDBOMB0GA1UdDgQWBBSjxFUE
RfnTvABRLAa34Ytkhz5vPzAfBgNVHSMEGDAWgBSjxFUERfnTvABRLAa34Ytkhz5v
PzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4GBAFLDS7zNhlrQYSQO5KIj
z2RJe3fj4rLPklo3TmP5KLvendG+LErE2cbKPqnhQ2oVoj6u9tWVwo/g03PMrrnL
KrDm39slYD/1KoE5kB4l/p6KVOdeJ4I6xcgu9rnkqqHzDwI4v7e8/D3WZbpiFUsY
vaZhjNYKWQf79l6zXfOvphzJ
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAO4V/jiMoICoMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMjMyMCAXDTEyMDIxNjExMTAyM1oYDzIxMTIwMTIzMTExMDIzWjAT
MREwDwYDVQQDDAhDT01QLTIzMjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
......@@ -34,6 +65,32 @@ signature-certificate-list =
If1a2ZoqHRxoNo2yTmm7TSYRORWVS+vvfjY=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAIlBksrZVkK8MA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMzU3MCAXDTEyMDEyNjEwNTUyOFoYDzIxMTIwMTAyMTA1NTI4WjAT
MREwDwYDVQQDDAhDT01QLTM1NzCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
ts+iGUwi44vtIfwXR8DCnLtHV4ydl0YTK2joJflj0/Ws7mz5BYkxIU4fea/6+VF3
i11nwBgYgxQyjNztgc9u9O71k1W5tU95yO7U7bFdYd5uxYA9/22fjObaTQoC4Nc9
mTu6r/VHyJ1yRsunBZXvnk/XaKp7gGE9vNEyJvPn2bkCAwEAAaNQME4wHQYDVR0O
BBYEFKuGIYu8+6aEkTVg62BRYaD11PILMB8GA1UdIwQYMBaAFKuGIYu8+6aEkTVg
62BRYaD11PILMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAMoTRpBxK
YLEZJbofF7gSrRIcrlUJYXfTfw1QUBOKkGFFDsiJpEg4y5pUk1s5Jq9K3SDzNq/W
it1oYjOhuGg3al8OOeKFrU6nvNTF1BAvJCl0tr3POai5yXyN5jlK/zPfypmQYxE+
TaqQSGBJPVXYt6lrq/PRD9ciZgKLOwEqK8w=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAPHoWu90gbsgMA0GCSqGSIb3DQEBBQUAMBQxEjAQBgNV
BAMMCXZpZmlibm9kZTAeFw0xMjAzMTkyMzIwNTVaFw0xMzAzMTkyMzIwNTVaMBQx
EjAQBgNVBAMMCXZpZmlibm9kZTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
ozBijpO8PS5RTeKTzA90vi9ezvv4vVjNaguqT4UwP9+O1+i6yq1Y2W5zZxw/Klbn
oudyNzie3/wqs9VfPmcyU9ajFzBv/Tobm3obmOqBN0GSYs5fyGw+O9G3//6ZEhf0
NinwdKmrRX+d0P5bHewadZWIvlmOupcnVJmkks852BECAwEAAaNQME4wHQYDVR0O
BBYEFF9EtgfZZs8L2ZxBJxSiY6eTsTEwMB8GA1UdIwQYMBaAFF9EtgfZZs8L2ZxB
JxSiY6eTsTEwMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAc43YTfc6
baSemaMAc/jz8LNLhRE5dLfLOcRSoHda8y0lOrfe4lHT6yP5l8uyWAzLW+g6s3DA
Yme/bhX0g51BmI6gjKJo5DoPtiXk/Y9lxwD3p7PWi+RhN+AZQ5rpo8UfwnnN059n
yDuimQfvJjBFMVrdn9iP6SfMjxKaGk6gVmI=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAKRvzcy7OH0UMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtNzcyMCAXDTEyMDgxMDE1NDI1MVoYDzIxMTIwNzE3MTU0MjUxWjAT
MREwDwYDVQQDDAhDT01QLTc3MjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
......@@ -46,99 +103,112 @@ signature-certificate-list =
5pW18Ry5Ie7iFK4cQMerZwWPxBodEbAteYlRsI6kePV7Gf735Y1RpuN8qZ2sYL6e
x2IMeSwJ82BpdEI5niXxB+iT0HxhmR+XaMI=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB+DCCAWGgAwIBAgIJAKGd0vpks6T/MA0GCSqGSIb3DQEBBQUAMBQxEjAQBgNV
BAMMCUNPTVAtMTU4NDAgFw0xMzA2MjAxMjE5MjBaGA8yMTEzMDUyNzEyMTkyMFow
FDESMBAGA1UEAwwJQ09NUC0xNTg0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
gQDZTH9etPUC+wMZQ3UIiOwyyCfHsJ+7duCFYjuo1uZrhtDt/fp8qb8qK9ob+df3
EEYgA0IgI2j/9jNUEnKbc5+OrfKznzXjrlrH7zU8lKBVNCLzQuqBKRNajZ+UvO8R
nlqK2jZCXP/p3HXDYUTEwIR5W3tVCEn/Vda4upTLcPVE5wIDAQABo1AwTjAdBgNV
HQ4EFgQU7KXaNDheQWoy5uOU01tn1M5vNkEwHwYDVR0jBBgwFoAU7KXaNDheQWoy
5uOU01tn1M5vNkEwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOBgQASmqCU
Znbvu6izdicvjuE3aKnBa7G++Fdp2bdne5VCwVbVLYCQWatB+n4crKqGdnVply/u
+uZ16u1DbO9rYoKgWqjLk1GfiLw5v86pd5+wZd5I9QJ0/Sbz2vZk5S4ciMIGwArc
m711+GzlW5xe6GyH9SZaGOPAdUbI6JTDwLzEgA==
-----END CERTIFICATE-----
[versions]
numpy = 1.6.2
Jinja2 = 2.6
Werkzeug = 0.8.3
apache-libcloud = 0.12.1
Werkzeug = 0.9.4
apache-libcloud = 0.13.2
async = 0.6.1
buildout-versions = 1.7
collective.recipe.template = 1.10
erp5.util = 0.4.36
gitdb = 0.5.4
hexagonit.recipe.cmmi = 1.6
lxml = 3.1.0
itsdangerous = 0.23
lxml = 3.2.3
meld3 = 0.6.10
plone.recipe.command = 1.1
psutil = 1.1.1
pycrypto = 2.6
slapos.cookbook = 0.73.1
slapos.recipe.template = 2.4.2
slapos.toolbox = 0.33.1
rdiff-backup = 1.0.5
slapos.cookbook = 0.84.2
slapos.recipe.cmmi = 0.2
slapos.recipe.download = 1.0.dev-r4053
slapos.toolbox = 0.37.4
smmap = 0.8.2
websockify = 0.3.0
websockify = 0.5.1
z3c.recipe.scripts = 1.0.1
# Required by:
# slapos.core==0.35.1
# slapos.toolbox==0.33.1
Flask = 0.9
# slapos.toolbox==0.37.4
Flask = 0.10.1
# Required by:
# slapos.toolbox==0.33.1
# slapos.toolbox==0.37.4
GitPython = 0.3.2.RC1
# Required by:
# slapos.toolbox==0.33.1
# slapos.toolbox==0.37.4
atomize = 0.1.1
# Required by:
# slapos.toolbox==0.33.1
# paramiko==1.12.0
ecdsa = 0.9
# Required by:
# slapos.toolbox==0.37.4
feedparser = 5.1.3
# Required by:
# hexagonit.recipe.cmmi==1.6
hexagonit.recipe.download = 1.6nxd002
# slapos.cookbook==0.84.2
inotifyx = 0.2.0-1
# Required by:
# slapos.cookbook==0.73.1
inotifyx = 0.2.0
# slapos.cookbook==0.84.2
lock-file = 2.0
# Required by:
# slapos.cookbook==0.73.1
# slapos.cookbook==0.84.2
netaddr = 0.7.10
# Required by:
# slapos.core==0.35.1
netifaces = 0.8
netifaces = 0.8-1
# Required by:
# slapos.toolbox==0.33.1
paramiko = 1.10.0
# websockify==0.5.1
numpy = 1.7.1
# Required by:
# slapos.toolbox==0.33.1
psutil = 0.6.1
# slapos.toolbox==0.37.4
paramiko = 1.12.0
# Required by:
# slapos.core==0.35.1
pyflakes = 0.6.1
# Required by:
# slapos.cookbook==0.73.1
pytz = 2012j
pyflakes = 0.7.3
# Required by:
# slapos.cookbook==0.73.1
# slapos.core==0.35.1
# slapos.toolbox==0.33.1
setuptools = 0.6c12dev-r88846
# slapos.cookbook==0.84.2
pytz = 2013.7
# Required by:
# slapos.cookbook==0.73.1
# slapos.toolbox==0.33.1
# slapos.cookbook==0.84.2
# slapos.toolbox==0.37.4
slapos.core = 0.35.1
# Required by:
# slapos.core==0.35.1
supervisor = 3.0b1
supervisor = 3.0
# Required by:
# slapos.core==0.35.1
unittest2 = 0.5.1
# Required by:
# slapos.cookbook==0.73.1
# slapos.toolbox==0.33.1
# slapos.cookbook==0.84.2
# slapos.toolbox==0.37.4
xml-marshaller = 0.9.7
# Required by:
......
software/kvm/template/kvm-export.sh.in 0 → 100644
View file @ d007050d
#!/bin/bash
# Create a backup of the disk image of the virtual machine
BACKUP_PATH=${:backup-disk-path}
QMP_CLIENT=${buildout:directory}/software_release/bin/qemu-qmp-client
if [ ! -f $DISK_PATH ]; then
echo "Nothing to backup, disk image doesn't exist yet."
exit 0;
fi
$QMP_CLIENT --socket ${kvm-instance:socket-path} --drive-backup $BACKUP_PATH
software/kvm/template/kvm-import.sh.in 0 → 100644
View file @ d007050d
#!/bin/bash
DISK_PATH=${:disk-path}
BACKUP_PATH=${:backup-disk-path}
# TODO: Use rdiff
rm $DISK_PATH
cp $BACKUP_PATH $DISK_PATH
software/lxc/instance-lxc.cfg
View file @ d007050d
......@@ -68,9 +68,8 @@ bridge = !!BRIDGE_NAME!!
interface = lxc$${slap-network-information:network-interface}
[passwd]
recipe = slapos.cookbook:pwgen
file = $${buildout:directory}/.password
pwgen-binary = ${pwgen:location}/bin/pwgen
recipe = slapos.cookbook:generate.password
storage-path = $${buildout:directory}/.password
[shellinabox]
recipe = slapos.cookbook:shellinabox
......@@ -79,7 +78,7 @@ port = 8080
shell = ${lxc:location}/bin/lxc-console -n $${uuid:uuid}
wrapper = $${rootdirectory:bin}/shellinaboxd_raw
shellinabox-binary = ${shellinabox:location}/bin/shellinaboxd
password = $${passwd:password}
password = $${passwd:passwd}
directory = $${buildout:directory}/
login-shell = $${rootdirectory:bin}/login
certificate-directory = $${directory:shellinabox}
......
software/lxc/software.cfg
View file @ d007050d
......@@ -10,7 +10,6 @@ extends =
../../component/xz-utils/buildout.cfg
../../component/tar/buildout.cfg
../../component/shellinabox/buildout.cfg
../../component/pwgen/buildout.cfg
../../component/bash/buildout.cfg
../../component/coreutils/buildout.cfg
......@@ -23,7 +22,6 @@ parts =
slapos-toolbox
lxc
shellinabox
pwgen
[template]
recipe = slapos.recipe.template
......
software/kvm/instance-kvm.cfg.in → software/openstack/instance-compute.cfg
View file @ d007050d
......@@ -6,11 +6,24 @@
[buildout]
parts =
certificate-authority
publish-kvm-connection-information
kvm-instance
kvm-promise
tunnel-ipv6-keystone
tunnel-ipv6-keystone-admin
tunnel-ipv6-keystone-compute
tunnel-ipv6-novnc
tunnel-ipv6-glance
tunnel-ipv6-s3api
tunnel-ipv6-spice
tunnel-ipv6-novadb
tunnel-ipv6-rabbit
tunnel-ipv6-ec2
tunnel-ipv6-neutron
nova-configure
websockify-sighandler
novnc-promise
frontend-promise
publish-kvm-connection-information
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
......@@ -28,6 +41,8 @@ promises = $${:etc}/promise
novnc-conf = $${:etc}/novnc
run = $${:var}/run
ca-dir = $${:srv}/ssl
nova = $${:bin}/nova
log = $${:var}/log
[create-mac]
recipe = slapos.cookbook:generate.mac
......@@ -39,30 +54,51 @@ storage-path = $${directory:srv}/passwd
bytes = 8
[kvm-instance]
# XXX-Cedric: change "KVM" recipe to simple "create wrappers". No need for this
# Specific code
recipe = slapos.cookbook:kvm
vnc-ip = $${slap-network-information:local-ipv4}
vnc-passwd = $${gen-passwd:passwd}
ipv4 = $${slap-network-information:local-ipv4}
ipv6 = $${slap-network-information:global-ipv6}
vnc-ip = $${:ipv4}
vnc-port = 5901
# XXX-Cedric: should be named "default-cdrom-iso"
default-disk-image = ${debian-amd64-netinst.iso:location}/${debian-amd64-netinst.iso:filename}
nbd-host = $${slap-parameter:nbd-host}
nbd-port = $${slap-parameter:nbd-port}
nbd2-host = $${slap-parameter:nbd2-host}
nbd2-port = $${slap-parameter:nbd2-port}
tap = $${slap-network-information:network-interface}
tap-interface = $${slap-network-information:network-interface}
disk-path = $${directory:srv}/virtual.qcow2
disk-size = $${slap-parameter:disk-size}
disk-type = $${slap-parameter:disk-type}
socket-path = $${directory:var}/qmp_socket
pid-path = $${directory:run}/pid_file
pid-file-path = $${directory:run}/kvm_run.pid
smp-count = $${slap-parameter:cpu-count}
ram-size = $${slap-parameter:ram-size}
mac-address = $${create-mac:mac-address}
# XXX-Cedric: should be named runner-wrapper-path and controller-wrapper-path
runner-path = $${directory:services}/kvm
controller-path = $${directory:scripts}/kvm_controller
ssh-port = 22222
use-tap = False
nat-rules = 22:$${:ssh-port} 80:8080 443:4443
6tunnel-wrapper-path = $${directory:services}/6tunnel
virtual-hard-drive-url = $${slap-parameter:virtual-hard-drive-url}
virtual-hard-drive-md5sum = $${slap-parameter:virtual-hard-drive-md5sum}
shell-path = ${dash:location}/bin/dash
qemu-path = ${kvm:location}/bin/qemu-system-x86_64
qemu-img-path = ${kvm:location}/bin/qemu-img
passwd = $${gen-passwd:passwd}
6tunnel-path = ${6tunnel:location}/bin/6tunnel
[kvm-promise]
recipe = slapos.cookbook:check_port_listening
......@@ -70,12 +106,142 @@ path = $${directory:promises}/vnc_promise
hostname = $${kvm-instance:vnc-ip}
port = $${kvm-instance:vnc-port}
[ssh-port-check]
recipe = slapos.cookbook:check_port_listening
path = $${directory:nova}/ssh_port_check
hostname = $${tunnel-ipv6-ssh:ipv4}
port = $${tunnel-ipv6-ssh:ipv4-port}
[nova-conf]
recipe = slapos.recipe.template
url = ${compute-script:location}/${compute-script:filename}
output = $${directory:nova}/nova_configure.sh
mode = 0700
nova-url = https://github.com/openstack/nova.git
slap-ipv4 = $${slap-network-information:local-ipv4}
nova-passwd = $${slap-parameter:master-passwd}
vnc-url = $${request-openstack-vnc-frontend:connection-url}
nova-host = $${slap-parameter:host}
[nova-conf-run]
recipe = slapos.recipe.template
url = ${compute-script-run:location}/${compute-script-run:filename}
output = $${directory:nova}/nova_configure.py
mode = 0700
host-ip = $${tunnel-ipv6-ssh:ipv6}
ssh-port = $${tunnel-ipv6-ssh:ipv6-port}
system-user = $${slap-parameter:system-user}
system-passwd = $${slap-parameter:system-passwd}
nova-configure = $${nova-conf:output}
nova-result = $${directory:log}
ssh-check-script = $${ssh-port-check:path}
python_path = ${buildout:executable}
eggs-dir = ${buildout:eggs-directory}
instance = compute
[nova-configure]
recipe = slapos.cookbook:wrapper
command-line = $${nova-conf-run:output}
wrapper-path = $${directory:scripts}/nova-configure
##########CONFIGURE TUNNELS##########################
[tunnel-ipv6-base]
recipe = slapos.cookbook:ipv6toipv4
ipv6 = $${slap-network-information:global-ipv6}
ipv4 = $${slap-network-information:local-ipv4}
shell-path = ${dash:location}/bin/dash
6tunnel-path = ${6tunnel:location}/bin/6tunnel
[tunnel-ipv4-to6]
recipe = slapos.cookbook:ipv4toipv6
ipv6 = $${slap-parameter:master-address}
ipv4 = $${slap-network-information:local-ipv4}
shell-path = ${dash:location}/bin/dash
6tunnel-path = ${6tunnel:location}/bin/6tunnel
[tunnel-ipv6-ssh]
ipv6-port = $${kvm-instance:ssh-port}
ipv4-port = $${kvm-instance:ssh-port}
ipv6 = $${slap-network-information:global-ipv6}
ipv4 = $${slap-network-information:local-ipv4}
[tunnel-ipv6-keystone]
<= tunnel-ipv4-to6
ipv6-port = 5000
ipv4-port = 5000
runner-path = $${directory:services}/6tunnel-keystone
[tunnel-ipv6-keystone-admin]
<= tunnel-ipv4-to6
ipv6-port = 35357
ipv4-port = 35357
runner-path = $${directory:services}/6tunnel-keystone-admin
[tunnel-ipv6-keystone-compute]
<= tunnel-ipv4-to6
ipv6-port = 8774
ipv4-port = 8774
runner-path = $${directory:services}/6tunnel-keystone-compute
[tunnel-ipv6-neutron]
<= tunnel-ipv4-to6
ipv6-port = 9696
ipv4-port = 9696
runner-path = $${directory:services}/6tunnel-neutron
[tunnel-ipv6-novnc]
<= tunnel-ipv6-base
ipv6-port = 6080
ipv4-port = 6080
runner-path = $${directory:services}/6tunnel-novnc
[tunnel-ipv6-glance]
<= tunnel-ipv4-to6
ipv6-port = 9292
ipv4-port = 9292
runner-path = $${directory:services}/6tunnel-glance
[tunnel-ipv6-spice]
<= tunnel-ipv6-base
ipv6-port = 6082
ipv4-port = 6082
runner-path = $${directory:services}/6tunnel-spice
#I don't know if this is really usefull!!
[tunnel-ipv6-s3api]
<= tunnel-ipv4-to6
ipv6-port = 3333
ipv4-port = 3333
runner-path = $${directory:services}/6tunnel-s3api
[tunnel-ipv6-ec2]
<= tunnel-ipv4-to6
ipv6-port = 8773
ipv4-port = 8773
runner-path = $${directory:services}/6tunnel-ec2
[tunnel-ipv6-rabbit]
<= tunnel-ipv4-to6
ipv6-port = 5672
ipv4-port = 5672
runner-path = $${directory:services}/6tunnel-rabbit
[tunnel-ipv6-novadb]
<= tunnel-ipv4-to6
ipv6-port = 3306
ipv4-port = 3306
runner-path = $${directory:services}/6tunnel-novadb
#########INSTALL NOVNC FOR KVM IMAGE################
[novnc-instance]
recipe = slapos.cookbook:novnc
path = $${ca-novnc:executable}
ip = $${slap-network-information:global-ipv6}
port = 6080
port = 6090
vnc-ip = $${kvm-instance:vnc-ip}
vnc-port = $${kvm-instance:vnc-port}
novnc-location = ${noVNC:location}
......@@ -127,9 +293,10 @@ recipe = slapos.cookbook:generic.slapmonitor
db-path = $${directory:srv}/slapmonitor_database
##############REQUEST FRONTEND AND PUBLISH CONNEXION PARAMETERS############
[request-slave-frontend]
recipe = slapos.cookbook:requestoptional
software-url = $${slap-parameter:frontend-software-url}
software-url = $${slap-parameter:kvm-frontend-url}
server-url = $${slap-connection:server-url}
key-file = $${slap-connection:key-file}
cert-file = $${slap-connection:cert-file}
......@@ -142,36 +309,67 @@ config = host port
config-host = $${novnc-instance:ip}
config-port = $${novnc-instance:port}
return = url resource port domainname
#sla = instance_guid
#sla-instance_guid = $${slap-parameter:frontend-instance-guid}
[request-openstack-vnc-frontend]
recipe = slapos.cookbook:requestoptional
software-url = $${slap-parameter:kvm-frontend-url}
server-url = $${slap-connection:server-url}
key-file = $${slap-connection:key-file}
cert-file = $${slap-connection:cert-file}
computer-id = $${slap-connection:computer-id}
partition-id = $${slap-connection:partition-id}
name = OpenStack VNC Frontend
software-type = $${slap-parameter:frontend-software-type}
slave = true
config = host port
config-host = $${slap-network-information:global-ipv6}
config-port = $${tunnel-ipv6-novnc:ipv6-port}
return = url resource port domainname
sla = instance_guid
sla-instance_guid = $${slap-parameter:frontend-instance-guid}
[publish-kvm-connection-information]
recipe = slapos.cookbook:publish
backend-url = https://[$${novnc-instance:ip}]:$${novnc-instance:port}/vnc_auto.html?host=[$${novnc-instance:ip}]&port=$${novnc-instance:port}&encrypt=1
password = $${kvm-instance:passwd}
url = $${request-slave-frontend:connection-url}/vnc_auto.html?host=$${request-slave-frontend:connection-domainname}&port=$${request-slave-frontend:connection-port}&encrypt=1&path=$${request-slave-frontend:connection-resource}
vnc-backend-url = https://[$${novnc-instance:ip}]:$${novnc-instance:port}/vnc_auto.html?host=[$${novnc-instance:ip}]&port=$${novnc-instance:port}&encrypt=1&password=$${kvm-instance:vnc-passwd}
vnc-url = $${request-slave-frontend:connection-url}/vnc_auto.html?host=$${request-slave-frontend:connection-domainname}&port=$${request-slave-frontend:connection-port}&encrypt=1&path=$${request-slave-frontend:connection-resource}&password=$${kvm-instance:vnc-passwd}
ssh = ssh $${slap-parameter:system-user}@$${tunnel-ipv6-ssh:ipv6} -p $${tunnel-ipv6-ssh:ipv6-port}
ssh-defaul-passwd = $${slap-parameter:system-passwd}
local-ipv4 = $${slap-network-information:local-ipv4}
[frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/frontend_promise
url = $${publish-kvm-connection-information:url}
url = $${publish-kvm-connection-information:vnc-url}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
[slap-parameter]
# Default values if not specified
#frontend-instance-guid = SOFTINST-81
frontend-software-type = frontend
frontend-software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/tags/slapos-0.92:/software/kvm/software.cfg
kvm-frontend-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/tags/slapos-0.92:/software/kvm/software.cfg
frontend-instance-guid =
ram-size = 2048
disk-size = 20
disk-type = virtio
cpu-count = 4
nbd-port = 1024
nbd-host = debian.nbd.vifib.net
nbd-host = 2001:470:1f14:169:dd6b:3f84:9205:c750
nbd2-port = 1024
nbd2-host =
ram-size = 1024
disk-size = 10
disk-type = virtio
domain =
master-address = 2001:470:1f14:169:dd6b:3f84:9205:cd50
master-passwd = openstack
system-user = stack
system-passwd = openstack
#XXX- This host name should be changed as soon as possible, to avoid having two or more identical
#compute host name on the same openstack-Manager node.
host = nova.compute-$${slap-connection:computer-id}
cpu-count = 1
virtual-hard-drive-url =
virtual-hard-drive-md5sum =
\ No newline at end of file
software/openstack/instance-main.cfg 0 → 100644
View file @ d007050d
#############################
#
# Instanciate kvm
#
#############################
[buildout]
parts =
certificate-authority
kvm-instance
websockify-sighandler
kvm-promise
novnc-promise
request-openstack-frontend
nova-install
# openstack-frontend-promise
frontend-promise
publish-kvm-connection-information
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[directory]
recipe = slapos.cookbook:mkdirectory
etc = $${buildout:directory}/etc
bin = $${buildout:directory}/bin
srv = $${buildout:directory}/srv
var = $${buildout:directory}/var
scripts = $${:etc}/run
services = $${:etc}/service
promises = $${:etc}/promise
novnc-conf = $${:etc}/novnc
run = $${:var}/run
ca-dir = $${:srv}/ssl
nova = $${:bin}/nova
log = $${:var}/log
[create-mac]
recipe = slapos.cookbook:generate.mac
storage-path = $${directory:srv}/mac
[gen-passwd]
recipe = slapos.cookbook:generate.password
storage-path = $${directory:srv}/passwd
bytes = 8
[master-passwd]
recipe = slapos.cookbook:generate.password
storage-path = $${directory:srv}/openstack.passwd
bytes = 8
[kvm-instance]
recipe = slapos.cookbook:kvm
vnc-passwd = $${gen-passwd:passwd}
ipv4 = $${slap-network-information:local-ipv4}
ipv6 = $${slap-network-information:global-ipv6}
vnc-ip = $${:ipv4}
vnc-port = 5901
# XXX-Cedric: should be named "default-cdrom-iso"
default-disk-image = ${debian-amd64-netinst.iso:location}/${debian-amd64-netinst.iso:filename}
nbd-host = $${slap-parameter:nbd-host}
nbd-port = $${slap-parameter:nbd-port}
nbd2-host = $${slap-parameter:nbd2-host}
nbd2-port = $${slap-parameter:nbd2-port}
tap-interface = $${slap-network-information:network-interface}
disk-path = $${directory:srv}/virtual.qcow2
disk-size = $${slap-parameter:disk-size}
disk-type = $${slap-parameter:disk-type}
socket-path = $${directory:var}/qmp_socket
pid-file-path = $${directory:run}/kvm_run.pid
smp-count = $${slap-parameter:cpu-count}
ram-size = $${slap-parameter:ram-size}
mac-address = $${create-mac:mac-address}
# XXX-Cedric: should be named runner-wrapper-path and controller-wrapper-path
runner-path = $${directory:services}/kvm
controller-path = $${directory:scripts}/kvm_controller
ssh-port = 22222
use-tap = False
nat-rules = 22:$${:ssh-port} 80:$${tunnel-http:port} 443:$${tunnel-http:port-https} 5000:5000 6080:$${tunnel-http:port-vnc} 6082:6082 3333:3333 9292:9292 9696:9696 3306:3306 35357:35357
6tunnel-wrapper-path = $${directory:services}/6tunnel
virtual-hard-drive-url = $${slap-parameter:virtual-hard-drive-url}
virtual-hard-drive-md5sum = $${slap-parameter:virtual-hard-drive-md5sum}
shell-path = ${dash:location}/bin/dash
qemu-path = ${kvm:location}/bin/qemu-system-x86_64
qemu-img-path = ${kvm:location}/bin/qemu-img
6tunnel-path = ${6tunnel:location}/bin/6tunnel
[kvm-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/vnc_promise
hostname = $${kvm-instance:vnc-ip}
port = $${kvm-instance:vnc-port}
[tunnel-ipv6-base]
recipe = slapos.cookbook:ipv6toipv4
ipv6 = $${slap-network-information:global-ipv6}
ipv4 = $${slap-network-information:local-ipv4}
shell-path = ${dash:location}/bin/dash
6tunnel-path = ${6tunnel:location}/bin/6tunnel
[install-script]
recipe = slapos.recipe.template
url = ${openstack-grizzly:location}/${openstack-grizzly:filename}
output = $${directory:nova}/stack_controller_grizzly.sh
mode = 0700
#nova-url = https://github.com/openstack-dev/devstack.git
image-url = $${slap-parameter:glance-image-url}
image-name = $${slap-parameter:glance-image-name}
slap-ipv4 = $${slap-network-information:local-ipv4}
nova-passwd = $${master-passwd:passwd}
nova-user = $${slap-parameter:system-user}
project = $${slap-parameter:project}
user-name = $${slap-parameter:user-name}
vnc-frontend = $${request-openstack-vnc-frontend:connection-url}/vnc_auto.html?path=$${request-openstack-vnc-frontend:connection-resource}
[nova-conf-run]
recipe = slapos.recipe.template
url = ${compute-script-run:location}/${compute-script-run:filename}
output = $${directory:nova}/nova_configure.py
mode = 0700
host-ip = $${tunnel-ipv6-ssh:ipv6}
ssh-port = $${tunnel-ipv6-ssh:ipv6-port}
system-user = $${slap-parameter:system-user}
system-passwd = $${slap-parameter:system-passwd}
nova-configure = $${install-script:output}
keystone-endpoints-script = ${openstack-keystone-endpoints-script:location}/${openstack-keystone-endpoints-script:filename}
keystone-script = ${openstack-keystone-script:location}/${openstack-keystone-script:filename}
floating-ip-generate = ${network-floating-ip-generate:location}/${network-floating-ip-generate:filename}
signed-certificate-generate = ${auto-signed-certificate-generate:location}/${auto-signed-certificate-generate:filename}
nova-result = $${directory:log}
python_path = ${buildout:executable}
eggs-dir = ${buildout:eggs-directory}
instance = manage
[nova-install]
recipe = slapos.cookbook:wrapper
command-line = $${nova-conf-run:output}
wrapper-path = $${directory:scripts}/openstack-install
######################################################################
### PREDEFINED PORT AND HOST FOR SSH ACCESS TO OPENSTACK COMPUTER ####
[tunnel-ipv6-ssh]
ipv6-port = $${kvm-instance:ssh-port}
ipv4-port = $${kvm-instance:ssh-port}
ipv6 = $${slap-network-information:global-ipv6}
ipv4 = $${slap-network-information:local-ipv4}
##########################################################
### PREDEFINED PORT AND HOST FOR NOVA HORIZON SERVICE ####
[tunnel-http]
host = $${slap-network-information:global-ipv6}
port = 8080
port-https = 4443
port-vnc = 6080
##########################################################
[novnc-instance]
recipe = slapos.cookbook:novnc
path = $${ca-novnc:executable}
ip = $${slap-network-information:global-ipv6}
port = 6090
vnc-ip = $${kvm-instance:vnc-ip}
vnc-port = $${kvm-instance:vnc-port}
novnc-location = ${noVNC:location}
websockify-path = ${buildout:directory}/bin/websockify
ssl-key-path = $${ca-novnc:key-file}
ssl-cert-path = $${ca-novnc:cert-file}
[websockify-sighandler]
recipe = slapos.cookbook:signalwrapper
wrapper-path = $${directory:services}/websockify
wrapped-path = $${novnc-instance:path}
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = ${openssl:location}/bin/openssl
ca-dir = $${directory:ca-dir}
requests-directory = $${cadirectory:requests}
wrapper = $${directory:services}/certificate_authority
ca-private = $${cadirectory:private}
ca-certs = $${cadirectory:certs}
ca-newcerts = $${cadirectory:newcerts}
ca-crl = $${cadirectory:crl}
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:ca-dir}/requests/
private = $${directory:ca-dir}/private/
certs = $${directory:ca-dir}/certs/
newcerts = $${directory:ca-dir}/newcerts/
crl = $${directory:ca-dir}/crl/
[ca-novnc]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${directory:novnc-conf}/novnc.key
cert-file = $${directory:novnc-conf}/novnc.crt
executable = $${directory:bin}/novnc
wrapper = $${directory:bin}/websockify
[novnc-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/novnc_promise
hostname = $${novnc-instance:ip}
port = $${novnc-instance:port}
[kvm-monitor]
recipe = slapos.cookbook:generic.slapmonitor
db-path = $${directory:srv}/slapmonitor_database
[request-slave-frontend]
recipe = slapos.cookbook:requestoptional
software-url = $${slap-parameter:kvm-frontend-url}
server-url = $${slap-connection:server-url}
key-file = $${slap-connection:key-file}
cert-file = $${slap-connection:cert-file}
computer-id = $${slap-connection:computer-id}
partition-id = $${slap-connection:partition-id}
name = VNC Frontend
software-type = $${slap-parameter:frontend-software-type}
slave = true
config = host port
config-host = $${novnc-instance:ip}
config-port = $${novnc-instance:port}
return = url resource port domainname
sla = instance_guid
sla-instance_guid = $${slap-parameter:frontend-instance-guid}
[request-openstack-vnc-frontend]
recipe = slapos.cookbook:requestoptional
software-url = $${slap-parameter:kvm-frontend-url}
server-url = $${slap-connection:server-url}
key-file = $${slap-connection:key-file}
cert-file = $${slap-connection:cert-file}
computer-id = $${slap-connection:computer-id}
partition-id = $${slap-connection:partition-id}
name = OpenStack VNC Frontend
software-type = $${slap-parameter:frontend-software-type}
slave = true
config = host port
config-host = $${slap-network-information:global-ipv6}
config-port = $${tunnel-http:port-vnc}
return = url resource port domainname
sla = instance_guid
sla-instance_guid = $${slap-parameter:frontend-instance-guid}
[request-openstack-frontend]
<= slap-connection
#XXX - Don't use requestoptional because vnc frontend is a prerequisite to deploy openstack
recipe = slapos.cookbook:requestoptional
name = Frontend OpenStack
# XXX We have hardcoded SR URL here.
software-url = $${slap-parameter:frontend-software-url}
slave = true
config = url custom_domain
config-url = http://[$${tunnel-http:host}]:$${tunnel-http:port}/
return = site_url
config-custom_domain = $${slap-parameter:domain}
[publish-kvm-connection-information]
recipe = slapos.cookbook:publish
vnc-backend-url = https://[$${novnc-instance:ip}]:$${novnc-instance:port}/vnc_auto.html?host=[$${novnc-instance:ip}]&port=$${novnc-instance:port}&encrypt=1&password=$${kvm-instance:vnc-passwd}
vnc-url = $${request-slave-frontend:connection-url}/vnc_auto.html?host=$${request-slave-frontend:connection-domainname}&port=$${request-slave-frontend:connection-port}&encrypt=1&path=$${request-slave-frontend:connection-resource}&password=$${kvm-instance:vnc-passwd}
ssh = ssh $${slap-parameter:system-user}@$${tunnel-ipv6-ssh:ipv6} -p $${tunnel-ipv6-ssh:ipv6-port}
ssh-defaul-passwd = $${slap-parameter:system-passwd}
openstack-url = $${request-openstack-frontend:connection-site_url}/horizon
openstack-services-host = $${slap-network-information:global-ipv6}
openstack-services-passwd = $${master-passwd:passwd}
openstack-project = $${slap-parameter:project}
openstack-admin-user = admin
openstack-project-user = $${slap-parameter:user-name}
[frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/frontend_promise
url = $${publish-kvm-connection-information:vnc-url}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
[openstack-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/openstack_frontend
url = $${request-openstack-frontend:connection-site_url}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
match = OpenStack
[slap-parameter]
# Default values if not specified
frontend-software-type = frontend
kvm-frontend-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/tags/slapos-0.92:/software/kvm/software.cfg
frontend-software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
frontend-instance-guid =
ram-size = 2048
disk-size = 20
disk-type = virtio
virtual-hard-drive-url = https://[2001:470:1f14:169:e1f9:5f22:ff2b:6938]:8070/data/public/9052f7.php?dl=true
virtual-hard-drive-md5sum = 0e359c2aa6e9430326be15656eeb8a3c
cpu-count = 4
nbd-port = 1024
#Use Ubuntu 13.04 server
nbd-host = 2001:470:1f14:169:dd6b:3f84:9205:6218
nbd2-port = 1024
nbd2-host =
#Create this user and password on system
system-user = stack
system-passwd = openstack
#Project name and user-name for openstack. Password is generated by SlapOS
project = slapos
user-name = slapos
glance-image-url = http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img
glance-image-name = cirros-0.3.1-x86_64
domain =
software/openstack/instance.cfg 0 → 100644
View file @ d007050d
[buildout]
parts =
switch-softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${template-openstack-main:output}
compute = ${template-openstack-compute:output}
[slap-connection]
# part to migrate to new - separated words
computer-id = $${slap_connection:computer_id}
partition-id = $${slap_connection:partition_id}
server-url = $${slap_connection:server_url}
software-release-url = $${slap_connection:software_release_url}
key-file = $${slap_connection:key_file}
cert-file = $${slap_connection:cert_file}
\ No newline at end of file
software/openstack/software.cfg 0 → 100644
View file @ d007050d
[buildout]
extends =
../../component/6tunnel/buildout.cfg
../../component/curl/buildout.cfg
../../component/dash/buildout.cfg
../../component/dcron/buildout.cfg
../../component/gzip/buildout.cfg
../../component/qemu-kvm/buildout.cfg
../../component/logrotate/buildout.cfg
../../component/noVNC/buildout.cfg
../../component/openssl/buildout.cfg
../../stack/slapos.cfg
../../component/git/buildout.cfg
parts =
template
eggs
slapos.cookbook-repository
check-recipe
slapos-cookbook
develop =
${:parts-directory}/slapos.cookbook-repository
[eggs]
recipe = z3c.recipe.scripts
eggs =
${lxml-python:egg}
websockify
paramiko
slapos.cookbook
slapos.toolbox
[template-openstack-main]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-main.cfg
md5sum = 641a42ffa5dd4dee54b37356b12d3441
output = ${buildout:directory}/template-openstack-main.cfg
mode = 0644
[template-openstack-compute]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-compute.cfg
md5sum = 77c6188255f8f469f003f2db59d1a3b4
output = ${buildout:directory}/template-openstack-compute.cfg
mode = 0644
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
md5sum = 078532437caafea4d515fb27267de6ee
output = ${buildout:directory}/template.cfg
mode = 0644
[template-download]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/templates/${:filename}
mode = 0644
location = ${buildout:parts-directory}/${:_buildout_section_name_}
[compute-script]
<= template-download
filename = configure.sh.in
md5sum = 136339c0d704bb11af3839726e24c47f
[compute-script-run]
<= template-download
filename = ssh-configure.py.in
md5sum = 656db97dd54a7a0d96e7ce18f0be5c32
[network-floating-ip-generate]
<= template-download
filename = network.py
#md5sum =
[auto-signed-certificate-generate]
<= template-download
filename = generate_cert.py
#md5sum =
[openstack-install]
<= template-download
filename = openstack-install.sh.in
md5sum = 5cd174f8e94f5d975d7940ac65d8e9f4
[openstack-grizzly]
<= template-download
filename = stack_controller_grizzly.sh.in
#md5sum =
[openstack-keystone-script]
<= template-download
filename = keystone_basic.sh
#md5sum =
[openstack-keystone-endpoints-script]
<= template-download
filename = keystone_endpoints_basic.sh
#md5sum =
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.git
branch = openstack
git-executable = ${git:location}/bin/git
[check-recipe]
recipe = plone.recipe.command
stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link;
[networkcache]
# signature certificates of the following uploaders.
# Romain Courteaud
# Cedric de Saint Martin
# Test Agent
signature-certificate-list =
-----BEGIN CERTIFICATE-----
MIIB4DCCAUkCADANBgkqhkiG9w0BAQsFADA5MQswCQYDVQQGEwJGUjEZMBcGA1UE
CBMQRGVmYXVsdCBQcm92aW5jZTEPMA0GA1UEChMGTmV4ZWRpMB4XDTExMDkxNTA5
MDAwMloXDTEyMDkxNTA5MDAwMlowOTELMAkGA1UEBhMCRlIxGTAXBgNVBAgTEERl
ZmF1bHQgUHJvdmluY2UxDzANBgNVBAoTBk5leGVkaTCBnzANBgkqhkiG9w0BAQEF
AAOBjQAwgYkCgYEApYZv6OstoqNzxG1KI6iE5U4Ts2Xx9lgLeUGAMyfJLyMmRLhw
boKOyJ9Xke4dncoBAyNPokUR6iWOcnPHtMvNOsBFZ2f7VA28em3+E1JRYdeNUEtX
Z0s3HjcouaNAnPfjFTXHYj4um1wOw2cURSPuU5dpzKBbV+/QCb5DLheynisCAwEA
ATANBgkqhkiG9w0BAQsFAAOBgQBCZLbTVdrw3RZlVVMFezSHrhBYKAukTwZrNmJX
mHqi2tN8tNo6FX+wmxUUAf3e8R2Ymbdbn2bfbPpcKQ2fG7PuKGvhwMG3BlF9paEC
q7jdfWO18Zp/BG7tagz0jmmC4y/8akzHsVlruo2+2du2freE8dK746uoMlXlP93g
QUUGLQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAO4V/jiMoICoMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMjMyMCAXDTEyMDIxNjExMTAyM1oYDzIxMTIwMTIzMTExMDIzWjAT
MREwDwYDVQQDDAhDT01QLTIzMjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
wi/3Z8W9pUiegUXIk/AiFDQ0UJ4JFAwjqr+HSRUirlUsHHT+8DzH/hfcTDX1I5BB
D1ADk+ydXjMm3OZrQcXjn29OUfM5C+g+oqeMnYQImN0DDQIOcUyr7AJc4xhvuXQ1
P2pJ5NOd3tbd0kexETa1LVhR6EgBC25LyRBRae76qosCAwEAAaNQME4wHQYDVR0O
BBYEFMDmW9aFy1sKTfCpcRkYnP6zUd1cMB8GA1UdIwQYMBaAFMDmW9aFy1sKTfCp
cRkYnP6zUd1cMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAskbFizHr
b6d3iIyN+wffxz/V9epbKIZVEGJd/6LrTdLiUfJPec7FaxVCWNyKBlCpINBM7cEV
Gn9t8mdVQflNqOlAMkOlUv1ZugCt9rXYQOV7rrEYJBWirn43BOMn9Flp2nibblby
If1a2ZoqHRxoNo2yTmm7TSYRORWVS+vvfjY=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAKRvzcy7OH0UMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtNzcyMCAXDTEyMDgxMDE1NDI1MVoYDzIxMTIwNzE3MTU0MjUxWjAT
MREwDwYDVQQDDAhDT01QLTc3MjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
o7aipd6MbnuGDeR1UJUjuMLQUariAyQ2l2ZDS6TfOwjHiPw/mhzkielgk73kqN7A
sUREx41eTcYCXzTq3WP3xCLE4LxLg1eIhd4nwNHj8H18xR9aP0AGjo4UFl5BOMa1
mwoyBt3VtfGtUmb8whpeJgHhqrPPxLoON+i6fIbXDaUCAwEAAaNQME4wHQYDVR0O
BBYEFEfjy3OopT2lOksKmKBNHTJE2hFlMB8GA1UdIwQYMBaAFEfjy3OopT2lOksK
mKBNHTJE2hFlMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAaNRx6YN2
M/p3R8/xS6zvH1EqJ3FFD7XeAQ52WuQnKSREzuw0dsw12ClxjcHiQEFioyTiTtjs
5pW18Ry5Ie7iFK4cQMerZwWPxBodEbAteYlRsI6kePV7Gf735Y1RpuN8qZ2sYL6e
x2IMeSwJ82BpdEI5niXxB+iT0HxhmR+XaMI=
-----END CERTIFICATE-----
[versions]
Jinja2 = 2.7.1
MarkupSafe = 0.18
Werkzeug = 0.9.4
apache-libcloud = 0.13.2
async = 0.6.1
buildout-versions = 1.7
cliff = 1.4.5
cmd2 = 0.6.7
gitdb = 0.5.4
itsdangerous = 0.23
lxml = 3.2.4
meld3 = 0.6.10
paramiko = 1.12.0
plone.recipe.command = 1.1
pyOpenSSL = 0.13.1
pycrypto = 2.6.1
pyparsing = 2.0.1
requests = 2.0.1
slapos.recipe.build = 0.12
slapos.recipe.cmmi = 0.2
slapos.recipe.download = 1.0.dev-r4053
slapos.recipe.template = 2.5
slapos.toolbox = 0.37.4
smmap = 0.8.2
websockify = 0.5.1
z3c.recipe.scripts = 1.0.1
# Required by:
# slapos.core==0.35.1
# slapos.toolbox==0.37.4
Flask = 0.10.1
# Required by:
# slapos.toolbox==0.37.4
GitPython = 0.3.2.RC1
# Required by:
# slapos.toolbox==0.37.4
atomize = 0.1.1
# Required by:
# paramiko==1.12.0
ecdsa = 0.10
# Required by:
# slapos.toolbox==0.37.4
feedparser = 5.1.3
# Required by:
# slapos.cookbook==0.83.1
inotifyx = 0.2.0-1
# Required by:
# slapos.cookbook==0.83.1
lock-file = 2.0
# Required by:
# slapos.cookbook==0.83.1
netaddr = 0.7.10
# Required by:
# slapos.core==0.35.1
netifaces = 0.8-1
# Required by:
# websockify==0.5.1
numpy = 1.8.0
# Required by:
# slapos.toolbox==0.37.4
psutil = 1.1.3
# Required by:
# slapos.core==0.35.1
pyflakes = 0.7.3
# Required by:
# slapos.cookbook==0.83.1
pytz = 2013.8
# Required by:
# slapos.cookbook==0.83.1
# slapos.toolbox==0.37.4
slapos.core = 0.35.1
# Required by:
# slapos.core==0.35.1
supervisor = 3.0
# Required by:
# slapos.core==0.35.1
unittest2 = 0.5.1
# Required by:
# slapos.cookbook==0.83.1
# slapos.toolbox==0.37.4
xml-marshaller = 0.9.7
# Required by:
# slapos.core==0.35.1
zope.interface = 4.0.5
\ No newline at end of file
software/openstack/templates/configure.sh.in 0 → 100644
View file @ d007050d
#!/usr/bin/env bash
#configure.sh: Should install and configure all needed openstack packages.
NETWORK_CONFIG ="/etc/network/interfaces"
GCC_EXEC=`which gcc`
NOV_URL="${:nova-url}"
NOVA_CONFIG="/etc/nova/nova.conf"
BASE_DIR=$HOME
EZ_SETUP="http://peak.telecommunity.com/dist/ez_setup.py"
EASY_INSTALL=`which easy_install`
PYTHON=`which python`
NOVA_USER=$USER
IPv4=`ip addr show eth0 | grep 'inet ' | awk '{print $2}' | cut -d/ -f1`
if [ -z "$IPv4" ];
then IPv4="127.0.0.1"
fi
echo " "
echo " "
echo " "
echo "#########################################################################################"
echo "######### ###########"
echo "######### AUTO CONFIGURE AND INSTALL SCRIPT FOR OPENSTACK NOVA-COMPUTE NODE ###########"
echo "######### ###########"
echo "######### ******Network ip adress: $IPv4 ###########"
echo "######### ###########"
echo "#########################################################################################"
echo " "
echo " "
echo " "
ip link set eth0 promisc on
cat >$NETWORK_CONFIG <<EOF
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
# Bridge network interface for VM networks
auto br100
iface br100 inet static
address 192.168.100.1
netmask 255.255.255.0
bridge_stp off
bridge_fd 0
EOF
BRCTL_EXEC=`which brctl`
if [ -z "$BRCTL_EXEC" -o ! -x "$BRCTL_EXEC" ]; then
apt-get install -y bridge-utils ;
BRCTL_EXEC=`which brctl`
fi
$BRCTL_EXEC addbr br100
#Restart the network to take new network configuration.
/etc/init.d/networking restart
sudo apt-get update
sudo apt-get -y upgrade
if [ -z "$GCC_EXEC" -o ! -x "$GCC_EXEC" ]; then
apt-get install -y gcc g++ ;
fi
apt-get install -y python-dev
apt-get install -y gnutls-bin gnutls-dev
apt-get install -y libdevmapper-dev libdevmapper
apt-get install -y python-all-dev
apt-get install -y kvm libvirt-bin
apt-get install -y python-libvirt
apt-get install -y libxml2-dev libxslt-dev
apt-get install -y python-lxml
GIT_EXEC=`which git`
if [ -z "$GIT_EXEC" -o ! -x "$GIT_EXEC" ]; then
apt-get install -y git ;
GIT_EXEC=`which git`
fi
GIT_EXEC=`which git`
#Remove all pip tmp cache is to prevent installation failure
rm -rf /tmp/pip_build_root/*
#Install python packages*/
if [ -z "$EASY_INSTALL" -o ! -x "$EASY_INSTALL" ]; then
cd $BASE_DIR; wget $EZ_SETUP -O ez_setup.py ;
$PYTHON $BASE_DIR/ez_setup.py ;
EASY_INSTALL=`which easy_install` ;
fi
$GIT_EXEC clone $NOV_URL $BASE_DIR/nova;
chown -R $NOVA_USER: $BASE_DIR/nova
$EASY_INSTALL pip
$EASY_INSTALL pbr
cd $BASE_DIR/nova; $PYTHON setup.py install
#Create Nova Working directory
mkdir -p /opt/stack
mkdir -p /opt/stack/data
mkdir -p /opt/stack/data/nova
mkdir -p /opt/stack/data/nova/instances
mkdir -p /opt/stack/log
NOVA_COMPUTE_EXEX=`which nova-compute`
if [ -z "$NOVA_COMPUTE_EXEX" -o ! -x "$NOVA_COMPUTE_EXEX" ]; then
echo "ERROR: can't find nova-compute executable file!!!!"
exit 1
fi
cp -rf $BASE_DIR/nova/etc/nova /etc/
cat >$NOVA_CONFIG <<EOF
[DEFAULT]
firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver
compute_driver = libvirt.LibvirtDriver
flat_interface = eth0
flat_network_bridge = br100
vlan_interface = eth0
public_interface = br100
network_manager = nova.network.manager.FlatDHCPManager
host = ${:nova-host}
glance_api_servers = ${:slap-ipv4}:9292
rabbit_password = ${:nova-passwd}
rabbit_host = ${:slap-ipv4}
rpc_backend = nova.openstack.common.rpc.impl_kombu
ec2_dmz_host = ${:slap-ipv4}
vncserver_proxyclient_address = $IPv4
vncserver_listen = $IPv4
vnc_enabled = true
xvpvncproxy_base_url = http://$IPv4:6081/console
novncproxy_base_url = ${:vnc-url}/vnc_auto.html
logging_exception_prefix = %(color)s%(asctime)s.%(msecs)03d TRACE %(name)s ESC[01;35m%(instance)sESC[00m
logging_debug_format_suffix = ESC[00;33mfrom (pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)dESC[00m
logging_default_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [ESC[00;36m-%(color)s] ESC[01;35m%(instance)s%(color)s%(message)sESC[00m
logging_context_format_string = %(asctime)s.%(msecs)03d %(color)s%(levelname)s %(name)s [ESC[01;36m%(request_id)s ESC[00;36m%(user_name)s %(project_name)s%(color)s] ESC[01;35m%(instance)s%(color)s%(message)sESC[00m
instances_path = /opt/stack/data/nova/instances
lock_path = /opt/stack/data/nova
state_path = /opt/stack/data/nova
log_dir = /opt/stack/log
volume_api_class = nova.volume.cinder.API
enabled_apis = ec2,osapi_compute,metadata
instance_name_template = instance-%08x
libvirt_cpu_mode = none
libvirt_type = qemu
sql_connection = mysql://root:${:nova-passwd}@${:slap-ipv4}/nova?charset=utf8
my_ip = $IPv4
osapi_compute_extension = nova.api.openstack.compute.contrib.standard_extensions
s3_port = 3333
s3_host = ${:slap-ipv4}
ec2_host = ${:slap-ipv4}
rabbit_host = ${:slap-ipv4}
osapi_compute_listen = ${:slap-ipv4}
keystone_ec2_url = http://${:slap-ipv4}:5000/v2.0/ec2tokens
default_floating_pool = public
#iscsi_ip_prefix = 192.168.100
fixed_range =
force_dhcp_release = True
dhcpbridge_flagfile = /etc/nova/nova.conf
scheduler_driver = nova.scheduler.filter_scheduler.FilterScheduler
rootwrap_config = /etc/nova/rootwrap.conf
api_paste_config = /etc/nova/api-paste.ini
allow_resize_to_same_host = True
auth_strategy = keystone
debug = True
verbose = True
# Network settings
network_api_class=nova.network.neutronv2.api.API
neutron_url=http://${:slap-ipv4}:9696
neutron_auth_strategy=keystone
neutron_admin_tenant_name=service
neutron_admin_username=neutron
neutron_admin_password=${:nova-passwd}
neutron_admin_auth_url=http://${:slap-ipv4}:35357/v2.0
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
[osapi_v3]
enabled = True
[spice]
enabled = false
html5proxy_base_url = http://$IPv4:6082/spice_auto.html
EOF
#Add Nova-compute in init.d If file not exist now
cat > /etc/init.d/nova-compute << EOF
#! /bin/sh
### BEGIN INIT INFO
# Provides: skeleton
# Required-Start: \$remote_fs \$syslog
# Required-Stop: \$remote_fs \$syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Example initscript
# Description: This file should be used to construct scripts to be
# placed in /etc/init.d.
### END INIT INFO
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="OpenStack Conpute Node"
NAME=nova-compute
DAEMON=$NOVA_COMPUTE_EXEX
DAEMON_ARGS="--config-file $NOVA_CONFIG"
PIDFILE=/var/run/\$NAME.pid
SCRIPTNAME=/etc/init.d/\$NAME
# Exit if the package is not installed
[ -x "\$DAEMON" ] || exit 0
# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions
#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
start-stop-daemon --start --quiet --background -m --oknodo --pidfile \$PIDFILE --exec \$DAEMON --test > /dev/null \\
|| return 1
start-stop-daemon --start --quiet --background -m --oknodo --pidfile \$PIDFILE --exec \$DAEMON -- \\
\$DAEMON_ARGS \\
|| return 2
}
#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile \$PIDFILE --name \$NAME
RETVAL="\$?"
[ "\$RETVAL" = 2 ] && return 2
# Wait for children to finish too if this is a daemon that forks
# and if the daemon is only ever run from this initscript.
# If the above conditions are not satisfied then add some other code
# that waits for the process to drop all resources that could be
# needed by services started subsequently. A last resort is to
# sleep for some time.
start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec \$DAEMON
[ "\$?" = 2 ] && return 2
# Many daemons don't delete their pidfiles when they exit.
rm -f \$PIDFILE
return "\$RETVAL"
}
#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
#
# If the daemon can reload its configuration without
# restarting (for example, when it is sent a SIGHUP),
# then implement that here.
#
start-stop-daemon --stop --signal 1 --quiet --pidfile \$PIDFILE --name \$NAME
return 0
}
case "\$1" in
start)
[ "\$VERBOSE" != no ] && log_daemon_msg "Starting \$DESC" "\$NAME"
do_start
case "\$?" in
0|1) [ "\$VERBOSE" != no ] && log_end_msg 0 ;;
2) [ "\$VERBOSE" != no ] && log_end_msg 1 ;;
esac
;;
stop)
[ "\$VERBOSE" != no ] && log_daemon_msg "Stopping \$DESC" "\$NAME"
do_stop
case "\$?" in
0|1) [ "\$VERBOSE" != no ] && log_end_msg 0 ;;
2) [ "\$VERBOSE" != no ] && log_end_msg 1 ;;
esac
;;
status)
status_of_proc "\$DAEMON" "\$NAME" && exit 0 || exit \$?
;;
#reload|force-reload)
#
# If do_reload() is not implemented then leave this commented out
# and leave 'force-reload' as an alias for 'restart'.
#
#log_daemon_msg "Reloading \$DESC" "\$NAME"
#do_reload
#log_end_msg \$?
#;;
restart|force-reload)
#
# If the "reload" option is implemented then remove the
# 'force-reload' alias
#
log_daemon_msg "Restarting \$DESC" "\$NAME"
do_stop
case "\$?" in
0|1)
do_start
case "\$?" in
0) log_end_msg 0 ;;
1) log_end_msg 1 ;; # Old process is still running
*) log_end_msg 1 ;; # Failed to start
esac
;;
*)
# Failed to stop
log_end_msg 1
;;
esac
;;
*)
echo "Usage: \$SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
exit 3
;;
esac
:
EOF
chmod +x /etc/init.d/nova-compute
update-rc.d nova-compute defaults
/etc/init.d/nova-compute restart > /dev/null 2>&1
exit 0
software/openstack/templates/generate_cert.py 0 → 100644
View file @ d007050d
#!/usr/bin/env python
from OpenSSL import crypto, SSL
from socket import gethostname
from pprint import pprint
from time import gmtime, mktime
from os.path import exists, join
import sys
CERT_FILE = "certificate.crt"
KEY_FILE = "certificate.key"
def create_self_signed_cert(cert_dir):
"""
If datacard.crt and datacard.key don't exist in cert_dir, create a new
self-signed cert and keypair and write them into that directory.
"""
if not exists(join(cert_dir, CERT_FILE)) \
or not exists(join(cert_dir, KEY_FILE)):
# create a key pair
k = crypto.PKey()
k.generate_key(crypto.TYPE_RSA, 1024)
# create a self-signed cert
cert = crypto.X509()
cert.get_subject().C = "XX"
cert.get_subject().ST = "State"
cert.get_subject().L = "City"
cert.get_subject().O = "Sample Compagny"
cert.get_subject().OU = "Sample Organization"
cert.get_subject().CN = "XXXX"
cert.set_serial_number(1000)
cert.gmtime_adj_notBefore(0)
cert.gmtime_adj_notAfter(10*365*24*60*60)
cert.set_issuer(cert.get_subject())
cert.set_pubkey(k)
cert.sign(k, 'sha1')
open(join(cert_dir, CERT_FILE), "wt").write(
crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
open(join(cert_dir, KEY_FILE), "wt").write(
crypto.dump_privatekey(crypto.FILETYPE_PEM, k))
if __name__ == '__main__':
if len(sys.argv) > 1:
path = sys.argv[1]
basename = sys.argv[2]
CERT_FILE = basename + ".crt"
KEY_FILE = basename + ".key"
else:
path = "."
create_self_signed_cert(path)
exit(0)
software/openstack/templates/keystone_basic.sh 0 → 100644
View file @ d007050d
#!/bin/bash -xe
#
# Keystone basic configuration
# Mainly inspired by https://github.com/openstack/keystone/blob/master/tools/sample_data.sh
# Modified by Bilel Msekni / Institut Telecom
#
# Modified by Alain Takoudjou Kamdem
#
# Support: openstack@lists.launchpad.net
# License: Apache Software License (ASL) 2.0
#
source localrc
HOST_IP=${HOST_IP:-10.10.100.51}
ADMIN_PASSWORD=${ADMIN_PASSWORD:-openstack}
SERVICE_PASSWORD=${SERVICE_PASSWORD:-openstack}
export SERVICE_TOKEN="ADMIN"
export SERVICE_ENDPOINT="http://${HOST_IP}:35357/v2.0"
SERVICE_TENANT_NAME=${SERVICE_TENANT_NAME:-service}
get_id () {
echo `$@ | awk '/ id / { print $4 }'`
}
# Tenants
ADMIN_TENANT=$(get_id keystone tenant-create --name=admin)
SERVICE_TENANT=$(get_id keystone tenant-create --name=$SERVICE_TENANT_NAME)
# Users
ADMIN_USER=$(get_id keystone user-create --name=admin --pass="$ADMIN_PASSWORD" --email=admin@domain.com)
# Roles
ADMIN_ROLE=$(get_id keystone role-create --name=admin)
KEYSTONEADMIN_ROLE=$(get_id keystone role-create --name=KeystoneAdmin)
KEYSTONESERVICE_ROLE=$(get_id keystone role-create --name=KeystoneServiceAdmin)
# Add Roles to Users in Tenants
keystone user-role-add --user-id $ADMIN_USER --role-id $ADMIN_ROLE --tenant-id $ADMIN_TENANT
keystone user-role-add --user-id $ADMIN_USER --role-id $KEYSTONEADMIN_ROLE --tenant-id $ADMIN_TENANT
keystone user-role-add --user-id $ADMIN_USER --role-id $KEYSTONESERVICE_ROLE --tenant-id $ADMIN_TENANT
# The Member role is used by Horizon and Swift
MEMBER_ROLE=$(get_id keystone role-create --name=Member)
# Configure service users/roles
NOVA_USER=$(get_id keystone user-create --name=nova --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=nova@domain.com)
keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $NOVA_USER --role-id $ADMIN_ROLE
GLANCE_USER=$(get_id keystone user-create --name=glance --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=glance@domain.com)
keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $GLANCE_USER --role-id $ADMIN_ROLE
QUANTUM_USER=$(get_id keystone user-create --name=quantum --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=quantum@domain.com)
keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $QUANTUM_USER --role-id $ADMIN_ROLE
CINDER_USER=$(get_id keystone user-create --name=cinder --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=cinder@domain.com)
keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $CINDER_USER --role-id $ADMIN_ROLE
software/openstack/templates/keystone_endpoints_basic.sh 0 → 100644
View file @ d007050d
#!/bin/bash -xe
#
# Keystone basic Endpoints
# Mainly inspired by https://github.com/openstack/keystone/blob/master/tools/sample_data.sh
# Modified by Bilel Msekni / Institut Telecom
#
# Modified by Alain Takoudjou Kamdem
#
# Support: openstack@lists.launchpad.net
# License: Apache Software License (ASL) 2.0
#
source localrc
# Host address
HOST_IP=${HOST_IP:-10.10.100.51}
EXT_HOST_IP=$PUBLIC_ADDRESS
# MySQL definitions
MYSQL_USER=openstack
MYSQL_DATABASE=keystone
MYSQL_HOST=$HOST_IP
MYSQL_PASSWORD=$MYSQLPASS
# Keystone definitions
KEYSTONE_REGION=RegionOne
export SERVICE_TOKEN=ADMIN
export SERVICE_ENDPOINT="http://${HOST_IP}:35357/v2.0"
while getopts "u:D:p:m:K:R:E:T:vh" opt; do
case $opt in
u)
MYSQL_USER=$OPTARG
;;
D)
MYSQL_DATABASE=$OPTARG
;;
p)
MYSQL_PASSWORD=$OPTARG
;;
m)
MYSQL_HOST=$OPTARG
;;
K)
MASTER=$OPTARG
;;
R)
KEYSTONE_REGION=$OPTARG
;;
E)
export SERVICE_ENDPOINT=$OPTARG
;;
T)
export SERVICE_TOKEN=$OPTARG
;;
v)
set -x
;;
h)
cat <<EOF
Usage: $0 [-m mysql_hostname] [-u mysql_username] [-D mysql_database] [-p mysql_password]
[-K keystone_master ] [ -R keystone_region ] [ -E keystone_endpoint_url ]
[ -T keystone_token ]
Add -v for verbose mode, -h to display this message.
EOF
exit 0
;;
\?)
echo "Unknown option -$OPTARG" >&2
exit 1
;;
:)
echo "Option -$OPTARG requires an argument" >&2
exit 1
;;
esac
done
if [ -z "$KEYSTONE_REGION" ]; then
echo "Keystone region not set. Please set with -R option or set KEYSTONE_REGION variable." >&2
missing_args="true"
fi
if [ -z "$SERVICE_TOKEN" ]; then
echo "Keystone service token not set. Please set with -T option or set SERVICE_TOKEN variable." >&2
missing_args="true"
fi
if [ -z "$SERVICE_ENDPOINT" ]; then
echo "Keystone service endpoint not set. Please set with -E option or set SERVICE_ENDPOINT variable." >&2
missing_args="true"
fi
if [ -z "$MYSQL_PASSWORD" ]; then
echo "MySQL password not set. Please set with -p option or set MYSQL_PASSWORD variable." >&2
missing_args="true"
fi
if [ -n "$missing_args" ]; then
exit 1
fi
keystone service-create --name nova --type compute --description 'OpenStack Compute Service'
keystone service-create --name cinder --type volume --description 'OpenStack Volume Service'
keystone service-create --name glance --type image --description 'OpenStack Image Service'
keystone service-create --name keystone --type identity --description 'OpenStack Identity'
keystone service-create --name ec2 --type ec2 --description 'OpenStack EC2 service'
keystone service-create --name quantum --type network --description 'OpenStack Networking service'
create_endpoint () {
case $1 in
compute)
keystone endpoint-create --region $KEYSTONE_REGION --service-id $2 --publicurl 'http://'"$EXT_HOST_IP"':8774/v2/$(tenant_id)s' --adminurl 'http://'"$HOST_IP"':8774/v2/$(tenant_id)s' --internalurl 'http://'"$HOST_IP"':8774/v2/$(tenant_id)s'
;;
volume)
keystone endpoint-create --region $KEYSTONE_REGION --service-id $2 --publicurl 'http://'"$EXT_HOST_IP"':8776/v1/$(tenant_id)s' --adminurl 'http://'"$HOST_IP"':8776/v1/$(tenant_id)s' --internalurl 'http://'"$HOST_IP"':8776/v1/$(tenant_id)s'
;;
image)
keystone endpoint-create --region $KEYSTONE_REGION --service-id $2 --publicurl 'http://'"$EXT_HOST_IP"':9292/v2' --adminurl 'http://'"$HOST_IP"':9292/v2' --internalurl 'http://'"$HOST_IP"':9292/v2'
;;
identity)
keystone endpoint-create --region $KEYSTONE_REGION --service-id $2 --publicurl 'http://'"$EXT_HOST_IP"':5000/v2.0' --adminurl 'http://'"$HOST_IP"':35357/v2.0' --internalurl 'http://'"$HOST_IP"':5000/v2.0'
;;
ec2)
keystone endpoint-create --region $KEYSTONE_REGION --service-id $2 --publicurl 'http://'"$EXT_HOST_IP"':8773/services/Cloud' --adminurl 'http://'"$HOST_IP"':8773/services/Admin' --internalurl 'http://'"$HOST_IP"':8773/services/Cloud'
;;
network)
keystone endpoint-create --region $KEYSTONE_REGION --service-id $2 --publicurl 'http://'"$EXT_HOST_IP"':9696/' --adminurl 'http://'"$HOST_IP"':9696/' --internalurl 'http://'"$HOST_IP"':9696/'
;;
esac
}
for i in compute volume image object-store identity ec2 network; do
id=`mysql -h "$MYSQL_HOST" -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE" -ss -e "SELECT id FROM service WHERE type='"$i"';"` || exit 1
create_endpoint $i $id
done
software/openstack/templates/network.py 0 → 100644
View file @ d007050d
#!/usr/bin/env python
#Print Network address and generate range of 60 IPV4s for Openstack floatings IPs
import os
import sys
from netaddr import IPNetwork
def getNetwork(ipaddress, mask):
net = str(IPNetwork('%s/%s' % (ipaddress, mask)).cidr.network)
net_cidr = str(IPNetwork('%s/%s' % (ipaddress, mask)).cidr)
items = ipaddress.split('.')
base = "%s.%s.%s" % (items[0], items[1], items[2])
if int(items[3])+60 < 254:
ranges = "%s.%s %s.%s" % (base, (int(items[3])+1),
base, (int(items[3])+60))
else:
ranges = "%s.%s %s.%s" % (base, (int(items[3])-1),
base, (int(items[3])-60))
return net + " " + net_cidr + " " + ranges
if __name__ == '__main__':
print getNetwork(sys.argv[1], sys.argv[2])
exit(0)
\ No newline at end of file
software/openstack/templates/openstack-install.sh.in 0 → 100644
View file @ d007050d
#!/usr/bin/env bash
#configure.sh: Should automate Openstack installation with all needed packages.
REPO_URL="${:nova-url}"
NOVA_CONFIG="/etc/nova/nova.conf"
BASE_DIR="/opt/openstack"
INSTALL_DIR="/opt/stack"
INSTALL_LOG="/opt/openstack/log"
NOVA_PASSWD="${:nova-passwd}"
NOVA_USER="${:nova-user}"
FLOATING_RANGE=172.24.4.224/28
GIT_EXEC=`which git`
IPv4=`ip addr show eth0 | grep 'inet ' | awk '{print $2}' | cut -d/ -f1`
if [ -z "$IPv4" ];
then IPv4="127.0.0.1"
fi
sudo apt-get update
sudo apt-get -y upgrade
if [ -z "$GIT_EXEC" -o ! -x "$GIT_EXEC" ]; then
apt-get install -y git ;
fi
GIT_EXEC=`which git`
$GIT_EXEC clone $REPO_URL $BASE_DIR;
cd $BASE_DIR
$GIT_EXEC reset --hard ${:devstack-revision}
mkdir -p $INSTALL_LOG
cp -f $BASE_DIR/samples/local.sh $BASE_DIR/local.sh
cat >$BASE_DIR/localrc <<EOF
DATABASE_PASSWORD=$NOVA_PASSWD
RABBIT_PASSWORD=$NOVA_PASSWD
SERVICE_TOKEN=$NOVA_PASSWD
SERVICE_PASSWORD=$NOVA_PASSWD
ADMIN_PASSWORD=$NOVA_PASSWD
HOST_IP=$IPv4
LOGFILE=$INSTALL_LOG/stack.sh.log
LOGDAYS=7
# Nova Network Configuration
FLAT_INTERFACE=eth0
FIXED_RANGE=10.0.0.0/24
FIXED_NETWORK_SIZE=256
FLOATING_RANGE=$FLOATING_RANGE
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
# Optional, to enable tempest configuration as part of devstack
enable_service tempest
EOF
chmod 600 $BASE_DIR/localrc
#Install Openstack...
chown -R $NOVA_USER: $BASE_DIR
#sudo dpkg --configure -a
cd $BASE_DIR; ./stack.sh
cd $BASE_DIR; ./local.sh
if grep "$BASE_DIR/rejoin-stack.sh" /etc/rc.local > /dev/null
then
echo "Skipping configuring rc.local file..."
else
sudo sed -n '$!p' /etc/rc.local > temp-rc.local;
sudo mv temp-rc.local /etc/rc.local;
sudo echo -e "sudo losetup -f $INSTALL_DIR/data/stack-volumes-backing-file \n$BASE_DIR/rejoin-stack.sh > /dev/null \n\nexit 0" >> /etc/rc.local;
sudo chmod +x /etc/rc.local
fi
#Allow openstack images to access to internet
#sudo iptables -t nat -A POSTROUTING -s $FLOATING_RANGE -j MASQUERADE
exit 0
\ No newline at end of file
software/openstack/templates/ssh-configure.py.in 0 → 100644
View file @ d007050d
#!${:python_path}
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
# Echo client program
import os
import traceback
import sys
import time
pythonPath = []
eggs = '${:eggs-dir}'
for item in os.listdir(eggs):
path = os.path.join(eggs, item)
pythonPath.append(path)
sys.path[0:0] = pythonPath
import paramiko
password = '${:system-passwd}'
username = '${:system-user}'
port = ${:ssh-port}
hostname = '${:host-ip}'
ssh_run = os.path.join('${:nova-result}', 'nova-configure.log')
ssh_check = os.path.join('${:nova-result}', 'ssh_check.log')
nova_configure = '${:nova-configure}'
keystone_script = '${:keystone-script}'
keystone_endpoints_script = '${:keystone-endpoints-script}'
floating_ip_generate_script = '${:floating-ip-generate}'
auto_signed_certificate_script = '${:signed-certificate-generate}'
instance = '${:instance}'.strip() #Type: manage or compute
def getClient():
try:
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
print '*** Connecting...'
client.connect(hostname, port, username, password)
return client
except Exception, e:
print "Could not connect to host... Will retry after a few minutes"
return None
def installCompute():
#Install nova components and services
print "Installing nova..."
install_command = "echo %s | sudo -S /bin/bash -xe configure.sh 2>&1 | tee install.log" % password
stdin, stdout, stderr = client.exec_command(install_command)
with open(ssh_run, 'a') as output:
output.write(stdout.read())
#Check nova service status
print "Checking nova service status..."
nova_command = 'echo %s | sudo -S /etc/init.d/nova-compute status' % password
stdin, stdout, stderr = client.exec_command(nova_command)
with open(ssh_check, 'w') as output:
output.write(stdout.read())
def installManage():
#copy keystone configure and network address generate scripts
sftp = client.open_sftp()
result = sftp.put(keystone_script, 'keystone_basic.sh' )
print 'Keystone Basic Configuration script has been copied successfully! File size: %s' % result.st_size
result = sftp.put(keystone_endpoints_script, 'keystone_endpoints_basic.sh' )
print 'Keystone Endpoints Configuration script has been copied successfully! File size: %s' % result.st_size
result = sftp.put(floating_ip_generate_script, 'network.py' )
print 'Network generate script has been copied successfully! File size: %s' % result.st_size
result = sftp.put(auto_signed_certificate_script, 'generate_cert.py')
print 'Auto-signed certificate script has been copied successfully! File size: %s' % result.st_size
sftp.close()
#Install nova components and services
print "Downloading and installing Openstack on computer..."
install_command = "echo %s | sudo -S /bin/bash -xe configure.sh 2>&1 | tee install.log" % password
stdin, stdout, stderr = client.exec_command(install_command)
with open(ssh_run, 'w') as output:
output.write(stdout.read())
#Check nova service status
print "Checking nova service status..."
nova_command = 'echo %s | sudo -S nova-manage service list' % password
stdin, stdout, stderr = client.exec_command(nova_command)
with open(ssh_check, 'w') as output:
output.write(stdout.read())
if __name__ == '__main__':
client = None
while not client:
client = getClient()
time.sleep(5)
try:
#Try to run openstack command before all other system process are started
time.sleep(20)
#Write file configure.sh to use.
sftp = client.open_sftp()
result = sftp.put(nova_configure, 'configure.sh' )
print 'Nova configuration scipt has been copied successfully! File size: %s' % result.st_size
sftp.close()
if instance == "manage":
installManage()
elif instance == "compute":
installCompute()
client.close()
except Exception, e:
print '*** Caught exception: %s: %s' % (e.__class__, e)
traceback.print_exc()
try:
client.close()
except:
pass
sys.exit(1)
\ No newline at end of file
software/openstack/templates/stack_controller_grizzly.sh.in 0 → 100644
View file @ d007050d
#!/bin/bash -xe
#stack_controller_grizzly.sh: Should install and configure all needed openstack packages.
#This script should work on SlapOS environment, using a Virtual Machine
# Warning: this script may not work on Ubuntu 12.04 - Use ubuntu 13.04
export DEBIAN_FRONTEND=noninteractive
###############################################################################################
##### Check Installation and internet connexion
###############################################################################################
if [ -f .installed ]; then
echo "An installation of openstack already exist. Exiting..."
exit 0;
fi
wget -q --tries=10 --timeout=20 http://google.com
if [[ $? -eq 0 ]]; then
echo "OK: Succefully check internet connection with success. Go next ...";
else
echo "ERROR: We can't install openstack without internet connection. Please check it before retry installation...";
exit 1;
fi
###############################################################################################
apt-get update
apt-get upgrade -y
apt-get dist-upgrade -y
###############################################################################################
##### Configure Network
###############################################################################################
apt-get install -y openvswitch-switch openvswitch-datapath-dkms
apt-get install -y vlan bridge-utils
#br-int will be used for VM integration
ovs-vsctl add-br br-int
#br-ex is used to make to access the internet (not covered in this guide)
ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex eth0
if [ -f network.conf ]
then
IPv4=`cat network.conf | cut -d: -f1`
MASK=`cat network.conf | cut -d: -f2`
GATEWAY=`cat network.conf | cut -d: -f3`
BCAST=`cat network.conf | cut -d: -f4`
else
GATEWAY=`route | grep default | awk '{print $2}'`
MASK=`ifconfig eth0 | egrep '(inet ad)d?r:' | awk '{ print $4}' | cut -d: -f2`
BCAST=`ifconfig eth0 | egrep '(inet ad)d?r:' | awk '{ print $3}' | cut -d: -f2`
IPv4=`ip addr show eth0 | grep 'inet ' | awk '{print $2}' | cut -d/ -f1`
echo "$IPv4:$MASK:$GATEWAY:$BCAST" > network.conf
fi
/bin/cat << EOF > /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
#For Exposing OpenStack API over the internet
# VM internet Access
auto eth0
iface eth0 inet manual
up ifconfig \$IFACE 0.0.0.0 up
up ip link set \$IFACE promisc on
down ip link set \$IFACE promisc off
down ifconfig \$IFACE down
auto br-ex
iface br-ex inet static
address $IPv4
netmask $MASK
broadcast $BCAST
gateway $GATEWAY
dns-nameservers 8.8.8.8
#Not internet connected(used for OpenStack management)
#auto eth0
#iface eth0 inet static
#address 10.10.100.51
#netmask 255.255.255.0
EOF
chmod 644 /etc/network/interfaces
/etc/init.d/networking restart
###############################################################################################
##### Network Fowarding
###############################################################################################
sed -i 's/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/' /etc/sysctl.conf
# To save you from rebooting, perform the following
sysctl net.ipv4.ip_forward=1
iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
iptables --append FORWARD --in-interface br-ex -j ACCEPT
#Add iptable command to rc.local to allow auto reconfiguring of network NAT after reboot
sudo sed -n '$!p' /etc/rc.local > temp-rc.local
sudo mv temp-rc.local /etc/rc.local
sudo chmod +x /etc/rc.local
/bin/cat << EOF >> /etc/rc.local
iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
iptables --append FORWARD --in-interface br-ex -j ACCEPT
exit 0
EOF
###############################################################################################
##### Variables to use
###############################################################################################
/bin/cat << EOF > localrc
MYSQLPASS=${:nova-passwd}
ADMIN_PASSWORD=${:nova-passwd}
SERVICE_PASSWORD=${:nova-passwd}
RABBIT_PASS=${:nova-passwd}
HOST_IP=$IPv4
PUBLIC_ADDRESS=$IPv4
EXT_HOST_IP=$IPv4
USER_PASSWORD=\$ADMIN_PASSWORD
USER_NAME=${:user-name}
PROJECT=${:project}
EOF
source localrc
CURRENT_DIR=`pwd`
VNC_BASE_URL="${:vnc-frontend}"
if [ -z "`echo $VNC_BASE_URL | egrep https?://`" ]; then
VNC_BASE_URL=https://$PUBLIC_ADDRESS:6080/vnc_auto.html
fi
###############################################################################################
##### Install Mysql, ntp, MysqlDB and RabbitMQ
###############################################################################################
apt-get install -y mysql-server python-mysqldb
mysqladmin -u root password $MYSQLPASS
/sbin/stop mysql
sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
service mysql restart
sleep 5
apt-get install -y rabbitmq-server
apt-get install -y ntp
#Comment the ubuntu NTP servers
sed -i 's/server 0.ubuntu.pool.ntp.org/#server 0.ubuntu.pool.ntp.org/g' /etc/ntp.conf
sed -i 's/server 1.ubuntu.pool.ntp.org/#server 1.ubuntu.pool.ntp.org/g' /etc/ntp.conf
sed -i 's/server 2.ubuntu.pool.ntp.org/#server 2.ubuntu.pool.ntp.org/g' /etc/ntp.conf
sed -i 's/server 3.ubuntu.pool.ntp.org/#server 3.ubuntu.pool.ntp.org/g' /etc/ntp.conf
sed -e 's/server ntp.ubuntu.com/server $PUBLIC_ADDRESS/g' /etc/ntp.conf
service ntp restart
sleep 1.5
##############################################################################
## Create MySQL accounts and databases of Nova, Glance, Keystone and Cinder
##############################################################################
/bin/cat << EOF | /usr/bin/mysql -uroot -p$MYSQLPASS
DROP DATABASE IF EXISTS keystone;
DROP DATABASE IF EXISTS glance;
DROP DATABASE IF EXISTS nova;
DROP DATABASE IF EXISTS cinder;
DROP DATABASE IF EXISTS horizon;
DROP DATABASE IF EXISTS quantum;
CREATE DATABASE keystone;
CREATE DATABASE glance;
CREATE DATABASE nova;
CREATE DATABASE cinder;
CREATE DATABASE horizon;
CREATE DATABASE quantum;
GRANT ALL ON keystone.* TO 'openstack'@'%' IDENTIFIED BY '$MYSQLPASS';
GRANT ALL ON glance.* TO 'openstack'@'%' IDENTIFIED BY '$MYSQLPASS';
GRANT ALL ON nova.* TO 'openstack'@'%' IDENTIFIED BY '$MYSQLPASS';
GRANT ALL ON cinder.* TO 'openstack'@'%' IDENTIFIED BY '$MYSQLPASS';
GRANT ALL ON horizon.* TO 'openstack'@'%' IDENTIFIED BY '$MYSQLPASS';
GRANT ALL ON quantum.* TO 'openstack'@'%' IDENTIFIED BY '$MYSQLPASS';
EOF
###############################################################################################
##### Install Keystone
###############################################################################################
apt-get install -y keystone
service keystone status
CONF=/etc/keystone/keystone.conf
cp $CONF $CONF.orig
/bin/sed \
-e "s/^#*connection *=.*/connection = mysql:\/\/openstack:$MYSQLPASS@$HOST_IP\/keystone/" \
$CONF.orig > $CONF
# -e "s/^#* *admin_token *=.*/admin_token = $ADMIN_PASSWORD/" \
service keystone restart
sleep 1.5
keystone-manage db_sync
sleep 1.5
cd $CURRENT_DIR
source localrc
chmod +x keystone_basic.sh
chmod +x keystone_endpoints_basic.sh
./keystone_basic.sh
./keystone_endpoints_basic.sh
/bin/cat << EOF > creds
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=$ADMIN_PASSWORD
export OS_AUTH_URL="http://$EXT_HOST_IP:5000/v2.0/"
EOF
source creds
keystone user-list
###############################################################################################
##### Install and configure Glance
###############################################################################################
apt-get install -y glance
service glance-api status
service glance-registry status
CONF=/etc/glance/glance-api.conf
cp $CONF $CONF.orig
/bin/sed \
-e "s#^sql_connection *=.*#sql_connection = mysql://openstack:$MYSQLPASS@$HOST_IP/glance#" \
-e 's/^#*flavor *=.*/flavor = keystone/' \
$CONF.orig > $CONF
CONF=/etc/glance/glance-api-paste.ini
cp $CONF $CONF.orig
sed -e "/delay_auth_decision/a\auth_host = $HOST_IP" \
-e "/delay_auth_decision/a\auth_port = 35357" \
-e "/delay_auth_decision/a\auth_protocol = http" \
-e "/delay_auth_decision/a\admin_tenant_name = service" \
-e "/delay_auth_decision/a\admin_user = glance" \
-e "/delay_auth_decision/a\admin_password = $ADMIN_PASSWORD" \
$CONF.orig > $CONF
# -e "s/^auth_host *=.*/auth_host = $HOST_IP/" \
# -e 's/%SERVICE_TENANT_NAME%/service/' \
# -e 's/%SERVICE_USER%/glance/' \
# -e "s/%SERVICE_PASSWORD%/$SERVICE_PASSWORD/" \
# -e 's[^#* *config_file *=.*[config_file = /etc/glance/glance-api-paste.ini[' \
# -e "s/127.0.0.1/$HOST_IP/" \
# -e "s/localhost/$HOST_I/" \
# -e "s/^rabbit_host *=.*/rabbit_host = 127.0.0.1/" \
# -e 's/^notifier_strategy *=.*/notifier_strategy = rabbit/' \
# -e "s/^rabbit_host *=.*/rabbit_host = $HOST_IP/" \
# -e 's/^rabbit_userid *=.*/rabbit_userid = nova/' \
# -e "s/^rabbit_password *=.*/rabbit_password = $RABBIT_PASS/" \
# -e "s/^rabbit_virtual_host *=.*/rabbit_virtual_host = \/nova/" \
CONF=/etc/glance/glance-registry.conf
cp $CONF $CONF.orig
/bin/sed \
-e "s/^sql_connection *=.*/sql_connection = mysql:\/\/openstack:$MYSQLPASS@$HOST_IP\/glance/" \
-e 's/^#*flavor *=.*/flavor=keystone/' \
$CONF.orig > $CONF
# -e "s/127.0.0.1/$HOST_IP/" \
# -e "s/localhost/$HOST_IP/" \
# -e 's/^#* *config_file *=.*/config_file = \/etc\/glance\/glance-registry-paste.ini/' \
# -e "s/^auth_host *=.*/auth_host = $HOST_IP/" \
# -e 's/%SERVICE_TENANT_NAME%/service/' \
# -e 's/%SERVICE_USER%/glance/' \
# -e "s/%SERVICE_PASSWORD%/$ADMIN_PASSWORD/" \
CONF=/etc/glance/glance-registry-paste.ini
cp $CONF $CONF.orig
sed -e "/keystoneclient.middleware.auth_token:filter_factory/a\auth_host = $HOST_IP" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\auth_port = 35357" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\auth_protocol = http" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\admin_tenant_name = service" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\admin_user = glance" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\admin_password = $ADMIN_PASSWORD" \
$CONF.orig > $CONF
service glance-api restart; service glance-registry restart
sleep 1.5
glance-manage db_sync
service glance-registry restart; service glance-api restart
sleep 1.5
glance image-create --name ${:image-name} --is-public true --container-format bare --disk-format qcow2 --location ${:image-url}
glance image-list
###############################################################################################
##### Install and configure Quantum
###############################################################################################
apt-get install -y quantum-server quantum-plugin-openvswitch quantum-plugin-openvswitch-agent dnsmasq quantum-dhcp-agent quantum-l3-agent
#Replace configuration file:
CONF=/etc/quantum/api-paste.ini
cp $CONF $CONF.orig
sed -e "/keystoneclient.middleware.auth_token:filter_factory/a\auth_host = $HOST_IP" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\auth_port = 35357" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\auth_protocol = http" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\admin_tenant_name = service" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\admin_user = quantum" \
-e "/keystoneclient.middleware.auth_token:filter_factory/a\admin_password = $ADMIN_PASSWORD" \
$CONF.orig > $CONF
CONF=/etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
cp $CONF $CONF.orig
sed -e "s/^sql_connection *=.*/sql_connection = mysql:\/\/openstack:$MYSQLPASS@$HOST_IP\/quantum/" \
-e "s/^# firewall_driver *=.*/firewall_driver = quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver/" \
-e "/^# Example: bridge_mappings/a\tenant_network_type = gre" \
-e "/^# Example: bridge_mappings/a\tunnel_id_ranges = 1:1000" \
-e "/^# Example: bridge_mappings/a\integration_bridge = br-int" \
-e "/^# Example: bridge_mappings/a\tunnel_bridge = br-tun" \
-e "/^# Example: bridge_mappings/a\local_ip = $HOST_IP" \
-e "/^# Example: bridge_mappings/a\enable_tunneling = True" \
$CONF.orig > $CONF
CONF=/etc/quantum/metadata_agent.ini
cp $CONF $CONF.orig
sed -e "s/^auth_url *=.*/auth_url = http:\/\/$HOST_IP:35357\/v2.0/" \
-e 's/%SERVICE_TENANT_NAME%/service/' \
-e 's/%SERVICE_USER%/quantum/' \
-e "s/%SERVICE_PASSWORD%/$ADMIN_PASSWORD/" \
-e "s/^# nova_metadata_ip *=.*/nova_metadata_ip = 127.0.0.1/" \
-e "s/^# nova_metadata_port *=.*/nova_metadata_port = 8775/" \
-e "s/^# metadata_proxy_shared_secret *=.*/metadata_proxy_shared_secret = helloOpenStack/" \
$CONF.orig > $CONF
CONF=/etc/quantum/quantum.conf
cp $CONF $CONF.orig
sed -e "s/^auth_host *=.*/auth_host = $HOST_IP/" \
-e 's/%SERVICE_TENANT_NAME%/service/' \
-e 's/%SERVICE_USER%/quantum/' \
-e "s/%SERVICE_PASSWORD%/$ADMIN_PASSWORD/" \
$CONF.orig > $CONF
echo "Giving quantum user passwordless sudo privileges"
# UEC images ``/etc/sudoers`` does not have a ``#includedir``, add one
grep -q "^#includedir.*/etc/sudoers.d" /etc/sudoers ||
echo "#includedir /etc/sudoers.d" >> /etc/sudoers
( umask 226 && echo "quantum ALL=(ALL) NOPASSWD:ALL" \
> /etc/sudoers.d/quantum_sudoers )
cd /etc/init.d/; for i in $( ls quantum-* ); do sudo service $i restart; done
sleep 1.5
service dnsmasq restart
sleep 1.5
###############################################################################################
##### Install and configure Nova
###############################################################################################
apt-get -y install cpu-checker
#kvm-ok
#sleep 1.5
apt-get install -y kvm libvirt-bin pm-utils
CONF=/etc/libvirt/qemu.conf
cp $CONF $CONF.orig
/bin/cat << EOF >> $CONF
cgroup_device_acl = [
"/dev/null", "/dev/full", "/dev/zero",
"/dev/random", "/dev/urandom",
"/dev/ptmx", "/dev/kvm", "/dev/kqemu",
"/dev/rtc", "/dev/hpet","/dev/net/tun"
]
EOF
service dbus restart && service libvirt-bin restart
sleep 1.5
virsh net-destroy default
virsh net-undefine default
CONF=/etc/libvirt/libvirtd.conf
cp $CONF $CONF.orig
/bin/sed \
-e 's/^#listen_tls *=.*/listen_tls = 0/' \
-e 's/^#listen_tcp *=.*/listen_tcp = 1/' \
-e 's/^#auth_tcp *=.*/auth_tcp = "none"/' \
$CONF.orig > $CONF
CONF=/etc/init/libvirt-bin.conf
cp $CONF $CONF.orig
/bin/sed \
-e 's/^env libvirtd_opts=.*/env libvirtd_opts="-d -l"/' \
$CONF.orig > $CONF
CONF=/etc/default/libvirt-bin
cp $CONF $CONF.orig
/bin/sed \
-e 's/^libvirtd_opts=.*/libvirtd_opts="-d -l"/' \
$CONF.orig > $CONF
service dbus restart && service libvirt-bin restart
sleep 1.5
#Install Nova-* services
apt-get install -y nova-api nova-cert novnc nova-consoleauth nova-scheduler nova-novncproxy nova-doc nova-conductor nova-compute-kvm
cd /etc/init.d/; for i in $( ls nova-* ); do service $i status; cd; done
CONF=/etc/nova/api-paste.ini
cp $CONF $CONF.orig
/bin/sed \
-e "s/^auth_host *=.*/auth_host = $HOST_IP/" \
-e 's/%SERVICE_TENANT_NAME%/service/' \
-e 's/%SERVICE_USER%/nova/' \
-e "s/%SERVICE_PASSWORD%/$ADMIN_PASSWORD/" \
-e "s/^#signing_dir *=.*/signing_dirname = \/tmp\/keystone-signing-nova/" \
$CONF.orig > $CONF
#Generate certificate to use for nova
cd $CURRENT_DIR
chmod +x generate_cert.py
mkdir -p /etc/nova/ssl
./generate_cert.py /etc/nova/ssl nova
chown -R nova: /etc/nova/ssl
chmod -R 700 /etc/nova/ssl
#Configure nova
CONF=/etc/nova/nova.conf
cp $CONF $CONF.orig
/bin/cat << EOF > $CONF
[DEFAULT]
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/run/lock/nova
verbose=True
api_paste_config=/etc/nova/api-paste.ini
compute_scheduler_driver=nova.scheduler.simple.SimpleScheduler
rabbit_host=$HOST_IP
nova_url=http://$HOST_IP:8774/v1.1/
sql_connection=mysql://openstack:$MYSQLPASS@$HOST_IP/nova
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
# Auth
use_deprecated_auth=false
auth_strategy=keystone
# Imaging service
glance_api_servers=$HOST_IP:9292
image_service=nova.image.glance.GlanceImageService
# Vnc configuration
novnc_enabled=true
ssl_only=true
cert=/etc/nova/ssl/nova.crt
key=/etc/nova/ssl/nova.key
novncproxy_base_url=$VNC_BASE_URL
novncproxy_port=6080
vncserver_proxyclient_address=$HOST_IP
vncserver_listen=0.0.0.0
# Network settings
network_api_class=nova.network.quantumv2.api.API
quantum_url=http://$HOST_IP:9696
quantum_auth_strategy=keystone
quantum_admin_tenant_name=service
quantum_admin_username=quantum
quantum_admin_password=$SERVICE_PASSWORD
quantum_admin_auth_url=http://$HOST_IP:35357/v2.0
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
#If you want Quantum + Nova Security groups
firewall_driver=nova.virt.firewall.NoopFirewallDriver
security_group_api=quantum
#If you want Nova Security groups only, comment the two lines above and uncomment line -1-.
#-1-firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
#Metadata
service_quantum_metadata_proxy = True
quantum_metadata_proxy_shared_secret = helloOpenStack
metadata_host = $HOST_IP
metadata_listen = 127.0.0.1
metadata_listen_port = 8775
# Compute #
compute_driver=libvirt.LibvirtDriver
# Cinder #
volume_api_class=nova.volume.cinder.API
osapi_volume_listen_port=5900
EOF
CONF=/etc/nova/nova-compute.conf
cp $CONF $CONF.orig
/bin/cat << EOF > $CONF
[DEFAULT]
libvirt_type=qemu
libvirt_ovs_bridge=br-int
libvirt_vif_type=ethernet
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
libvirt_use_virtio_for_bridges=True
EOF
nova-manage db sync
cd /etc/init.d/; for i in $( ls nova-* ); do sudo service $i restart; done
sleep 1.5
nova-manage service list
sleep 1.5
###############################################################################################
##### Install and configure Cinder
###############################################################################################
apt-get install -y cinder-api cinder-scheduler cinder-volume iscsitarget open-iscsi iscsitarget-dkms
sed -i 's/false/true/g' /etc/default/iscsitarget
service iscsitarget start
service open-iscsi start
CONF=/etc/cinder/cinder.conf
cp $CONF $CONF.orig
/bin/cat << EOF > $CONF
[DEFAULT]
rootwrap_config = /etc/cinder/rootwrap.conf
api_paste_confg = /etc/cinder/api-paste.ini
iscsi_helper = ietadm
volume_name_template = volume-%s
volume_group = cinder-volumes
verbose = True
auth_strategy = keystone
state_path = /var/lib/cinder
lock_path = /var/lock/cinder
volumes_dir = /var/lib/cinder/volumes
# LOGGING
log_file=cinder.log
log_dir=/var/log/cinder
# OSAPI
osapi_volume_extension = cinder.api.openstack.volume.contrib.standard_extensions
osapi_max_limit = 2000
# RABBIT
#rabbit_host=$HOST_IP
#rabbit_virtual_host=/nova
#rabbit_userid=nova
#rabbit_password=$RABBIT_PASS
# MYSQL
sql_connection = mysql://openstack:$MYSQLPASS@$HOST_IP/cinder
debug = True
EOF
CONF=/etc/cinder/api-paste.ini
cp $CONF $CONF.orig
/bin/sed \
-e "s/^service_host *=.*/service_host = $PUBLIC_ADDRESS/" \
-e "s/^auth_host *=.*/auth_host = $HOST_IP/" \
-e 's/%SERVICE_TENANT_NAME%/service/' \
-e 's/%SERVICE_USER%/cinder/' \
-e "s/%SERVICE_PASSWORD%/$ADMIN_PASSWORD/" \
$CONF.orig > $CONF
cinder-manage db sync
CINDER_VOL_DIR=/var/lib/cinder
cd $CINDER_VOL_DIR
dd if=/dev/zero of=cinder-volumes bs=1 count=0 seek=2G
losetup /dev/loop2 cinder-volumes
#Skip this part
#fdisk /dev/loop2
#n
#p
#1
#ENTER
#ENTER
#t
#8e
#w
pvcreate /dev/loop2
vgcreate cinder-volumes /dev/loop2
/bin/cat << EOF > /etc/init.d/cinder-setup-backing-file
losetup /dev/loop2 $CINDER_VOL_DIR/cinder-volumes
exit 0
EOF
sudo chmod 755 /etc/init.d/cinder-setup-backing-file
sudo ln -s /etc/init.d/cinder-setup-backing-file /etc/rc2.d/S10cinder-setup-backing-file
cd $CURRENT_DIR
###############################################################################################
##### Restore user access on files
###############################################################################################
for i in nova keystone glance cinder
do
chown -R $i /etc/$i
done
rm -r /var/log/cinder/cinder*
cd /etc/init.d/; for i in $( ls cinder-* ); do sudo service $i restart; done
sleep 1.5
cd /etc/init.d/; for i in $( ls cinder-* ); do sudo service $i status; done
sleep 1.5
###############################################################################################
##### Install Horizon
###############################################################################################
apt-get -y install openstack-dashboard memcached
dpkg --purge openstack-dashboard-ubuntu-theme
service apache2 restart; service memcached restart
###############################################################################################
##### Configure quantum network
###############################################################################################
apt-get install -y python-netaddr
cd $CURRENT_DIR
function get_id () {
echo `"$@" | awk '/ id / { print $4 }'`
}
source creds
MEMDER_ROLE=$(keystone role-list | grep " Member" | awk '{print $2}')
ADMIN_ROLE=$(keystone role-list | grep " admin" | awk '{print $2}')
L3_AGENT_ID=$(quantum agent-list | grep " L3 agent" | awk '{print $2}')
PROJECT_ID=$(get_id keystone tenant-create --name $PROJECT)
USER_ID=$(get_id keystone user-create --name=$USER_NAME --pass=$USER_PASSWORD --tenant-id $PROJECT_ID --email=$USER_NAME@domain.com)
keystone user-role-add --tenant-id $PROJECT_ID --user-id $USER_ID --role-id $MEMDER_ROLE
quantum net-create --tenant-id $PROJECT_ID net_$PROJECT
SUBNET_ID=$(get_id quantum subnet-create --tenant-id $PROJECT_ID net_$PROJECT 50.50.1.0/24 --dns_nameservers list=true 8.8.8.7 8.8.8.8)
ROUTER_ID=$(get_id quantum router-create --tenant-id $PROJECT_ID router_$PROJECT)
quantum l3-agent-router-add $L3_AGENT_ID router_$PROJECT
quantum router-interface-add $ROUTER_ID $SUBNET_ID
cd /etc/init.d/; for i in $( ls quantum-* ); do sudo service $i restart; done
sleep 8
cd $CURRENT_DIR
chmod +x network.py
EXT_NET_ID=$(get_id quantum net-create --tenant-id $ADMIN_ROLE ext_net --router:external=True)
NETWORK=`./network.py $IPv4 $MASK | cut -d' ' -f2`
POOL_START=`./network.py $IPv4 $MASK | cut -d' ' -f3`
POOL_END=`./network.py $IPv4 $MASK | cut -d' ' -f4`
quantum subnet-create --tenant-id $ADMIN_ROLE --allocation-pool start=$POOL_START,end=$POOL_END --gateway $IPv4 ext_net $NETWORK --enable_dhcp=False
quantum router-gateway-set $ROUTER_ID $EXT_NET_ID
/bin/cat << EOF > creds_$PROJECT
export OS_TENANT_NAME=$PROJECT
export OS_USERNAME=$USER_NAME
export OS_PASSWORD=$USER_PASSWORD
export OS_AUTH_URL="http://$PUBLIC_ADDRESS:5000/v2.0/"
EOF
source creds_$PROJECT
nova --no-cache secgroup-add-rule default icmp -1 -1 0.0.0.0/0
nova --no-cache secgroup-add-rule default tcp 22 22 0.0.0.0/0
quantum floatingip-create ext_net
#End Installation ...
echo "done" > $CURRENT_DIR/.installed
\ No newline at end of file
software/slapos-in-partition/common.cfg 0 → 100644
View file @ d007050d
[buildout]
extends =
../../component/apache/buildout.cfg
../../component/bash/buildout.cfg
../../component/dcron/buildout.cfg
../../component/dropbear/buildout.cfg
../../component/gzip/buildout.cfg
../../component/logrotate/buildout.cfg
../../stack/slapos.cfg
parts =
instance-profile
slapos-cookbook
eggs
# Add hosting location of testing version of slapos.core
find-links +=
http://www.nexedi.org/static/packages/source/slapos.core-testing/
[environment]
recipe = collective.recipe.environment
[instance-profile]
# 3 advantages of using jinja2 for ALL templates:
# 1/ Explicit scope (pythonic style, we explicitely list what we want to be in the scope)
# 2/ No troubles between $ and $$ (more simple)
# 3/ We can explicitely define the path of executables (i.e
# in software, define httpd-executable = ${apache:location}/bin/httpd
# and in instance, just use httpd-executable without bother where it is actually
# (location can change inside of the component, from bin to sbin for example).
recipe = slapos.recipe.template:jinja2
template = ${:_profile_base_location_}/instance.cfg.jinja2
rendered = ${buildout:directory}/instance.cfg
#md5sum = 4861be4a581686feef9f9edea865d7ee
mode = 0644
context =
key bin_directory buildout:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key path environment:PATH
raw httpd_executable ${apache:location}/bin/httpd
raw bash_executable ${bash:location}/bin/bash
raw dcron_executable ${dcron:location}/sbin/crond
raw dropbear_executable ${dropbear:location}/sbin/dropbear
raw dropbearkey_executable ${dropbear:location}/bin/dropbearkey
raw gzip_executable ${gzip:location}/bin/gzip
raw gunzip_executable ${gzip:location}/bin/gunzip
raw logrotate_executable ${logrotate:location}/sbin/logrotate
raw slapos_configuration_file_template_path ${slapos-configuration-file-template:target}
raw httpd_configuration_file_template_path ${httpd-configuration-file-template:target}
[slapos-configuration-file-template]
# Download the template of slapos.cfg
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/template/slapos.cfg.in
#md5sum =
target = ${buildout:directory}/slapos.cfg.in
mode = 0644
[httpd-configuration-file-template]
# Download the template of httpd.conf
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/template/httpd.conf.in
mode = 0644
#md5sum =
#target = ${
[eggs]
recipe = zc.recipe.egg
eggs =
collective.recipe.template
# Add slapos.libnetworkcache to path of slapos.core.
[slapos-cookbook]
eggs =
${lxml-python:egg}
slapos.cookbook
cliff
hexagonit.recipe.download
inotifyx
netaddr
netifaces
requests
slapos.core
supervisor
xml_marshaller
pytz
slapos.libnetworkcache
software/slapos-in-partition/instance.cfg.jinja2 0 → 100644
View file @ d007050d
[buildout]
parts =
slapos-configuration-file
cron-entry-slapos
slapos-node-status-wrapper
slapos-node-format-wrapper-script
httpd-wrapper
cron
logrotate
logrotate-entry-httpd
logrotate-entry-slapos
sshkeys-dropbear
dropbear-server-add-authorized-key
sshkeys-authority
publish-connection-informations
dropbear-promise
httpd-promise
slapos-promise
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[instance-parameter]
recipe = slapos.cookbook:slapconfiguration
computer = ${slap_connection:computer_id}
partition = ${slap_connection:partition_id}
url = ${slap_connection:server_url}
key = ${slap_connection:key_file}
cert = ${slap_connection:cert_file}
configuration.master-url = https://slap.vifib.com
configuration.authorized-key =
# Create all needed directories
[directory]
recipe = slapos.cookbook:mkdirectory
mode = 0750
etc = ${buildout:directory}/etc/
var = ${buildout:directory}/var/
srv = ${buildout:directory}/srv/
bin = ${buildout:directory}/bin/
sshkeys = ${:srv}/sshkeys
service = ${:etc}/service/
script = ${:etc}/run/
ssh = ${:etc}/ssh/
log = ${:var}/log/
run = ${:var}/run/
backup = ${:srv}/backup/
promises = ${:etc}/promise/
slapos-partitions-certificate-repository = ${:var}/pki
software-root = ${:srv}/slapos-software
instance-root = ${:srv}/slapos-instance
slapos-log = ${:log}/slapos
{% for i in range(0,10) %}
slappart{{i}} = ${:instance-root}/slappart{{i}}
{% endfor %}
cron-entries = ${:etc}/cron.d
crontabs = ${:etc}/crontabs
cronstamps = ${:etc}/cronstamps
logrotate-entries = ${:etc}/logrotate.d
logrotate-backup = ${:backup}/logrotate
httpd-log = ${:log}/httpd
########
# Deploy slapos.cfg, computer certificates and slapos node wrapper
########
[slapos-computer-certificate-file]
recipe = collective.recipe.template
input = inline:${instance-parameter:configuration.computer-certificate}
output = ${directory:var}/slapos-computer.crt
[slapos-computer-key-file]
recipe = collective.recipe.template
input = inline:${instance-parameter:configuration.computer-key}
output = ${directory:var}/slapos-computer.key
[computer-definition-file]
recipe = collective.recipe.template
input = inline:
[computer]
{% for i in range(0,10|int) %}
[partition_{{i}}]
address = ${instance-parameter:ipv4-random}/255.255.255.0 ${instance-parameter:ipv6-random}/64
pathname = slappart{{i}}
user = dummy
network_interface = dummy
{% endfor %}
output = ${directory:etc}/slapos-computer-definition.cfg
[slapos-configuration-file]
recipe = slapos.recipe.template
url = {{ slapos_configuration_file_template_path }}
output = ${directory:etc}/slapos.cfg
#md5sum = 4861be4a581686feef9f9edea865d7ee
software-root = ${directory:software-root}
instance-root = ${directory:instance-root}
master-url = ${instance-parameter:configuration.master-url}
computer-id = ${instance-parameter:configuration.computer-id}
# XXX should be a parameter
partition-amount = 10
computer-definition-file = ${computer-definition-file:output}
computer-xml = ${directory:var}/slapos.xml
computer-key-file = ${slapos-computer-key-file:output}
computer-certificate-file = ${slapos-computer-certificate-file:output}
certificate-repository-path = ${directory:slapos-partitions-certificate-repository}
[slapos-node-instance-wrapper]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/slapos node instance --cfg ${slapos-configuration-file:output} --pidfile ${directory:run}/slapos-instance.pid --logfile ${directory:slapos-log}/slapos-instance.log
wrapper-path = ${directory:bin}/slapos-node-instance
parameters-extra = true
[slapos-node-software-wrapper]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/slapos node software --cfg ${slapos-configuration-file:output} --pidfile ${directory:run}/slapos-software.pid --logfile ${directory:slapos-log}/slapos-software.log
wrapper-path = ${directory:bin}/slapos-node-software
parameters-extra = true
[slapos-node-report-wrapper]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/slapos node report --cfg ${slapos-configuration-file:output} --pidfile ${directory:run}/slapos-report.pid --logfile ${directory:slapos-log}/slapos-report.log
wrapper-path = ${directory:bin}/slapos-node-report
parameters-extra = true
[slapos-node-status-wrapper]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/slapos node status --cfg ${slapos-configuration-file:output}
wrapper-path = ${directory:bin}/slapos-node-status
parameters-extra = true
[slapos-node-format-wrapper]
recipe = slapos.cookbook:wrapper
command-line = {{ bin_directory }}/slapos node format --cfg ${slapos-configuration-file:output} --logfile=${directory:slapos-log}/slapos-node-format.log --now
wrapper-path = ${directory:bin}/slapos-node-format
parameters-extra = true
[slapos-node-format-wrapper-script]
# Create a wrapper of the wrapper in etc/run
recipe = collective.recipe.template
input = inline:#!{{ bash_executable }}
false
while [ ! $? -eq 0 ]; do
${slapos-node-format-wrapper:wrapper-path}
done
output = ${directory:script}/slapos-node-format
mode = 700
#########
# Deploy some http server to see logs online
#########
# XXX could it be something lighter?
[httpd-configuration-file]
recipe = slapos.recipe.template
url = {{ httpd_configuration_file_template_path }}
output = ${directory:etc}/httpd.conf
# md5sum =
listening-ip = ${instance-parameter:ipv6-random}
listening-port = 8080
htdocs = ${directory:log}
pid-file = ${directory:run}/httpd.pid
access-log = ${directory:httpd-log}/access-log
error-log = ${directory:httpd-log}/error-log
document-root = ${directory:log}
# XXX logrotate for httpd
[httpd-wrapper]
recipe = slapos.cookbook:wrapper
apache-executable = {{ httpd_executable }}
command-line = ${:apache-executable} -f ${httpd-configuration-file:output} -DFOREGROUND
wrapper-path = ${directory:service}/httpd
# generated parameter containing url to use for other sections
url = http://[${httpd-configuration-file:listening-ip}]/
#########
# Deploy logrotate
#########
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = {{ logrotate_executable }}
gzip-binary = {{ gzip_executable }}
gunzip-binary = {{ gunzip_executable }}
# Directories
wrapper = ${directory:bin}/logrotate
conf = ${directory:etc}/logrotate.conf
logrotate-entries = ${directory:logrotate-entries}
backup = ${directory:logrotate-backup}
state-file = ${directory:srv}/logrotate.status
[logrotate-entry-httpd]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = httpd
log = ${httpd-configuration-file:access-log} ${httpd-configuration-file:error-log}
frequency = daily
rotate-num = 30
post = {{ bin_directory }}/killpidfromfile $${apache-configuration:pid-file} SIGUSR1
sharedscripts = true
notifempty = true
create = true
[logrotate-entry-slapos]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = slapos
log = ${directory:slapos-log}/*.log
frequency = daily
rotate-num = 30
#post = {{ bin_directory }}/killpidfromfile ${nginx-configuration:pid-file} SIGUSR1
sharedscripts = true
notifempty = true
create = true
###########
# Deploy cron and configure it
###########
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
wrapper = ${directory:bin}/cron_simplelogger
log = ${directory:log}/crond.log
[cron]
recipe = slapos.cookbook:cron
dcrond-binary = {{ dcron_executable }}
cron-entries = ${directory:cron-entries}
crontabs = ${directory:crontabs}
cronstamps = ${directory:cronstamps}
catcher = ${cron-simplelogger:wrapper}
binary = ${directory:service}/crond
[cron-entry-slapos]
recipe = collective.recipe.template
# Add current PATH to environment, otherwise, gcc is not able to find its own cc1.
# We don't add it in the top of the script, because dcron disallow it.
# XXX: maybe it works if we take PATH from instance, not software.
input = inline:
* * * * * PATH={{ path }} ${slapos-node-instance-wrapper:wrapper-path} > /dev/null 2>&1
* * * * * PATH={{ path }} ${slapos-node-software-wrapper:wrapper-path} > /dev/null 2>&1
* * * * * PATH={{ path }} ${slapos-node-report-wrapper:wrapper-path} > /dev/null 2>&1
output = ${directory:cron-entries}/slapos
[cron-entry-logrotate]
<= cron
recipe = slapos.cookbook:cron.d
name = logrotate
frequency = 0 0 * * *
command = $${logrotate:wrapper}
# XXX what to do for slapformat?
#########
# Deploy dropbear (minimalist SSH server)
#########
[sshkeys-directory]
recipe = slapos.cookbook:mkdirectory
requests = ${directory:sshkeys}/requests/
keys = ${directory:sshkeys}/keys/
[sshkeys-authority]
recipe = slapos.cookbook:sshkeys_authority
request-directory = ${sshkeys-directory:requests}
keys-directory = ${sshkeys-directory:keys}
wrapper = ${directory:service}/sshkeys_authority
keygen-binary = {{ dropbearkey_executable }}
[dropbear-server]
recipe = slapos.cookbook:dropbear
host = ${instance-parameter:ipv6-random}
port = 2222
home = ${directory:ssh}
wrapper = ${directory:bin}/raw_sshd
shell = {{ bash_executable }}
rsa-keyfile = ${directory:ssh}/server_key.rsa
dropbear-binary = {{ dropbear_executable }}
[sshkeys-dropbear]
<= sshkeys-authority
recipe = slapos.cookbook:sshkeys_authority.request
name = dropbear
type = rsa
executable = ${dropbear-server:wrapper}
public-key = ${dropbear-server:rsa-keyfile}.pub
private-key = ${dropbear-server:rsa-keyfile}
wrapper = ${directory:service}/sshd
[dropbear-server-add-authorized-key]
<= dropbear-server
recipe = slapos.cookbook:dropbear.add_authorized_key
key = ${instance-parameter:configuration.authorized-key}
#########
# Send informations to SlapOS Master
#########
[publish-connection-informations]
recipe = slapos.cookbook:publish
log-viewer-url = http://[${httpd-configuration-file:listening-ip}]:${httpd-configuration-file:listening-port}
ssh_command = ssh ${dropbear-server:host} -p ${dropbear-server:port}
#########
# Deploy promises scripts
#########
[dropbear-promise]
recipe = slapos.cookbook:check_port_listening
path = ${directory:promises}/dropbear
hostname = ${dropbear-server:host}
port = ${dropbear-server:port}
[httpd-promise]
recipe = slapos.cookbook:check_port_listening
path = ${directory:promises}/httpd
hostname = ${httpd-configuration-file:listening-ip}
port = ${httpd-configuration-file:listening-port}
[slapos-promise]
recipe = collective.recipe.template
input = inline:#!/{{ bash_executable }}
{{ bin_directory }}/slapgrid-supervisorctl ${slapos-configuration-file:output} status watchdog | grep RUNNING
output = ${directory:promises}/slapos
mode = 0700
software/slapos-in-partition/software.cfg 0 → 100644
View file @ d007050d
# Production profile of slapos-in-partition
# Exactly the same as common.cfg, but:
# 1/ Use a defined set of Python eggs instead of using the latest available
# ones from Pypi, to ensure stability;
# 2/ Define list of trusted certificates for the cache.
[buildout]
extends = common.cfg
[networkcache]
# signature certificates of the following uploaders.
# Romain Courteaud
# Sebastien Robin
# Kazuhiko Shiozaki
# Cedric de Saint Martin
# Yingjie Xu
# Gabriel Monnerat
# Łukasz Nowak
# Test Agent (Automatic update from tests)
# Aurélien Calonne
signature-certificate-list =
-----BEGIN CERTIFICATE-----
MIIB4DCCAUkCADANBgkqhkiG9w0BAQsFADA5MQswCQYDVQQGEwJGUjEZMBcGA1UE
CBMQRGVmYXVsdCBQcm92aW5jZTEPMA0GA1UEChMGTmV4ZWRpMB4XDTExMDkxNTA5
MDAwMloXDTEyMDkxNTA5MDAwMlowOTELMAkGA1UEBhMCRlIxGTAXBgNVBAgTEERl
ZmF1bHQgUHJvdmluY2UxDzANBgNVBAoTBk5leGVkaTCBnzANBgkqhkiG9w0BAQEF
AAOBjQAwgYkCgYEApYZv6OstoqNzxG1KI6iE5U4Ts2Xx9lgLeUGAMyfJLyMmRLhw
boKOyJ9Xke4dncoBAyNPokUR6iWOcnPHtMvNOsBFZ2f7VA28em3+E1JRYdeNUEtX
Z0s3HjcouaNAnPfjFTXHYj4um1wOw2cURSPuU5dpzKBbV+/QCb5DLheynisCAwEA
ATANBgkqhkiG9w0BAQsFAAOBgQBCZLbTVdrw3RZlVVMFezSHrhBYKAukTwZrNmJX
mHqi2tN8tNo6FX+wmxUUAf3e8R2Ymbdbn2bfbPpcKQ2fG7PuKGvhwMG3BlF9paEC
q7jdfWO18Zp/BG7tagz0jmmC4y/8akzHsVlruo2+2du2freE8dK746uoMlXlP93g
QUUGLQ==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB8jCCAVugAwIBAgIJAPu2zchZ2BxoMA0GCSqGSIb3DQEBBQUAMBIxEDAOBgNV
BAMMB3RzeGRldjMwHhcNMTExMDE0MTIxNjIzWhcNMTIxMDEzMTIxNjIzWjASMRAw
DgYDVQQDDAd0c3hkZXYzMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCrPbh+
YGmo6mWmhVb1vTqX0BbeU0jCTB8TK3i6ep3tzSw2rkUGSx3niXn9LNTFNcIn3MZN
XHqbb4AS2Zxyk/2tr3939qqOrS4YRCtXBwTCuFY6r+a7pZsjiTNddPsEhuj4lEnR
L8Ax5mmzoi9nE+hiPSwqjRwWRU1+182rzXmN4QIDAQABo1AwTjAdBgNVHQ4EFgQU
/4XXREzqBbBNJvX5gU8tLWxZaeQwHwYDVR0jBBgwFoAU/4XXREzqBbBNJvX5gU8t
LWxZaeQwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOBgQA07q/rKoE7fAda
FED57/SR00OvY9wLlFEF2QJ5OLu+O33YUXDDbGpfUSF9R8l0g9dix1JbWK9nQ6Yd
R/KCo6D0sw0ZgeQv1aUXbl/xJ9k4jlTxmWbPeiiPZEqU1W9wN5lkGuLxV4CEGTKU
hJA/yXa1wbwIPGvX3tVKdOEWPRXZLg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB7jCCAVegAwIBAgIJAJWA0jQ4o9DGMA0GCSqGSIb3DQEBBQUAMA8xDTALBgNV
BAMMBHg2MXMwIBcNMTExMTI0MTAyNDQzWhgPMjExMTEwMzExMDI0NDNaMA8xDTAL
BgNVBAMMBHg2MXMwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANdJNiFsRlkH
vq2kHP2zdxEyzPAWZH3CQ3Myb3F8hERXTIFSUqntPXDKXDb7Y/laqjMXdj+vptKk
3Q36J+8VnJbSwjGwmEG6tym9qMSGIPPNw1JXY1R29eF3o4aj21o7DHAkhuNc5Tso
67fUSKgvyVnyH4G6ShQUAtghPaAwS0KvAgMBAAGjUDBOMB0GA1UdDgQWBBSjxFUE
RfnTvABRLAa34Ytkhz5vPzAfBgNVHSMEGDAWgBSjxFUERfnTvABRLAa34Ytkhz5v
PzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4GBAFLDS7zNhlrQYSQO5KIj
z2RJe3fj4rLPklo3TmP5KLvendG+LErE2cbKPqnhQ2oVoj6u9tWVwo/g03PMrrnL
KrDm39slYD/1KoE5kB4l/p6KVOdeJ4I6xcgu9rnkqqHzDwI4v7e8/D3WZbpiFUsY
vaZhjNYKWQf79l6zXfOvphzJ
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAO4V/jiMoICoMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMjMyMCAXDTEyMDIxNjExMTAyM1oYDzIxMTIwMTIzMTExMDIzWjAT
MREwDwYDVQQDDAhDT01QLTIzMjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
wi/3Z8W9pUiegUXIk/AiFDQ0UJ4JFAwjqr+HSRUirlUsHHT+8DzH/hfcTDX1I5BB
D1ADk+ydXjMm3OZrQcXjn29OUfM5C+g+oqeMnYQImN0DDQIOcUyr7AJc4xhvuXQ1
P2pJ5NOd3tbd0kexETa1LVhR6EgBC25LyRBRae76qosCAwEAAaNQME4wHQYDVR0O
BBYEFMDmW9aFy1sKTfCpcRkYnP6zUd1cMB8GA1UdIwQYMBaAFMDmW9aFy1sKTfCp
cRkYnP6zUd1cMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAskbFizHr
b6d3iIyN+wffxz/V9epbKIZVEGJd/6LrTdLiUfJPec7FaxVCWNyKBlCpINBM7cEV
Gn9t8mdVQflNqOlAMkOlUv1ZugCt9rXYQOV7rrEYJBWirn43BOMn9Flp2nibblby
If1a2ZoqHRxoNo2yTmm7TSYRORWVS+vvfjY=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAIlBksrZVkK8MA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMzU3MCAXDTEyMDEyNjEwNTUyOFoYDzIxMTIwMTAyMTA1NTI4WjAT
MREwDwYDVQQDDAhDT01QLTM1NzCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
ts+iGUwi44vtIfwXR8DCnLtHV4ydl0YTK2joJflj0/Ws7mz5BYkxIU4fea/6+VF3
i11nwBgYgxQyjNztgc9u9O71k1W5tU95yO7U7bFdYd5uxYA9/22fjObaTQoC4Nc9
mTu6r/VHyJ1yRsunBZXvnk/XaKp7gGE9vNEyJvPn2bkCAwEAAaNQME4wHQYDVR0O
BBYEFKuGIYu8+6aEkTVg62BRYaD11PILMB8GA1UdIwQYMBaAFKuGIYu8+6aEkTVg
62BRYaD11PILMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAMoTRpBxK
YLEZJbofF7gSrRIcrlUJYXfTfw1QUBOKkGFFDsiJpEg4y5pUk1s5Jq9K3SDzNq/W
it1oYjOhuGg3al8OOeKFrU6nvNTF1BAvJCl0tr3POai5yXyN5jlK/zPfypmQYxE+
TaqQSGBJPVXYt6lrq/PRD9ciZgKLOwEqK8w=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAPHoWu90gbsgMA0GCSqGSIb3DQEBBQUAMBQxEjAQBgNV
BAMMCXZpZmlibm9kZTAeFw0xMjAzMTkyMzIwNTVaFw0xMzAzMTkyMzIwNTVaMBQx
EjAQBgNVBAMMCXZpZmlibm9kZTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
ozBijpO8PS5RTeKTzA90vi9ezvv4vVjNaguqT4UwP9+O1+i6yq1Y2W5zZxw/Klbn
oudyNzie3/wqs9VfPmcyU9ajFzBv/Tobm3obmOqBN0GSYs5fyGw+O9G3//6ZEhf0
NinwdKmrRX+d0P5bHewadZWIvlmOupcnVJmkks852BECAwEAAaNQME4wHQYDVR0O
BBYEFF9EtgfZZs8L2ZxBJxSiY6eTsTEwMB8GA1UdIwQYMBaAFF9EtgfZZs8L2ZxB
JxSiY6eTsTEwMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAc43YTfc6
baSemaMAc/jz8LNLhRE5dLfLOcRSoHda8y0lOrfe4lHT6yP5l8uyWAzLW+g6s3DA
Yme/bhX0g51BmI6gjKJo5DoPtiXk/Y9lxwD3p7PWi+RhN+AZQ5rpo8UfwnnN059n
yDuimQfvJjBFMVrdn9iP6SfMjxKaGk6gVmI=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAMNZBmoIOXPBMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtMTMyMCAXDTEyMDUwMjEyMDQyNloYDzIxMTIwNDA4MTIwNDI2WjAT
MREwDwYDVQQDDAhDT01QLTEzMjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
6peZQt1sAmMAmSG9BVxxcXm8x15kE9iAplmANYNQ7z2YO57c10jDtlYlwVfi/rct
xNUOKQtc8UQtV/fJWP0QT0GITdRz5X/TkWiojiFgkopza9/b1hXs5rltYByUGLhg
7JZ9dZGBihzPfn6U8ESAKiJzQP8Hyz/o81FPfuHCftsCAwEAAaNQME4wHQYDVR0O
BBYEFNuxsc77Z6/JSKPoyloHNm9zF9yqMB8GA1UdIwQYMBaAFNuxsc77Z6/JSKPo
yloHNm9zF9yqMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAl4hBaJy1
cgiNV2+Z5oNTrHgmzWvSY4duECOTBxeuIOnhql3vLlaQmo0p8Z4c13kTZq2s3nhd
Loe5mIHsjRVKvzB6SvIaFUYq/EzmHnqNdpIGkT/Mj7r/iUs61btTcGUCLsUiUeci
Vd0Ozh79JSRpkrdI8R/NRQ2XPHAo+29TT70=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB9jCCAV+gAwIBAgIJAKRvzcy7OH0UMA0GCSqGSIb3DQEBBQUAMBMxETAPBgNV
BAMMCENPTVAtNzcyMCAXDTEyMDgxMDE1NDI1MVoYDzIxMTIwNzE3MTU0MjUxWjAT
MREwDwYDVQQDDAhDT01QLTc3MjCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
o7aipd6MbnuGDeR1UJUjuMLQUariAyQ2l2ZDS6TfOwjHiPw/mhzkielgk73kqN7A
sUREx41eTcYCXzTq3WP3xCLE4LxLg1eIhd4nwNHj8H18xR9aP0AGjo4UFl5BOMa1
mwoyBt3VtfGtUmb8whpeJgHhqrPPxLoON+i6fIbXDaUCAwEAAaNQME4wHQYDVR0O
BBYEFEfjy3OopT2lOksKmKBNHTJE2hFlMB8GA1UdIwQYMBaAFEfjy3OopT2lOksK
mKBNHTJE2hFlMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAaNRx6YN2
M/p3R8/xS6zvH1EqJ3FFD7XeAQ52WuQnKSREzuw0dsw12ClxjcHiQEFioyTiTtjs
5pW18Ry5Ie7iFK4cQMerZwWPxBodEbAteYlRsI6kePV7Gf735Y1RpuN8qZ2sYL6e
x2IMeSwJ82BpdEI5niXxB+iT0HxhmR+XaMI=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB+DCCAWGgAwIBAgIJAKGd0vpks6T/MA0GCSqGSIb3DQEBBQUAMBQxEjAQBgNV
BAMMCUNPTVAtMTU4NDAgFw0xMzA2MjAxMjE5MjBaGA8yMTEzMDUyNzEyMTkyMFow
FDESMBAGA1UEAwwJQ09NUC0xNTg0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
gQDZTH9etPUC+wMZQ3UIiOwyyCfHsJ+7duCFYjuo1uZrhtDt/fp8qb8qK9ob+df3
EEYgA0IgI2j/9jNUEnKbc5+OrfKznzXjrlrH7zU8lKBVNCLzQuqBKRNajZ+UvO8R
nlqK2jZCXP/p3HXDYUTEwIR5W3tVCEn/Vda4upTLcPVE5wIDAQABo1AwTjAdBgNV
HQ4EFgQU7KXaNDheQWoy5uOU01tn1M5vNkEwHwYDVR0jBBgwFoAU7KXaNDheQWoy
5uOU01tn1M5vNkEwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOBgQASmqCU
Znbvu6izdicvjuE3aKnBa7G++Fdp2bdne5VCwVbVLYCQWatB+n4crKqGdnVply/u
+uZ16u1DbO9rYoKgWqjLk1GfiLw5v86pd5+wZd5I9QJ0/Sbz2vZk5S4ciMIGwArc
m711+GzlW5xe6GyH9SZaGOPAdUbI6JTDwLzEgA==
-----END CERTIFICATE-----
[versions]
slapos.libnetworkcache = 0.13.4
Jinja2 = 2.7.1
MarkupSafe = 0.18
Pygments = 1.6
Werkzeug = 0.9.4
buildout-versions = 1.7
cliff = 1.4.4
cmd2 = 0.6.5.1
collective.recipe.environment = 0.2.0
collective.recipe.template = 1.10
inotifyx = 0.2.0-1
itsdangerous = 0.23
lxml = 3.2.3
meld3 = 0.6.10
netaddr = 0.7.10
netifaces = 0.8-1
pytz = 2013d
requests = 1.2.3
slapos.cookbook = 0.83.1
slapos.core = 1.0.0rc5
slapos.recipe.cmmi = 0.2
slapos.recipe.download = 1.0.dev-r4053
slapos.recipe.template = 2.5
supervisor = 3.0
xml-marshaller = 0.9.7
# Required by:
# slapos.core==1.0.0rc5
Flask = 0.10.1
# Required by:
# slapos.core==1.0.0rc5
bpython = 0.12
# Required by:
# slapos.core==1.0.0rc5
ipython = 1.0.0
# Required by:
# slapos.cookbook==0.83.1
lock-file = 2.0
# Required by:
# slapos.core==1.0.0rc5
zope.interface = 4.0.5
software/slapos-in-partition/template/httpd.conf.in 0 → 100644
View file @ d007050d
# Apache static configuration
# Automatically generated
# Basic server configuration
PidFile "${:pid-file}"
Listen [${:listening-ip}]:${:listening-port}
ServerAdmin someone@email
DefaultType text/plain
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
DocumentRoot "${:document-root}"
# Log configuration
ErrorLog "${:error-log}"
LogLevel warn
LogFormat "%h %{REMOTE_USER}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %{REMOTE_USER}i %l %u %t \"%r\" %>s %b" common
CustomLog "${:access-log}" common
# Allow cross site scripting
Header set Access-Control-Allow-Origin "*"
# List of modules
LoadModule unixd_module modules/mod_unixd.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule mime_module modules/mod_mime.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule headers_module modules/mod_headers.so
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule autoindex_module modules/mod_autoindex.so
<Directory />
Options Indexes FollowSymLinks
IndexOptions FancyIndexing
order allow,deny
Allow from All
</Directory>
software/slapos-in-partition/template/slapos.cfg.in 0 → 100644
View file @ d007050d
[slapos]
software_root = ${:software-root}
instance_root = ${:instance-root}
master_url = ${:master-url}
key_file = ${:computer-key-file}
cert_file = ${:computer-certificate-file}
certificate_repository_path = ${:certificate-repository-path}
computer_id = ${:computer-id}
maximal_delay = 0
# Don't check if we are using root
root_check = false
[slapformat]
alter_user = false
alter_network = false
input_definition_file = ${:computer-definition-file}
computer_xml = ${:computer-xml}
partition_amount = ${:partition-amount}
create_tap = false
software/slaprunner/common.cfg
View file @ d007050d
......@@ -7,6 +7,7 @@ extends =
../../component/dropbear/buildout.cfg
../../component/git/buildout.cfg
../../component/lxml-python/buildout.cfg
../../component/nginx/buildout.cfg
../../component/rsync/buildout.cfg
../../stack/flask.cfg
../../stack/shacache-client.cfg
......@@ -14,68 +15,151 @@ extends =
../../stack/slapos.cfg
parts =
slapos.cookbook-repository
rdiff-backup
template
eggs
nginx
simple-proxy
node-frontend-template
http-proxy
npm-modules
instance-runner-import
instance-runner-export
slapos-cookbook
# XXX: we have to manually add this for resilience
rdiff-backup
collective.recipe.template-egg
####################
## Node JS proxy
####################
[simple-proxy]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/simple-proxy.js
location = ${buildout:parts-directory}/${:_buildout_section_name_}
md5sum = 86e2231b3f65587b56d9be63e21a4e05
filename = simple-proxy.js
mode = 0644
[node-frontend-template]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/node-frontend.in
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = node-frontend.in
md5sum = 72904152860dddb30ca936dac5bbf4cd
mode = 0644
[http-proxy]
# https://github.com/nodejitsu/node-http-proxy
recipe = slapos.recipe.build:download-unpacked
#XXX-Cedric : use upstream when merged
url = https://github.com/desaintmartin/node-http-proxy/archive/20120621.zip
md5sum = 621e5fca448cbea137c5d847d780d84d
[npm-modules]
recipe = plone.recipe.command
destination = ${buildout:parts-directory}/${:_buildout_section_name_}
location = ${buildout:parts-directory}/${:_buildout_section_name_}
command =
export HOME=${:location};
rm -fr ${:destination} &&
mkdir -p ${:destination} &&
cd ${:destination} &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install colors@0.6.0-1 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install socket.io@0.8.7 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install socket.io-client@0.8.7 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install optimist@0.3.1 &&
${nodejs:location}/bin/node ${nodejs:location}/bin/npm install pkginfo@0.2.3
# slapos-cookbook
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
output = ${buildout:directory}/template.cfg
md5sum = 5de75f295f9382a587343718bb1be124
md5sum = 92a2f3bcd5ff79e3b61ca4a8bacb73ec
mode = 0644
[template-runner]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner.cfg
output = ${buildout:directory}/template-runner.cfg
md5sum = e1d9aeeb3b02dfb3578eddfddd44d053
mode = 0644
[instance-resilient]
recipe = slapos.recipe.template:jinja2
template = ${:_profile_base_location_}/instance-resilient.cfg.jinja2
rendered = ${buildout:directory}/instance-resilient.cfg
md5sum = f533d354da36e1bb10819fab8a90109a
context = key buildout buildout:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
import-list = file parts template-parts:destination
file replicated template-replicated:destination
md5sum = bcd1ee4dd126d2c6e9461f7753fc83b7
mode = 0644
[instance-runner-import]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner-import.cfg.in
output = ${buildout:directory}/instance-runner-import.cfg
md5sum = b37ec3af1898834041d8032ff755bac3
md5sum = f16cb60bb16632e652bea69cd5cdd9b7
mode = 0644
[instance-runner-export]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-runner-export.cfg.in
output = ${buildout:directory}/instance-runner-export.cfg
md5sum = 4028924d0edb61bdcfbf03bb2bac43b8
md5sum = 9ce3482e64a0c957f7a5f25ad0dc03ae
mode = 0644
[template-resilient]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/instance-resilient.cfg.jinja2
md5sum = 2562a6ac6893cc71a7328d7064aff599
filename = instance-resilient.cfg.jinja2
mode = 0644
[template-resilient-test]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/instance-resilient-test.cfg.jinja2
md5sum = ac772d3a1cce4072acfabd563df449bb
filename = instance-resilient-test.cfg.jinja2
mode = 0644
[template_nginx_conf]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/nginx_conf.in
md5sum = 09b7677dfc6b23c1f58e67fd06a7625e
filename = nginx_conf.in
mode = 0644
[template_launcher]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/launcher.in
md5sum = c7f8b6e9ae84aa94686a9cbaaa3dd693
filename = launcher.in
mode = 0644
location = ${buildout:parts-directory}/${:_buildout_section_name_}
[eggs]
recipe = z3c.recipe.scripts
eggs =
${lxml-python:egg}
cns.recipe.symlink
erp5.util
hexagonit.recipe.download
inotifyx
lock-file
netaddr
slapos.cookbook
slapos.libnetworkcache
slapos.toolbox[flask_auth]
slapos.core
xml_marshaller
pytz
# Add slapos.libnetworkcache to path of slapos.core so that slaprunner can build SRs using cache
[slapos-cookbook]
eggs =
${lxml-python:egg}
slapos.cookbook
cliff
hexagonit.recipe.download
inotifyx
netaddr
netifaces
requests
slapos.core
supervisor
xml_marshaller
pytz
slapos.libnetworkcache
software/slaprunner/development.cfg
View file @ d007050d
......@@ -10,24 +10,24 @@ extends = common.cfg
parts +=
slapos.cookbook-repository
# slapos.toolbox-repository
slapos.toolbox-repository
erp5.util-repository
check-recipe
# slapos.core-repository
# check-recipe
develop =
${:parts-directory}/slapos.toolbox-repository
${:parts-directory}/slapos.cookbook-repository
# ${:parts-directory}/slapos.toolbox-repository
${:parts-directory}/erp5.util-repository
# ${:parts-directory}/slapos.core-repository
#[slapos.toolbox-repository]
#recipe = slapos.recipe.build:gitclone
#repository = http://git.erp5.org/repos/slapos.toolbox.git
#branch = slaprunner-resiliency
#git-executable = ${git:location}/bin/git
[slapos.toolbox-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/slapos.toolbox.git
branch = kvmresiliency
git-executable = ${git:location}/bin/git
[slapos.cookbook-repository]
recipe = slapos.recipe.build:gitclone
......@@ -35,6 +35,13 @@ repository = http://git.erp5.org/repos/slapos.git
branch = slaprunner
git-executable = ${git:location}/bin/git
# Used for resiliency tests only
[erp5.util-repository]
recipe = slapos.recipe.build:gitclone
repository = http://git.erp5.org/repos/erp5.git
branch = scalability-master2
git-executable = ${git:location}/bin/git
#[slapos.core-repository]
#recipe = slapos.recipe.build:gitclone
#repository = http://git.erp5.org/repos/slapos.core.git
......@@ -47,6 +54,7 @@ stop-on-error = true
update-command = ${:command}
command =
grep parts ${buildout:develop-eggs-directory}/slapos.cookbook.egg-link &&
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link
grep parts ${buildout:develop-eggs-directory}/slapos.toolbox.egg-link &&
grep parts ${buildout:develop-eggs-directory}/erp5.util.egg-link
# grep parts ${buildout:develop-eggs-directory}/slapos.core.egg-link &&
software/slaprunner/instance-resilient-test.cfg.jinja2 0 → 100644
View file @ d007050d
[buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
parts =
deploy-resiliency-test
request-resilient-instance
deploy-standalone-resiliency-test
[directory]
recipe = slapos.cookbook:mkdirectory
home = ${buildout:directory}
etc = ${:home}/etc/
var = ${:home}/var/
srv = ${:home}/srv/
bin = ${:home}/bin/
tmp = ${:home}/tmp/
log = ${:var}/log/
services = ${:etc}/service/
scripts = ${:etc}/run/
[deploy-resiliency-test]
recipe = slapos.cookbook:wrapper
testnode-parameters = --test-result-path={{ slapparameter_dict.get('test-result-path') }} --revision={{ slapparameter_dict.get('test-suite-revision') }} --node-title={{ slapparameter_dict.get('scalability-launcher-title') }} --test-suite={{ slapparameter_dict.get('test-suite') }} --test-suite-master-url={{ slapparameter_dict.get('test-suite-master-url') }} --log-path=${directory:log}
test-parameters = server_url=${slap-connection:server-url} key_file=${slap-connection:key-file} cert_file=${slap-connection:cert-file} computer_id=${slap-connection:computer-id} partition_id=${slap-connection:partition-id} software=${slap-connection:software-release-url} namebase=runner slaprunner_rootinstance_name='${request-resilient-instance:name}'
command-line = {{ bin_directory }}/runResiliencyTest ${:testnode-parameters} ${:test-parameters}
wrapper-path = ${directory:scripts}/runResiliencyTestSuite
[deploy-standalone-resiliency-test]
# Used to manually run the resilient test if we don't have a running testnode.
recipe = slapos.cookbook:wrapper
test-suite-title = slaprunner
command-line = {{ bin_directory }}/runStandaloneResiliencyTest --test-suite-title=${:test-suite-title} ${deploy-resiliency-test:test-parameters}
wrapper-path = ${directory:bin}/runStandaloneResiliencyTestSuite
[request-resilient-instance]
<= slap-connection
recipe = slapos.cookbook:request
software-url = ${slap-connection:software-release-url}
software-type = resilient
name = Resilient Instance (Root Instance)
{% set cluster_parameter_dict = slapparameter_dict.get('cluster', {}) -%}
config = resiliency-backup-periodicity frontend-domain cloud9-frontend-domain {{ cluster_parameter_dict.keys() | join(' ') }}
{% for key, value in cluster_parameter_dict.items() -%}
config-{{ key }} = {{ dumps(value) }}
{% endfor -%}
config-resiliency-backup-periodicity = *
# XXX hardcoded
config-frontend-domain = google.com
config-cloud9-frontend-domain = google.com
return = backend_url
[slap-parameter]
software/slaprunner/instance-resilient.cfg.jinja2
View file @ d007050d
......@@ -13,30 +13,19 @@ parts +=
{{ parts.replicate("runner", "3") }}
publish-connection-informations
{{ replicated.replicate("runner", "3", "runner-export", "runner-import") }}
{{ replicated.replicate("runner", "3", "runner-export", "runner-import", slapparameter_dict=slapparameter_dict) }}
# Bubble up the parameters
[request-runner]
return = url ssh-public-key ssh-url notification-id ip backend_url url cloud9_url ssh_command password_recovery_code
config = instance-amount debug domain number authorized-key notify ip-list namebase runner1-computer-guid pbs-runner1-computer-guid runner2-computer-guid pbs-runner2-computer-guid runner3-computer-guid pbs-runner3-computer-guid
# XXX Cedric LN Ugly hack, resilient stack and slaprunner stack sharing too much ssh sections
config-authorized-key = ${request-pbs-runner-1:connection-ssh-key} ${request-pbs-runner-2:connection-ssh-key} ${slap-parameter:authorized-key}
config-instance-amount = ${slap-parameter:instance-amount}
config-debug = ${slap-parameter:debug}
config-runner1-computer-guid = ${slap-parameter:runner1-computer-guid}
config-pbs-runner1-computer-guid = ${slap-parameter:pbs-runner1-computer-guid}
config-runner2-computer-guid = ${slap-parameter:runner2-computer-guid}
config-pbs-runner2-computer-guid = ${slap-parameter:pbs-runner2-computer-guid}
config-runner3-computer-guid = ${slap-parameter:runner3-computer-guid}
config-pbs-runner3-computer-guid = ${slap-parameter:pbs-runner3-computer-guid}
config-domain = ${slap-parameter:domain}
return = url ssh-public-key ssh-url notification-id ip backend_url url cloud9_url ssh_command password_recovery_code cloud9_backend_url
[publish-connection-informations]
recipe = slapos.cookbook:publish
1_info = Set your passord in slaprunner in order to access cloud9
backend_url = ${request-runner:connection-backend_url}
url = ${request-runner:connection-url}
cloud9_url = ${request-runner:connection-cloud9_url}
cloud9_backend_url = ${request-runner:connection-cloud9_backend_url}
ssh_command = ${request-runner:connection-ssh_command}
password_recovery_code = ${request-runner:connection-password_recovery_code}
......
software/slaprunner/instance-runner-export.cfg.in
View file @ d007050d
......@@ -3,9 +3,27 @@ extends = ${template-runner:output}
${pbsready-export:output}
parts +=
urls
slaprunner
cron-entry-backup
nginx_conf
nginx-launcher
cloud9
certificate-authority
ca-nginx
ca-node-frontend
slaprunner
test-runner
sshkeys-dropbear-runner
dropbear-server-add-authorized-key
sshkeys-authority
slaprunner-promise
slaprunner-frontend-promise
cloud9-promise
cloud9-frontend-promise
dropbear-promise
symlinks
node-frontend-promise
nginx-promise
publish-connection-informations
cron-entry-backup
[exporter]
recipe = slapos.cookbook:slaprunner.export
......@@ -18,10 +36,5 @@ rsync-binary = ${rsync:location}/bin/rsync
# Extends publish section with resilient parameters
[urls]
[publish-connection-informations]
<= resilient-publish-connection-parameter
backend_url = $${slaprunner:access-url}
url = $${request-frontend:connection-site_url}
cloud9_url = $${cloud9:access-url}
ssh_command = ssh $${dropbear-runner-server:host} -p $${dropbear-runner-server:port}
password_recovery_code = $${recovery-code:passwd}
software/slaprunner/instance-runner-import.cfg.in
View file @ d007050d
......@@ -2,11 +2,25 @@
extends = ${template-runner:output}
${pbsready-import:output}
parts +=
slaprunner
nginx_conf
nginx-launcher
cloud9
certificate-authority
ca-nginx
ca-node-frontend
slaprunner
test-runner
sshkeys-dropbear-runner
dropbear-server-add-authorized-key
sshkeys-authority
slaprunner-promise
cloud9-promise
dropbear-promise
symlinks
nginx-promise
# have to repeat the next one, as it's not inherited from pbsready-import
import-on-notification
import-on-notification
[importer]
recipe = slapos.cookbook:slaprunner.import
......
software/slaprunner/instance-runner.cfg
View file @ d007050d
[buildout]
parts =
nginx_conf
nginx-launcher
cloud9
certificate-authority
ca-nginx
ca-node-frontend
slaprunner
test-runner
sshkeys-dropbear-runner
......@@ -10,8 +15,12 @@ parts =
slaprunner-promise
slaprunner-frontend-promise
cloud9-promise
cloud9-frontend-promise
dropbear-promise
symlinks
request-cloud9-frontend
node-frontend-promise
nginx-promise
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
......@@ -25,6 +34,7 @@ etc = $${buildout:directory}/etc/
var = $${buildout:directory}/var/
srv = $${buildout:directory}/srv/
bin = $${buildout:directory}/bin/
tmp = $${buildout:directory}/tmp/
sshkeys = $${:srv}/sshkeys
services = $${:etc}/service/
......@@ -35,6 +45,9 @@ run = $${:var}/run/
backup = $${:srv}/backup/
promises = $${:etc}/promise/
test = $${:etc}/test/
nginx-data = $${directory:srv}/nginx
ca-dir = $${:srv}/ssl
[runnerdirectory]
recipe = slapos.cookbook:mkdirectory
......@@ -57,8 +70,8 @@ bytes = 4
# Deploy cloud9 and slaprunner
[cloud9]
recipe = slapos.cookbook:cloud9
ip = $${slap-network-information:global-ipv6}
port = 30000
ip = $${slap-network-information:local-ipv4}
port = 4443
wrapper = $${directory:services}/cloud9
working-directory = $${runnerdirectory:home}
git-binary = ${git:location}/bin/git
......@@ -87,7 +100,7 @@ private_key = $${sshkeys-dropbear-runner:private-key}
ipv4 = $${slap-network-information:local-ipv4}
ipv6 = $${slap-network-information:global-ipv6}
proxy_port = 50000
runner_port = 50000
runner_port = 50005
partition-amount = $${slap-parameter:instance-amount}
cloud9-url = $${cloud9:access-url}
wrapper = $${directory:services}/slaprunner
......@@ -145,31 +158,182 @@ wrapper = $${directory:services}/runner_sshd
recipe = slapos.cookbook:dropbear.add_authorized_key
key = $${slap-parameter:authorized-key}
#---------------------
#--
#-- Set node frontend
[node-frontend]
launcher = $${directory:bin}/node-frontend
ip = $${slap-network-information:global-ipv6}
port = $${cloud9:port}
access-url = https://[$${:ip}]:$${:port}
[node-frontend-launcher]
recipe = slapos.recipe.template:jinja2
template = ${node-frontend-template:location}/${node-frontend-template:filename}
rendered = $${node-frontend:launcher}
mode = 700
context =
key ip node-frontend:ip
key port node-frontend:port
key key ca-node-frontend:key-file
key certificate ca-node-frontend:cert-file
key backend_ip nginx-frontend:local-ip
key backend_port nginx-frontend:port
raw shell_path ${bash:location}/bin/bash
raw node_env ${buildout:parts-directory}:${npm-modules:location}/node_modules
raw node_path ${nodejs:location}/bin/node
raw conf_path ${simple-proxy:location}/${simple-proxy:filename}
#---------------------------
#--
#-- Set nginx frontend
[tempdirectory]
recipe = slapos.cookbook:mkdirectory
client_body_temp_path = $${directory:tmp}/client_body_temp_path
proxy_temp_path = $${directory:tmp}/proxy_temp_path
fastcgi_temp_path = $${directory:tmp}/fastcgi_temp_path
uwsgi_temp_path = $${directory:tmp}/uwsgi_temp_path
scgi_temp_path = $${directory:tmp}/scgi_temp_path
[nginx-frontend]
# Options
nb_workers = 2
# Network
local-ip = $${slap-network-information:local-ipv4}
port = 30001
global-ip = $${slap-network-information:global-ipv6}
global-port = $${slaprunner:runner_port}
# Backend
cloud9-ip = $${cloud9:ip}
cloud9-port = $${cloud9:port}
runner-ip = $${slaprunner:ipv4}
runner-port = $${slaprunner:runner_port}
# SSL
ssl-certificate = $${ca-nginx:cert-file}
ssl-key = $${ca-nginx:key-file}
# Log
path_pid = $${directory:run}/nginx.pid
path_log = $${directory:log}/nginx.log
path_access_log = $${directory:log}/nginx.access.log
path_error_log = $${directory:log}/nginx.error.log
path_tmp = $${buildout:directory}/tmp
# Config files
path_nginx_conf = $${directory:etc}/nginx.conf
# Executables
bin_nginx = ${nginx:location}/sbin/nginx
bin_launcher = $${directory:bin}/launcher
# Utils
path_shell = ${dash:location}/bin/dash
# Misc.
etc_dir = $${directory:etc}
[nginx_conf]
recipe = slapos.recipe.template:jinja2
template = ${template_nginx_conf:location}/${template_nginx_conf:filename}
rendered = $${nginx-frontend:path_nginx_conf}
context =
section param_nginx_frontend nginx-frontend
section param_tempdir tempdirectory
[nginx-launcher]
recipe = slapos.recipe.template:jinja2
template = ${template_launcher:location}/${template_launcher:filename}
rendered = $${nginx-frontend:bin_launcher}
mode = 700
context =
section param_nginx_frontend nginx-frontend
#--------------------
#--
#-- ssl certificates
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = ${openssl:location}/bin/openssl
ca-dir = $${directory:ca-dir}
requests-directory = $${cadirectory:requests}
wrapper = $${directory:services}/certificate_authority
ca-private = $${cadirectory:private}
ca-certs = $${cadirectory:certs}
ca-newcerts = $${cadirectory:newcerts}
ca-crl = $${cadirectory:crl}
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:ca-dir}/requests/
private = $${directory:ca-dir}/private/
certs = $${directory:ca-dir}/certs/
newcerts = $${directory:ca-dir}/newcerts/
crl = $${directory:ca-dir}/crl/
[ca-nginx]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${cadirectory:certs}/nginx_frontend.key
cert-file = $${cadirectory:certs}/nginx_frontend.crt
executable = $${nginx-launcher:rendered}
wrapper = $${directory:services}/nginx-frontend
# Put domain name
name = example.com
[ca-node-frontend]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${cadirectory:certs}/nodejs.key
cert-file = $${cadirectory:certs}/nodejs.crt
executable = $${node-frontend-launcher:rendered}
wrapper = $${directory:services}/node-frontend
# Put domain name
name = example.com
#--------------------
#--
#-- Request frontend
# Request frontend
[request-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
name = Frontend
name = SlapRunner Frontend
# XXX We have hardcoded SR URL here.
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config = url
config-url = $${slaprunner:access-url}
return = site_url
config = url domain
config-url = $${node-frontend:access-url}
config-domain = $${slap-parameter:frontend-domain}
return = site_url domain
[request-cloud9-frontend]
<= slap-connection
recipe = slapos.cookbook:requestoptional
name = Cloud9 Frontend
software-url = http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/apache-frontend/software.cfg
slave = true
config = url domain
config-url = $${node-frontend:access-url}
config-domain = $${slap-parameter:cloud9-frontend-domain}
return = site_url domain
#--------------------------------------
#--
#-- Send informations to SlapOS Master
# Send informations to SlapOS Master
[publish-connection-informations]
recipe = slapos.cookbook:publish
1_info = Set your passord in slaprunner in order to access cloud9
backend_url = $${slaprunner:access-url}
url = $${request-frontend:connection-site_url}
cloud9_url = $${cloud9:access-url}
url = https://$${request-frontend:connection-domain}
cloud9_backend_url = $${node-frontend:access-url}
cloud9_url = https://$${request-cloud9-frontend:connection-domain}
ssh_command = ssh $${dropbear-runner-server:host} -p $${dropbear-runner-server:port}
password_recovery_code = $${recovery-code:passwd}
#---------------------------
#--
#-- Deploy promises scripts
# Deploy promises scripts
[slaprunner-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/slaprunner
......@@ -179,7 +343,7 @@ port = $${slaprunner:runner_port}
[slaprunner-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/slaprunner_frontend
url = $${request-frontend:connection-site_url}
url = https://$${request-frontend:connection-domain}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
......@@ -190,6 +354,26 @@ url = http://$${cloud9:ip}:$${cloud9:port}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
[cloud9-frontend-promise]
recipe = slapos.cookbook:check_url_available
path = $${directory:promises}/cloud9-frontend-promise
url = $${publish-connection-informations:cloud9_url}
check-secure = 1
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
[node-frontend-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/node-frontend
hostname = $${node-frontend:ip}
port = $${node-frontend:port}
[nginx-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/nginx
hostname = $${nginx-frontend:local-ip}
port = $${nginx-frontend:port}
[dropbear-promise]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promises}/dropbear
......@@ -207,3 +391,6 @@ authorized-key =
# Default value of instances number in slaprunner
instance-amount = 10
debug = false
cloud9-frontend-domain =
frontend-domain =
software/slaprunner/instance.cfg
View file @ d007050d
......@@ -4,15 +4,50 @@ parts =
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[switch_softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${template-runner:output}
resilient = ${instance-resilient:rendered}
resilient = $${instance-resilient:rendered}
test = $${instance-resilient-test:rendered}
runner = ${template-runner:output}
runner-import = ${instance-runner-import:output}
runner-export = ${instance-runner-export:output}
frozen = ${instance-frozen:output}
pull-backup = ${template-pull-backup:output}
[instance-resilient]
recipe = slapos.recipe.template:jinja2
template = ${template-resilient:target}
rendered = $${buildout:directory}/instance-resilient.cfg
extensions = jinja2.ext.do
context = key buildout buildout:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
template-parts-destination = ${template-parts:destination}
template-replicated-destination = ${template-replicated:destination}
import-list = file parts :template-parts-destination
file replicated :template-replicated-destination
mode = 0644
[instance-resilient-test]
recipe = slapos.recipe.template:jinja2
template = ${template-resilient-test:location}/instance-resilient-test.cfg.jinja2
rendered = $${buildout:directory}/template-resilient-test.cfg
bin-directory = ${buildout:bin-directory}
context =
key bin_directory instance-resilient-test:bin-directory
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
key slapparameter_dict slap-configuration:configuration
mode = 0644
[slap-configuration]
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
\ No newline at end of file
software/slaprunner/launcher.in 0 → 100644
View file @ d007050d
#! {{ param_nginx_frontend['path_shell'] }}
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
# Run nginx
exec {{ param_nginx_frontend['bin_nginx'] }} -c {{ param_nginx_frontend['path_nginx_conf'] }}
software/slaprunner/nginx_conf.in 0 → 100644
View file @ d007050d
worker_processes {{ param_nginx_frontend['nb_workers'] }};
pid {{ param_nginx_frontend['path_pid'] }};
error_log {{ param_nginx_frontend['path_error_log'] }};
daemon off;
events {
worker_connections 1024;
accept_mutex off;
}
http {
default_type application/octet-stream;
access_log {{ param_nginx_frontend['path_access_log'] }} combined;
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen {{ param_nginx_frontend['local-ip'] }}:{{ param_nginx_frontend['port'] }};
server_name _;
keepalive_timeout 90s;
client_body_temp_path {{ param_tempdir['client_body_temp_path'] }};
proxy_temp_path {{ param_tempdir['proxy_temp_path'] }};
fastcgi_temp_path {{ param_tempdir['fastcgi_temp_path'] }};
uwsgi_temp_path {{ param_tempdir['uwsgi_temp_path'] }};
scgi_temp_path {{ param_tempdir['scgi_temp_path'] }};
location / {
auth_basic "Restricted";
auth_basic_user_file {{ param_nginx_frontend['etc_dir'] }}/.htpasswd;
proxy_pass http://{{ param_nginx_frontend['cloud9-ip'] }}:{{ param_nginx_frontend['cloud9-port'] }};
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen [{{ param_nginx_frontend['global-ip'] }}]:{{ param_nginx_frontend['global-port'] }} ssl;
server_name _;
ssl_certificate {{ param_nginx_frontend['ssl-certificate'] }};
ssl_certificate_key {{ param_nginx_frontend['ssl-key'] }};
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
keepalive_timeout 90s;
client_body_temp_path {{ param_tempdir['client_body_temp_path'] }};
proxy_temp_path {{ param_tempdir['proxy_temp_path'] }};
fastcgi_temp_path {{ param_tempdir['fastcgi_temp_path'] }};
uwsgi_temp_path {{ param_tempdir['uwsgi_temp_path'] }};
scgi_temp_path {{ param_tempdir['scgi_temp_path'] }};
location / {
proxy_pass http://{{ param_nginx_frontend['runner-ip'] }}:{{ param_nginx_frontend['runner-port'] }};
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $http_host;
}
}
}
software/slaprunner/node-frontend.in 0 → 100755
View file @ d007050d
#!{{ shell_path }}
# BEWARE: This file is operated by slapgrid
# BEWARE: It will be overwritten automatically
export NODE_PATH={{ node_env }}
exec {{ node_path }} {{ conf_path }} {{ ip }} {{ port }} {{ key }} {{ certificate }} {{ backend_ip }} {{ backend_port }}
\ No newline at end of file
software/slaprunner/simple-proxy.js 0 → 100644
View file @ d007050d
/*****************************************************************************
*
* Copyright (c) 2013 Vifib SARL and Contributors. All Rights Reserved.
*
* WARNING: This program as such is intended to be used by professional
* programmers who take the whole responsibility of assessing all potential
* consequences resulting from its eventual inadequacies and bugs
* End users who are looking for a ready-to-use solution with commercial
* guarantees and support are strongly adviced to contract a Free Software
* Service Company
*
* This program is Free Software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 3
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*
*****************************************************************************/
var fs = require('fs'),
util = require('util'),
colors = require('colors'),
http = require('http'),
httpProxy = require('http-proxy');
var listenInterface = process.argv[2],
port = process.argv[3],
sslKeyFile = process.argv[4],
sslCertFile = process.argv[5],
backendIp = process.argv[6],
backendPort = process.argv[7];
if (process.argv.length < 8) {
console.error("Too few arguments. Exiting.");
process.exit(1);
}
var middleware = function (req, res, proxy) {
return proxy.proxyRequest(req, res,{
host: backendIp,
port: backendPort
});
};
middleware.proxyWebSocketRequest = function (req, socket, head, proxy) {
return proxy.proxyWebSocketRequest(req, socket, head,{
host: backendIp,
port: backendPort
});
};
/**
* Create server
*/
var proxyServer = httpProxy.createServer(
middleware,
{
https: {
key: fs.readFileSync(
sslKeyFile,
'utf8'
),
cert: fs.readFileSync(
sslCertFile,
'utf8'
)
},
source: {
host: listenInterface,
port: port
}}
);
console.log('HTTPS server starting and trying to listen on ' +
listenInterface + ':' + port);
// Release the beast.
proxyServer.listen(port, listenInterface);
software/slaprunner/software.cfg
View file @ d007050d
......@@ -5,7 +5,7 @@
# 2/ Define list of trusted certificates for the cache.
[buildout]
extends = development.cfg
extends = common.cfg
[networkcache]
# signature certificates of the following uploaders.
......@@ -65,7 +65,16 @@ slapos.recipe.template = 2.4.2
smmap = 0.8.2
xml-marshaller = 0.9.7
z3c.recipe.scripts = 1.0.1
lock-file = 2.0
rdiff-backup = 1.0.5
slapos.recipe.cmmi = 0.2
slapos.recipe.download = 1.0.dev-r4053
slapos.toolbox = 0.35.1
slapos.cookbook = 0.78.5
cliff = 1.4
cmd2 = 0.6.6
prettytable = 0.7.2
requests = 1.2.3
# Required by:
# slapos.core==0.34
# slapos.toolbox==0.34.0
......
software/trac-svn/instance-trac.cfg
View file @ d007050d
......@@ -308,9 +308,8 @@ githttpbackend = ${git:location}/libexec/git-core/git-http-backend
base-directory = $${trac-config:project_dir}/git
[trac-admin]
recipe = slapos.cookbook:pwgen
file = $${buildout:directory}/.password
pwgen-binary = ${pwgen:location}/bin/pwgen
recipe = slapos.cookbook:generate.password
storage-path = $${buildout:directory}/.password
user = TracAdmin
#---------------------
......@@ -330,7 +329,7 @@ eggs-dirs =
python-lib = ${python2.7:location}/lib
trac-admin = ${buildout:bin-directory}/trac-admin
admin-user = $${trac-admin:user}
admin-password = $${trac-admin:password}
admin-password = $${trac-admin:passwd}
#MySQL informations
mysql-username = $${mariadb-urlparse:username}
mysql-password = $${mariadb-urlparse:password}
......@@ -401,7 +400,7 @@ port = 9000
shell = $${shell:wrapper}
wrapper = $${rootdirectory:bin}/shellinaboxd_raw
shellinabox-binary = ${shellinabox:location}/bin/shellinaboxd
password = $${trac-admin:password}
password = $${trac-admin:passwd}
directory = $${inittrac:site-dir}
login-shell = $${rootdirectory:bin}/login
certificate-directory = $${directory:shellinabox}
......@@ -454,7 +453,7 @@ frontend_url = $${request-frontend:connection-site_url}
git = $${request-frontend:connection-site_url}git/
svn = $${request-frontend:connection-site_url}svn/
admin_user = $${trac-admin:user}
admin_password = $${trac-admin:password}
admin_password = $${trac-admin:passwd}
admin_shell = https://[$${shellinabox:ipv6}]:$${shellinabox:port}/
#----------------
......
software/trac-svn/software.cfg
View file @ d007050d
......@@ -41,7 +41,6 @@ extends =
../../component/lxml-python/buildout.cfg
../../component/mysql-python/buildout.cfg
../../component/git/buildout.cfg
../../component/pwgen/buildout.cfg
../../component/shellinabox/buildout.cfg
../../component/perl/buildout.cfg
......
stack/erp5/buildout.cfg
View file @ d007050d
......@@ -189,7 +189,7 @@ mode = 640
[template-kumofs]
< = template-jinja2-base
filename = instance-kumofs.cfg
md5sum = 90a321be12ee977800d590bf941021ef
md5sum = 40817014a41497bceb696e512436e670
extra-context =
key dash_location dash:location
key dcron_location dcron:location
......@@ -200,7 +200,7 @@ extra-context =
[template-tidstorage]
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/instance-tidstorage.cfg.in
md5sum = 5572d10b343bd2de90deeaf55dd0fdc7
md5sum = 2bc13fc6cd52c7b8e2b4ddb99a69974b
mode = 640
[template-cloudooo]
......@@ -414,6 +414,7 @@ eggs =
${pycrypto-python:egg}
lock_file
PyXML
Pympler
SOAPpy
cElementTree
chardet
......@@ -422,13 +423,14 @@ eggs =
erp5diff
inotifyx
ipdb
Jinja2
mechanize
numpy
ordereddict
paramiko
ply
pyflakes
pyPdf # should be replaced by PyPDF2, but it is not installable
pypdf2
python-magic
python-memcached
pytz
......@@ -446,6 +448,10 @@ eggs =
huBarcode
qrcode
spyne
httplib2
suds
# Needed for checking ZODB Components source code
pylint
# Zope
ZODB3
......@@ -546,8 +552,8 @@ scripts =
[versions]
# pin Acquisition and Products.DCWorkflow to Nexedi flavour of eggs
Acquisition = 2.13.7nxd001
Products.DCWorkflow = 2.2.3nxd002
Acquisition = 2.13.8nxd001
Products.DCWorkflow = 2.2.4nxd001
# specify dev version to be sure that an old released version is not used
cloudooo = 1.2.5-dev
......@@ -609,9 +615,7 @@ GitPython = 0.3.2.RC1
Jinja2 = 2.6
MySQL-python = 1.2.4
PIL = 1.1.7
Paste = 1.7.5.1
PasteDeploy = 1.5.0
PasteScript = 1.7.5
Products.CMFActionIcons = 2.1.3
Products.DCWorkflowGraph = 0.4.1
Products.ExternalEditor = 1.1.0
......@@ -651,7 +655,7 @@ inotifyx = 0.2.0
ipdb = 0.7
ipython = 0.13.2
meld3 = 0.6.10
mr.developer = 1.25
mr.developer = 1.26
netaddr = 0.7.10
netifaces = 0.8_1
ordereddict = 1.1
......@@ -686,3 +690,4 @@ xupdate-processor = 0.4
zope.app.debug = 3.4.1
zope.app.dependable = 3.5.1
zope.app.form = 4.0.2
pylint = 1.0.0
stack/erp5/instance-kumofs.cfg.in
View file @ d007050d
......@@ -34,8 +34,8 @@ gateway-wrapper = ${basedirectory:services}/kumofs_gateway
manager-wrapper = ${basedirectory:services}/kumofs_manager
server-wrapper = ${basedirectory:services}/kumofs_server
# Paths: Data
data-directory = ${directory:kumofs-data}
# Paths: Data (with 10M buckets and HDBTLARGE option)
data-path = ${directory:kumofs-data}/kumodb.tch#bnum=10485760#opts=l
# Paths: Logs
kumo-gateway-log = ${basedirectory:log}/kumo-gateway.log
......
stack/erp5/instance-tidstorage.cfg.in
View file @ d007050d
......@@ -331,7 +331,7 @@ runzope-binary = {{ bin_directory }}/runzope
bt5-repository-list =
[deadlock-debugger-password]
recipe = slapos.cookbook:pwgen.stable
recipe = slapos.cookbook:generate.password
[zope-conf-parameter-base]
ip = {{ ipv4 }}
......@@ -346,7 +346,7 @@ context =
key instance directory:instance
key instance_products directory:instance-products
raw deadlock_path /manage_debug_threads
key deadlock_debugger_password deadlock-debugger-password:password
key deadlock_debugger_password deadlock-debugger-password:passwd
key tidstorage_ip tidstorage:ip
key tidstorage_port tidstorage:port
key promise_path erp5-promise:promise-path
......
stack/resilient/README.txt
View file @ d007050d
......@@ -214,12 +214,12 @@ parts +=
{{ replicated.replicate("Name", "3",
"mysoftware-export", "mysoftware-import",
"ArgLeader","ArgBackup") }}
"ArgLeader","ArgBackup", slapparameter_dict=slapparameter_dict) }}
and it'll expend into the sections require to request Name0, Name1 and Name2,
backuped and resilient. The leader will expend the section [ArgLeader], backups
will expend [ArgBackup]. If you don't need to specify any options, you can
omit the last two arguments in replicate().
will expend [ArgBackup]. slapparameter_dict is the dict containing the parameters given to the instance. If you don't need to specify any options, you can
omit the last three arguments in replicate().
Since you will compile your template with jinja2, there should be no $${},
because it is not yet possible to use jinja2 -> buildout template.
......@@ -227,3 +227,36 @@ because it is not yet possible to use jinja2 -> buildout template.
To compile with jinja2, see jinja2's recipe.
Deploying your resilient software
---------------------------------
You can provide sla parameters to each request you make (a lot: for export, import and pbs).
example:
Here is a small example of parameters you can provide to control the deployment (case of a runner):
<?xml version='1.0' encoding='utf-8'?>
<instance>
<parameter id="-sla-1-computer_guid">COMP-GRP1</parameter>
<parameter id="-sla-pbs1-computer_guid">COMP-PBS1</parameter>
<parameter id="-sla-2-computer_guid">COMP-GROUP2</parameter>
<parameter id="-sla-runner2-computer_guid">COMP-RUN2</parameter>
<parameter id="-sla-2-network_guid">NET-2</parameter>
<parameter id="-sla-runner0-computer_guid">COMP-RUN0</parameter>
</instance>
Consequence on sla parameters by request:
* runner0: computer_guid = COMP-RUN0 (provided directly)
* runner1: computer_guid = COMP-GRP1 (provided by group 1)
* runner2: computer_guid = COMP-RUN2 (provided by group 2 but overided directly)
network_guid = NET-2 (provided by group 2)
* PBS 1: computer_guid = COMP-PBS1 (provided by group 1 but overided directly)
* PBS 2: computer_guid = COMP-GRP2 (provided by group 2)
network_guid = NET-2 (provided by group 2)
Parameters are analysed this way:
* If it starts with "-sla-" it is not transmitted to requested instance and is used to do the request as sla.
* -sla-foo-bar=example (foo being a magic key) will be use for each request considering "foo" as a key to use and the sla parameter is "bar". So for each group using the "foo" key, sla parameter "bar" is used with value "example"
About magic keys:
We can find 2 kinds of magic keys:
* id : example, in "-sla-2-foo" 2 is the magic key and the parameter will be used for each request with id 2 (in case of kvm: kvm2 and PBS 2)
* nameid : example, in "-sla-kvm2-foo", foo will be used for kvm2 request. Name for pbs is "pbs" -> "-sla-pbs2-foo".
IMPORTANT NOTE: in case the same foo parameter is asked for the group, the nameid key prevail
stack/resilient/TODO.txt 0 → 100644
View file @ d007050d
* Report, from pbs and from clone, when a backup failed
* Make sure, when a takeover is done, that "importer" script finishes to run while importer instance is changed into exporter.
* Test that, after a successful backup/takeover, another backup is possible and will be successful.
* PBSs and mirrors should monitor/replace themselves
* Report errors from backup
* If a PBS master is down and then back again, it might want to participate in the ongoing election, then.. what happens?
* If the network is partitioned (the two backups don't see each other, but each can see the slapos master) there will be two concurrent elections taking place, with two winners and two renames.
* How to ensure "synchronization" between two main instances? example: Wordpress: mysql is down, then replaced, then inconsistency between apache and the new mysql
* How to deal with big data? I.e how to have working backup/restore system of 1TB data with slow connection?
* How to be sure that elected importer contains a/ the latest data and b/ has finished to pull. We should prevent importer not having a/ and b/ to become the main.
* Should we crypt backed up data?
* If a PBS is lost, a new PBS should be created from another one, in order ot keep history
* If an election takes place and asks for a rename, but no slapgrid is running (or takes too much) then another election will take place
and ask to rename the previously selected winner, thus breaking the resiliency system.
stack/resilient/buildout.cfg
View file @ d007050d
[buildout]
extends =
../../component/apache/buildout.cfg
../../component/bash/buildout.cfg
../../component/dropbear/buildout.cfg
../../component/gzip/buildout.cfg
../../component/rdiff-backup/buildout.cfg
../../component/rsync/buildout.cfg
parts =
rdiff-backup
collective.recipe.template-egg
pbsready
pbsready-import
pbsready-export
template-replicated
template-parts
instance-frozen
template-resilient
# needed tools for resiliency
gzip
rdiff-backup
dash
[collective.recipe.template-egg]
recipe = zc.recipe.egg
eggs = collective.recipe.template
#----------------
#--
......@@ -29,7 +37,7 @@ parts =
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/pbsready.cfg.in
output = ${buildout:directory}/pbsready.cfg
md5sum = 9f4212a79f10bee8f6d75061943110e2
md5sum = 02a5f1741d6b732519c06b522dbe0d66
mode = 0644
[pbsready-import]
......@@ -38,7 +46,7 @@ mode = 0644
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/pbsready-import.cfg.in
output = ${buildout:directory}/pbsready-import.cfg
md5sum = 1b1308fd39476d48b5ca13db48ea6dc9
md5sum = 0f953067aac3e0132f72fc7e1ed38bd4
mode = 0644
[pbsready-export]
......@@ -47,27 +55,27 @@ mode = 0644
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/pbsready-export.cfg.in
output = ${buildout:directory}/pbsready-export.cfg
md5sum = 5d9e20c436fd307e8e4ab224a9a65792
md5sum = 135638b8c513c7723efb51e3d9182ae9
mode = 0644
[template-pull-backup]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-pull-backup.cfg.in
output = ${buildout:directory}/instance-pull-backup.cfg
md5sum = 453d96f5a6c1230c01c878cc7640bae6
md5sum = c49e5911b94078d87b94507fb4efc93b
mode = 0644
[template-replicated]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/template-replicated.cfg.in
md5sum = 9e20f283bf709c63c9c6692d5e1f8972
md5sum = 771e1ab7e7e77b35116c57bbae56ba62
mode = 0644
destination = ${buildout:directory}/template-replicated.cfg.in
[template-parts]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/template-parts.cfg.in
md5sum = c942f82552fcb42fc74a5f896e0cd5f3
md5sum = a3f55a20881c3f1ec4416662146c06f7
mode = 0644
destination = ${buildout:directory}/template-parts.cfg.in
......@@ -77,12 +85,18 @@ destination = ${buildout:directory}/template-parts.cfg.in
# which will run without removing any content because it raises an error.
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-frozen.cfg.in
md5sum = d21472f0e58f928fb827f2cbf22c4d4a
output = ${buildout:directory}/instance-frozen.cfg
[template-resilient]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/resilient.cfg.in
output = ${buildout:directory}/resilient.cfg
md5sum = 59e74d290d623de2c1e147e48f284fba
[resilient-web-takeover-cgi-script-download]
recipe = slapos.recipe.download
url = ${:_profile_base_location_}/resilient-web-takeover-cgi-script.py.in
md5sum = e6262c5cf9b1c4d1ea4d959fdcbe3070
mode = 0644
destination = ${buildout:directory}/resilient-web-takeover-cgi-script.py.in
[versions]
# Pin Jinja2 to 2.6, as 2.7 breaks current code
Jinja2 = 2.6
# ... And newer s.r.template requires Jinja2 >= 2.7
slapos.recipe.template = 2.4.2
stack/resilient/instance-frozen.cfg.in
View file @ d007050d
[buildout]
parts =
\ No newline at end of file
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
parts =
stack/resilient/instance-pull-backup.cfg.in
View file @ d007050d
......@@ -75,7 +75,7 @@ callbacks = $${directory:notifier-callbacks}
id-file = $${rootdirectory:etc}/notifier.id
equeue-socket = $${equeue:socket}
host = $${slap-network-information:global-ipv6}
port = 8080
port = 8088
wrapper = $${basedirectory:services}/notifier
server-binary = ${buildout:bin-directory}/pubsubserver
notifier-binary = ${buildout:bin-directory}/pubsubnotifier
......@@ -105,7 +105,9 @@ promises-directory = $${basedirectory:promises}
directory = $${directory:pbs-backup}
cron-entries = $${cron:cron-entries}
wrappers-directory = $${directory:pbs-wrappers}
notifier-url = http://[$${notifier:host}]:$${notifier:port}/
run-directory = $${basedirectory:run}
# XXX: this should be named "notifier-host"
notifier-url = http://[$${notifier:host}]:$${notifier:port}
slave-instance-list = $${slap-parameter:slave_instance_list}
......
stack/resilient/parameter-schema.json 0 → 100644
View file @ d007050d
{
"$schema": "http://json-schema.org/draft-04/schema",
"title": "Resiliency Parameters",
"description": "List of possible parameters used in the resilient stack",
"type": "object",
"properties": {
"-sla-0-computer_guid": {
"title": "Target computer for main instance",
"description": "Target computer GUID for main instance.",
"type": "string"
},
"-sla-1-computer_guid": {
"title": "Target computer for first clone",
"description": "Target computer for first clone and PBS.",
"type": "string"
},
"-sla-2-computer_guid": {
"title": "Target computer for second clone",
"description": "Target computer for second clone and PBS.",
"type": "string"
},
"resiliency-backup-periodicity": {
"title": "Periodicity of backup",
"description": "Periodicity of backup, in cron format. Default is every hour.",
"type": "string"
},
"remove-backup-older-than": {
"title": "Remove backups older than...",
"description": "Remove all the backups in PBS that are older than specified value. It should be rdiff-backup-compatible.",
"type": "string",
"default": "2W"
}
}
}
stack/resilient/pbsready-export.cfg.in
View file @ d007050d
......@@ -2,9 +2,36 @@
extends = ${pbsready:output}
parts +=
# Explicitely define extended parts from pbsready
# then add local parts
parts =
resiliency
logrotate
logrotate-entry-cron
logrotate-entry-equeue
cron
cron-entry-logrotate
sshkeys-authority
dropbear-server
sshkeys-dropbear
resilient-sshkeys-dropbear-promise
dropbear-server-pbs-authorized-key
notifier
cron-entry-backup
[resilient-directory]
recipe = slapos.cookbook:mkdirectory
home = $${buildout:directory}
var = $${:home}/var
pid = $${:var}/pid
# Define port of ssh server. It has to be different from import so that it
# supports export/import using same IP (slaprunner, slapos-in-partition,
# ipv4...)
[dropbear-server]
port = 22221
[resilient-publish-connection-parameter]
notification-id = http://[$${notifier:host}]:$${notifier:port}/get/$${notifier-exporter:name}
......@@ -18,6 +45,7 @@ title = Dumping $${slap-parameter:namebase}
executable = $${exporter:wrapper}
wrapper = $${rootdirectory:bin}/exporter
notify = $${slap-parameter:notify}
pidfile = $${resilient-directory:pid}/$${:name}.pid
[cron-entry-backup]
# Schedule the periodic database dump.
......@@ -25,5 +53,10 @@ notify = $${slap-parameter:notify}
<= cron
recipe = slapos.cookbook:cron.d
name = backup
frequency = 0 * * * *
frequency = $${slap-parameter:resiliency-backup-periodicity}
command = $${notifier-exporter:wrapper}
[slap-parameter]
# In cron.d format (i.e things like */15 * * * * are accepted).
resiliency-backup-periodicity = 0 0 * * *
stack/resilient/pbsready-import.cfg.in
View file @ d007050d
......@@ -2,12 +2,39 @@
extends = ${pbsready:output}
parts +=
# Explicitely define extended parts from pbsready
# then add local parts
parts =
resiliency
logrotate
logrotate-entry-cron
logrotate-entry-equeue
cron
cron-entry-logrotate
sshkeys-authority
dropbear-server
sshkeys-dropbear
resilient-sshkeys-dropbear-promise
dropbear-server-pbs-authorized-key
notifier
resilient-web-takeover-cgi-script
resilient-web-takeover-httpd-wrapper
resilient-web-takeover-httpd-promise
import-on-notification
resilient-publish-connection-parameter
[resilient-publish-connection-parameter]
notification-url = http://[$${notifier:host}]:$${notifier:port}/notify
takeover-url = http://[$${resilient-web-takeover-httpd-configuration-file:listening-ip}]:$${resilient-web-takeover-httpd-configuration-file:listening-port}/
takeover-password = $${resilient-web-takeover-password:passwd}
# Define port of ssh server. It has to be different from import so that it
# supports export/import using same IP (slaprunner, slapos-in-partition,
# ipv4...)
[dropbear-server]
port = 22220
[import-on-notification]
# notifier.callback runs a script when a notification (sent by a parent PBS)
......@@ -16,3 +43,67 @@ notification-url = http://[$${notifier:host}]:$${notifier:port}/notify
recipe = slapos.cookbook:notifier.callback
on-notification-id = $${slap-parameter:on-notification}
callback = $${importer:wrapper}
###########
# Deploy a webserver allowing to do takeover from a web browser.
###########
[resilient-web-takeover-password]
recipe = slapos.cookbook:generate.password
storage-path = $${directory:srv}/passwd
bytes = 8
[resilient-web-takeover-cgi-script]
recipe = collective.recipe.template
input = ${resilient-web-takeover-cgi-script-download:destination}
output = $${directory:cgi-bin}/web-takeover.cgi
password = $${resilient-web-takeover-password:passwd}
mode = 700
# XXX could it be something lighter?
# XXX Add SSL
[resilient-web-takeover-httpd-configuration-file]
recipe = collective.recipe.template
input = inline:
PidFile "$${:pid-file}"
Listen [$${:listening-ip}]:$${:listening-port}
ServerAdmin someone@email
DocumentRoot "$${:document-root}"
ErrorLog "$${:error-log}"
LoadModule unixd_module modules/mod_unixd.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule mime_module modules/mod_mime.so
LoadModule cgid_module modules/mod_cgid.so
LoadModule dir_module modules/mod_dir.so
ScriptSock $${:cgid-pid-file}
<Directory $${:document-root}>
# XXX: security????
Options +ExecCGI
AddHandler cgi-script .cgi
DirectoryIndex web-takeover.cgi
</Directory>
output = $${directory:etc}/resilient-web-takeover-httpd.conf
# md5sum =
listening-ip = $${slap-network-information:global-ipv6}
# XXX: randomize-me
listening-port = 9263
htdocs = $${directory:cgi-bin}
pid-file = $${directory:run}/resilient-web-takeover-httpd.pid
cgid-pid-file = $${directory:run}/resilient-web-takeover-httpd-cgid.pid
document-root = $${directory:cgi-bin}
error-log = $${directory:log}/resilient-web-takeover-httpd-error-log
[resilient-web-takeover-httpd-wrapper]
recipe = slapos.cookbook:wrapper
apache-executable = ${apache:location}/bin/httpd
command-line = $${:apache-executable} -f $${resilient-web-takeover-httpd-configuration-file:output} -DFOREGROUND
wrapper-path = $${basedirectory:services}/resilient-web-takeover-httpd
[resilient-web-takeover-httpd-promise]
recipe = slapos.cookbook:check_url_available
path = $${basedirectory:promises}/resilient-web-takeover-httpd
url = http://[$${resilient-web-takeover-httpd-configuration-file:listening-ip}]:$${resilient-web-takeover-httpd-configuration-file:listening-port}/
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
stack/resilient/pbsready.cfg.in
View file @ d007050d
......@@ -10,6 +10,7 @@ parts =
sshkeys-authority
dropbear-server
sshkeys-dropbear
resilient-sshkeys-dropbear-promise
dropbear-server-pbs-authorized-key
notifier
......@@ -49,6 +50,7 @@ crontabs = $${rootdirectory:etc}/crontabs
cronstamps = $${rootdirectory:etc}/cronstamps
logrotate-entries = $${rootdirectory:etc}/logrotate.d
logrotate-backup = $${basedirectory:backup}/logrotate
cgi-bin = $${rootdirectory:srv}/cgi-bin
#----------------
#--
......@@ -179,6 +181,27 @@ server-binary = ${buildout:bin-directory}/pubsubserver
notifier-binary = ${buildout:bin-directory}/pubsubnotifier
#----------------
#--
#-- Dropbear.
[dropbear-server]
recipe = slapos.cookbook:dropbear
host = $${slap-network-information:global-ipv6}
# Explicitely excludes to define "port" argument. It will be defined in
# pbs-ready-import.cfg.in and pbs-ready-export.cfg.in
home = $${directory:ssh}
wrapper = $${rootdirectory:bin}/raw_sshd
shell = $${rdiff-backup-server:wrapper}
rsa-keyfile = $${directory:ssh}/server_key.rsa
dropbear-binary = ${dropbear:location}/sbin/dropbear
[dropbear-server-pbs-authorized-key]
<= dropbear-server
recipe = slapos.cookbook:dropbear.add_authorized_key
key = $${slap-parameter:authorized-key}
#----------------
#--
#-- sshkeys
......@@ -205,30 +228,21 @@ public-key = $${dropbear-server:rsa-keyfile}.pub
private-key = $${dropbear-server:rsa-keyfile}
wrapper = $${basedirectory:services}/sshd
#----------------
#--
#-- Dropbear.
[dropbear-server]
recipe = slapos.cookbook:dropbear
host = $${slap-network-information:global-ipv6}
port = 2222
home = $${directory:ssh}
wrapper = $${rootdirectory:bin}/raw_sshd
shell = $${rdiff-backup-server:wrapper}
rsa-keyfile = $${directory:ssh}/server_key.rsa
dropbear-binary = ${dropbear:location}/sbin/dropbear
[dropbear-server-pbs-authorized-key]
<= dropbear-server
recipe = slapos.cookbook:dropbear.add_authorized_key
key = $${slap-parameter:authorized-key}
[resilient-sshkeys-dropbear-promise]
# Check that public key file exists and is not empty
recipe = collective.recipe.template
input = inline:#!${bash:location}/bin/bash
PUBLIC_KEY_CONTENT="$${sshkeys-dropbear:public-key-value}"
if [[ ! -n "$PUBLIC_KEY_CONTENT" || "$PUBLIC_KEY_CONTENT" == *None* ]]; then
exit 1
fi
output = $${basedirectory:promises}/public-key-existence
mode = 700
#----------------
#--
#-- Conncetion informations to re-use.
#-- Connection informations to re-use.
# XXX-Cedric: when "aggregation" system is done in libslap, directly publish.
[resilient-publish-connection-parameter]
recipe = slapos.cookbook:publish
......
stack/resilient/resilient-web-takeover-cgi-script.py.in 0 → 100644
View file @ d007050d
#!${buildout:executable}
import cgi
import cgitb
import os
import subprocess
import sys
cgitb.enable()
print "Content-Type: text/html"
print
form = cgi.FieldStorage()
if "password" not in form:
print """<html>
<body>
<h1>This is takeover web interface.</h1>
<p>Calling takeover will stop and freeze the current main instance, and make this clone instance the new main instance, replacing the old one.</p>
<p><b>Warning: submit the form only if you understand what you are doing.</b></p>
<p>Note: the password asked here can be found within the parameters of your SlapOS instance page.</p>
<form action="/">
Password: <input type="text" name="password">
<input type="submit" value="Take over" style="background: red;">
</form>
</body>
</html>"""
sys.exit(0)
if form['password'].value != '${:password}':
print "<H1>Error</H1>"
print "Password is invalid."
sys.exit(1)
# XXX hardcoded location
result = subprocess.check_output([os.path.expanduser("~/bin/takeover")], stderr=subprocess.STDOUT)
print 'Success.'
print '<pre>%s</pre>' % result
stack/resilient/resilient.cfg.in deleted 100644 → 0
View file @ 55b2bae2
## not used at the moment
[buildout]
parts =
request-pull-backup-server
[request-pull-backup-server]
<= slap-connection
recipe = slapos.cookbook:request
name = PBS (Pull Backup Server)
software-url = $${slap-connection:software-release-url}
software-type = pull-backup
return = ssh-key notification-url feeds-url
slave = false
\ No newline at end of file
stack/resilient/template-parts.cfg.in
View file @ d007050d
......@@ -4,18 +4,21 @@
request-{{namebase}}
request-{{namebase}}-2
resilient-request-{{namebase}}-public-key-promise
{% for i in range(1,nbbackup|int) %}
request-{{namebase}}-pseudo-replicating-{{i}}
request-{{namebase}}-pseudo-replicating-{{i}}-2
{% for id in range(1,nbbackup|int) %}
request-{{namebase}}-pseudo-replicating-{{id}}
request-{{namebase}}-pseudo-replicating-{{id}}-2
resilient-request-{{namebase}}-pseudo-replicating-{{id}}-public-key-promise
{% endfor %}
{% for i in range(1,nbbackup|int) %}
request-pbs-{{namebase}}-{{i}}
request-pull-backup-server-{{namebase}}-{{i}}
request-pull-backup-server-{{namebase}}-backup-{{i}}
{% for id in range(1,nbbackup|int) %}
request-pbs-{{namebase}}-{{id}}
resilient-request-pbs-{{namebase}}-{{id}}-public-key-promise
request-pull-backup-server-{{namebase}}-{{id}}
request-pull-backup-server-{{namebase}}-backup-{{id}}
{% endfor %}
{% endmacro %}
stack/resilient/template-replicated.cfg.in
View file @ d007050d
{% macro replicate(namebase, nbbackup, typeexport, typeimport, heriteLeader='', heriteBackup='') %}
{% macro replicate(namebase, nbbackup, typeexport, typeimport, heriteLeader='', heriteBackup='', slapparameter_dict={}) %}
{% set sla_parameter_dict = {} -%}
# prepare sla-parameters
{% if slapparameter_dict is defined -%}
{% for key in slapparameter_dict.keys() -%}
{% if key.startswith('-sla-') -%}
{% do sla_parameter_dict.__setitem__(key, slapparameter_dict.pop(key)) -%}
{% endif -%}
{% endfor -%}
{% endif -%}
[resilient-directory]
recipe = slapos.cookbook:mkdirectory
home = ${buildout:directory}
etc = ${:home}/etc
promise = ${:etc}/promise
## Tells the Backupable recipe that we want a backup
[resilient]
......@@ -17,13 +35,45 @@ software-url = ${slap-connection:software-release-url}
software-type = {{typeexport}}
name = {{namebase}}0
return = ssh-public-key ssh-url notification-id ip
config = number authorized-key notify ip-list namebase
config =
# Resilient related parameters
number authorized-key notify ip-list namebase
{% if slapparameter_dict is defined %}
# Software Instance related parameters
{% for parameter_name in slapparameter_dict.keys() %}{{parameter_name}} {% endfor %}
{% endif %}
config-number = 0
config-authorized-key = {% for id in range(1,nbbackup|int) %} ${request-pbs-{{namebase}}-{{id}}:connection-ssh-key}{% endfor %}
config-notify = {% for id in range(1,nbbackup|int) %} ${request-pbs-{{namebase}}-{{id}}:connection-notification-url}{% endfor %}
config-ip-list =
# Bubble up all the instance parameters to the requested export instance.
{% if slapparameter_dict is defined %}
{% for parameter_name, parameter_value in slapparameter_dict.items() %}config-{{parameter_name}} = {{parameter_value}}
{% endfor %}
{% endif %}
{% if sla_parameter_dict == {} -%}
sla = mode
sla-mode = unique_by_network
{% else %}
{% set sla_key_main = "-sla-%s%s-" % (namebase, 0) -%}
{% set sla_key_secondary = "-sla-%s-" % (0) -%}
{% set sla_key_main_length = sla_key_main | length -%}
{% set sla_key_secondary_length = sla_key_secondary | length -%}
{% set sla_dict = {} -%}
{% for key in sla_parameter_dict.keys() -%}
{% if key.startswith(sla_key_main) -%}
{% do sla_dict.__setitem__(key[sla_key_main_length:], sla_parameter_dict.get(key)) -%}
{% elif key.startswith(sla_key_secondary) and not sla_dict.has_key(key[sla_key_secondary_length:]) -%}
{% do sla_dict.__setitem__(key[sla_key_secondary_length:], sla_parameter_dict.get(key)) -%}
{% endif -%}
{% endfor -%}
{% if sla_dict %}
sla = {{ ' '.join(sla_dict.keys()) }}
{% for key, value in sla_dict.iteritems() -%}
sla-{{ key }} = {{ value }}
{% endfor -%}
{% endif -%}
{% endif -%}
{% for id in range(1,nbbackup|int) %}
......@@ -38,20 +88,40 @@ software-url = ${slap-connection:software-release-url}
software-type = {{typeimport}}
return = ssh-public-key ssh-url notification-url ip
pbs-notification-id = ${slap-connection:computer-id}-${slap-connection:partition-id}-{{namebase}}-push
pbs-notification-id = ${slap-connection:computer-id}-${slap-connection:partition-id}-{{namebase}}-{{id}}-push
config = number authorized-key on-notification ip-list namebase
config-number = {{id}}
config-authorized-key = ${request-pbs-{{namebase}}-{{id}}:connection-ssh-key}
config-on-notification = ${request-pbs-{{namebase}}-{{id}}:connection-feeds-url}${:pbs-notification-id}
config-ip-list =
{% if sla_parameter_dict == {} -%}
sla = mode
sla-mode = unique_by_network
{% else %}
{% set sla_key_main = "-sla-%s%s-" % (namebase, id) -%}
{% set sla_key_secondary = "-sla-%s-" % (id) -%}
{% set sla_key_main_length = sla_key_main | length -%}
{% set sla_key_secondary_length = sla_key_secondary | length -%}
{% set sla_dict = {} -%}
{% for key in sla_parameter_dict.keys() -%}
{% if key.startswith(sla_key_main) -%}
{% do sla_dict.__setitem__(key[sla_key_main_length:], sla_parameter_dict.get(key)) -%}
{% elif key.startswith(sla_key_secondary) and not sla_dict.has_key(key[sla_key_secondary_length:]) -%}
{% do sla_dict.__setitem__(key[sla_key_secondary_length:], sla_parameter_dict.get(key)) -%}
{% endif -%}
{% endfor -%}
{% if sla_dict %}
sla = {{ ' '.join(sla_dict.keys()) }}
{% for key, value in sla_dict.iteritems() -%}
sla-{{ key }} = {{ value }}
{% endfor -%}
{% endif %}
{% endif %}
sla = computer_guid
sla-computer_guid = ${slap-parameter:{{namebase}}{{id}}-computer-guid}
{% endfor -%}
{% endfor %}
[iplist]
config-ip-list = ${request-{{namebase}}:connection-ip}{% for j in range(1,nbbackup|int) %} ${request-{{namebase}}-pseudo-replicating-{{j}}:connection-ip}{% endfor %}
......@@ -59,11 +129,37 @@ config-ip-list = ${request-{{namebase}}:connection-ip}{% for j in range(1,nbback
<= request-{{namebase}}
iplist
[resilient-request-{{namebase}}-public-key-promise]
# Check that public-key-value parameter exists and is not empty
# XXX: maybe we should consider empty values to be non-nexistent.
recipe = collective.recipe.template
# XXX: don't use system executable
input = inline:#!/bin/sh
PUBLIC_KEY_CONTENT="${request-{{namebase}}-2:connection-ssh-public-key})"
if [[ ! -n "$PUBLIC_KEY_CONTENT" || "$PUBLIC_KEY_CONTENT" == *None* ]]; then
exit 1
fi
output = ${resilient-directory:promise}/resilient-request-{{namebase}}-public-key
mode = 700
{% for id in range(1,nbbackup|int) %}
[request-{{namebase}}-pseudo-replicating-{{id}}-2]
<= request-{{namebase}}-pseudo-replicating-{{id}}
iplist
[resilient-request-{{namebase}}-pseudo-replicating-{{id}}-public-key-promise]
# Check that public-key-value parameter exists and is not empty
# XXX: maybe we should consider empty values to be non-nexistent.
recipe = collective.recipe.template
# XXX: don't use system executable
input = inline:#!/bin/sh
PUBLIC_KEY_CONTENT="${request-{{namebase}}-pseudo-replicating-{{id}}-2:connection-ssh-public-key})"
if [[ ! -n "$PUBLIC_KEY_CONTENT" || "$PUBLIC_KEY_CONTENT" == *None* ]]; then
exit 1
fi
output = ${resilient-directory:promise}/resilient-request-{{namebase}}-pseudo-replicating-{{id}}-public-key
mode = 700
{% endfor %}
......@@ -75,8 +171,6 @@ config-ip-list = ${request-{{namebase}}:connection-ip}{% for j in range(1,nbback
## Having 3 backups pulling from the same PBS provides
##only availability, not resiliency
## WARNING : SLAVES ARE ALLOCATED AT RANDOM, THIS NEEDS TO BE FIXED.
[request-pbs-common]
<= slap-connection
recipe = slapos.cookbook:request
......@@ -90,21 +184,57 @@ software-type = pull-backup
name = PBS ({{namebase}} / {{id}})
return = ssh-key notification-url feeds-url
slave = false
sla = computer_guid
sla-computer_guid = ${slap-parameter:pbs-{{namebase}}{{id}}-computer-guid}
{% if sla_parameter_dict == {} -%}
sla = mode
sla-mode = unique_by_network
{% else %}
{% set sla_key_main = "-sla-%s%s-" % ("pbs", id) -%}
{% set sla_key_secondary = "-sla-%s-" % (id) -%}
{% set sla_key_main_length = sla_key_main | length -%}
{% set sla_key_secondary_length = sla_key_secondary | length -%}
{% set sla_dict = {} -%}
{% for key in sla_parameter_dict.keys() -%}
{% if key.startswith(sla_key_main) -%}
{% do sla_dict.__setitem__(key[sla_key_main_length:], sla_parameter_dict.get(key)) -%}
{% elif key.startswith(sla_key_secondary) and not sla_dict.has_key(key[sla_key_secondary_length:]) -%}
{% do sla_dict.__setitem__(key[sla_key_secondary_length:], sla_parameter_dict.get(key)) -%}
{% endif -%}
{% endfor -%}
{% if sla_dict %}
sla = {{ ' '.join(sla_dict.keys()) }}
{% for key, value in sla_dict.iteritems() -%}
sla-{{ key }} = {{ value }}
{% endfor %}
{% endif %}
{% endif %}
[resilient-request-pbs-{{namebase}}-{{id}}-public-key-promise]
# Check that public-key-value parameter exists and is not empty
# XXX: maybe we should consider empty values to be non-nexistent.
recipe = collective.recipe.template
# XXX: don't use system executable
input = inline:#!/bin/sh
PUBLIC_KEY_CONTENT="${request-pbs-{{namebase}}-{{id}}:connection-ssh-key}:connection-ssh-key})"
if [[ ! -n "$PUBLIC_KEY_CONTENT" || "$PUBLIC_KEY_CONTENT" == *None* ]]; then
exit 1
fi
output = ${resilient-directory:promise}/resilient-request-{{namebase}}-pseudo-replicating-{{id}}-public-key
mode = 700
[request-pull-backup-server-{{namebase}}-{{id}}]
<= request-pbs-common
name = PBS {{id}} pulling from ${request-{{namebase}}:name}
config = url name type server-key on-notification notify notification-id title
config = url name type server-key on-notification notify notification-id title remove-backup-older-than
config-url = ${request-{{namebase}}:connection-ssh-url}
config-name = ${slap-connection:computer-id}-${slap-connection:partition-id}-{{namebase}}-{{id}}
config-type = pull
config-server-key = ${request-{{namebase}}:connection-ssh-public-key}
config-on-notification = ${request-{{namebase}}:connection-notification-id}
config-notify = ${request-pbs-{{namebase}}-{{id}}:connection-notification-url}
config-notification-id = ${slap-connection:computer-id}-${slap-connection:partition-id}-{{namebase}}-{{id}}-pull
config-name = ${slap-connection:computer-id}-${slap-connection:partition-id}-{{namebase}}-{{id}}
config-title = Pulling from {{namebase}}
config-remove-backup-older-than = {{ slapparameter_dict.get('remove-backup-older-than', '2W') }}
slave = true
sla = instance_guid
sla-instance_guid = ${request-pbs-{{namebase}}-{{id}}:instance_guid}
......@@ -114,12 +244,12 @@ sla-instance_guid = ${request-pbs-{{namebase}}-{{id}}:instance_guid}
name = PBS pushing on ${request-{{namebase}}-pseudo-replicating-{{id}}:name}
config = url name type server-key on-notification notify notification-id title
config-url = ${request-{{namebase}}-pseudo-replicating-{{id}}:connection-ssh-url}
config-name = ${request-pull-backup-server-{{namebase}}-{{id}}:config-name}
config-type = push
config-server-key = ${request-{{namebase}}-pseudo-replicating-{{id}}:connection-ssh-public-key}
config-on-notification = ${request-pbs-{{namebase}}-{{id}}:connection-feeds-url}${request-pull-backup-server-{{namebase}}-{{id}}:config-notification-id}
config-notify = ${request-{{namebase}}-pseudo-replicating-{{id}}:connection-notification-url}
config-notification-id = ${request-{{namebase}}-pseudo-replicating-{{id}}:pbs-notification-id}
config-name = ${slap-connection:computer-id}-${slap-connection:partition-id}-{{namebase}}-{{id}}
config-title = Pushing to {{namebase}} backup {{id}}
slave = true
sla = instance_guid
......@@ -135,3 +265,4 @@ pbs-{{namebase}}{{id}}-computer-guid =
{% endfor %}
{% endmacro %}
stack/slapos.cfg
View file @ d007050d
......@@ -12,6 +12,7 @@ extensions +=
# Use shacache and lxml
extends =
../component/lxml-python/buildout.cfg
../component/python-openssl/buildout.cfg
# Separate from site eggs
allowed-eggs-from-site-packages =
......@@ -59,6 +60,7 @@ networkcache-section = networkcache
recipe = zc.recipe.egg
eggs =
${lxml-python:egg}
${python-openssl:egg}
slapos.cookbook
cliff
hexagonit.recipe.download
......@@ -73,11 +75,16 @@ eggs =
[versions]
# Use SlapOS patched zc.buildout
zc.buildout = 1.6.0-dev-SlapOS-010
zc.buildout = 1.6.0-dev-SlapOS-012
# zc.recipe.egg 2.x is for Buildout 2
zc.recipe.egg = 1.3.2
# Use own version of h.r.download to be able to open xz-like archives
hexagonit.recipe.download = 1.6nxd002
hexagonit.recipe.download = 1.7nxd002
# Use pinned version of setuptools. Other versions work, but changing
# version makes buildout recompile everything. Developers' nightmare.
setuptools = 0.9.8
# Official egg of prettytable has permission problems in EGG-INFO.
prettytable = 0.7.3-nxd001
[networkcache]
download-cache-url = http://www.shacache.org/shacache
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7