diff --git a/buildout/README-2.12.txt b/buildout/README-2.12.txt new file mode 100644 index 0000000000000000000000000000000000000000..1a982399701ce89d9be6be90561ae9a1b75dee91 --- /dev/null +++ b/buildout/README-2.12.txt @@ -0,0 +1,192 @@ +=========================== +ERP5 buildout for Zope 2.12 +=========================== + +Introduction +============ + +ERP5 Buildout provides a way to build and manage ERP5 software components +with all needed dependencies. + +ERP5 Buildout also provides a way to manage separate installation instances of +ERP5 to share non-data components of an ERP5 software installation from a +single location, allowing for easy component upgrade. + +Software +======== + +Software part shall be system independent. In perfect world it shall depend +only on: + + * C compiler + * standard C and C++ library + * operating system kernel + +As the world is not yet perfect, some additional build time requirements are +needed. See below for a way to acquire the list of system dependencies through +helpers. + +Setup +----- + +Checkout: https://svn.erp5.org/repos/public/erp5/trunk/buildout/ +For example: + + svn co https://svn.erp5.org/repos/public/erp5/trunk/buildout/ ~/erp5.buildout + +Run the Zope 2.12 buildout: + + $ cd ~/erp5.buildout + $ python2.6 bootstrap/bootstrap.py + $ bin/buildout -c buildout-2.12.cfg + +This will download and install the software components needed to run ERP5 on +Zope 2.12 including Zope 2.12 plus dependencies (including +Acquisition with _aq_dynamic patch) and CMF 2.2 plus dependencies. + +System dependency check +----------------------- + +Each software component in this buildout might require some system +dependencies, including development libraries and executables. +To query what is required for all components, please run: + + $ bin/buildout install show-requirements + +Minimal requirements +-------------------- + +At the very least, running buildout requires: + + * Python 2.4 or later including development files (e.g. python2.4-devel or + python2.4-dev package from your system package manager. A file like + /usr/lib*/python*/config/Makefile should be installed in the system. + XXX Since we compile our own python, are development files still necessary?) + * C development toolchain (Make, gcc, gpp, etc.) + * subversion (svn) client, to check-out this buildout. + +Post-build check +---------------- + +There isn't yet a post-build check for running ERP5 on Zope 2.12. + +When it's ready, you can check if all components are working correctly by +typing: + + $ make assert + +Distribution helpers +-------------------- + +In the 'helpers' directory there are shell scripts to prepare different +GNU/Linux distributions to run this buildout. + +For instance, to prepare Mandriva 2010.0 please type the following with root +privileges: + + helpers/mandriva2010.0.sh + +Please refer to the 'helpers' directory for other distributions. + +Instances +========= + +Note: Instance generation is still a work in progress. + +After the software components are built, we can generate ERP5 instance +buildouts from that software. + +Assuming the ERP5 software buildout is available in ~/erp5.buildout the +following sequence of steps should result in a working "instance" buildout: + +$ mkdir ~/instances # 0 +$ cd ~/instances # 1 +$ ln -s ~/erp5.buildout/*profiles* # 2 +$ cat > buildout.cfg # 3 + +[buildout] +extends-cache = instance-profiles/extends-cache +extends = + profiles/development-2.12.cfg + instance-profiles/software-home.inc + +parts = + mysql-instance + oood-instance + supervisor-instance +^D +$ ~/erp5.buildout/bin/bootstrap2.6 # 4 +$ bin/buildout -ov # 5 + +Notice how we managed to run buildout in "offline-mode" (-o). The software-home +configuration (along with the 'extends-cache' in the 'instance-profiles' +symlink) provides all the information and components that would otherwise have +to be downloaded. + +The steps above generate instance configurations for mysql and the +OpenOffice.org document conversion daemon. We need mysql, in particular, +to be running before configuring an actual ERP5 instance, so we'll start +supervisor: + +$ var/bin/supervisord # 6 + +Now edit buildout.cfg and add "runUnitTest" (w/o quotes) to 'buildout:parts'. +The "development-instance" part will be pulled in automatically as a +dependency: + +$ $EDITOR buildout.cfg # 7 + +Then run buildout again to finish the configuration + +$ bin/buildout -ov # 8 + +Now a fully configured development instance will be available in the directory +"var/development-instance", so you can do: + + $ var/development-site/bin/zopectl fg + +And see an ERP5 instance running on "http://localhost:18080/". The port '18080' +refers to the 'development-instance:http-address' setting in +'instance-profiles/development-2.12.cfg'. The file +'instance-profiles/zope-2.12.cfg' provides the "Manager" credentials you should +use (usually zope:zope), in the 'zope-instance-template:user' variable. + +You should also be able to run ERP5 unit tests like so: + + $ bin/runUnitTest --erp5_sql_connection_string="test@127.0.0.1:10002 test" testBusinessTemplate + +The '127.0.0.1:10002' coordinate above refers to the address of the configured +mysql instance, according to the settings 'configuration:mysql_host' and +'configuration:mysql_port' in 'instance-profiles/mysql.cfg'. + +TODO +==== + + * Adjust the 'runUnitTest' recipe to push the mysql server coordinates into + the 'bin/runUnitTest' script. + + * Refactor the .cfg files to reduce duplication and so that only the + 'instance-profiles' directory needs to be symlinked. Alternatively, push all + .cfg files into a single 'profiles' + directory. + + * Combine steps 2, 3 and 4 into a single step by creating a more powerful + 'bootstrap2.6' script. + + * Running 'buildout' twice in the instance (once to configure 'supervisor', + 'mysql' and 'oood' and once to setup the ZODB ERP5 instance) is confusing + and error-prone. A buildout shouldn't deal with persistent state, only with + file installation. Move the mysql database and ERP5 ZODB instance creation + procedures to dedicated scripts in 'bin/' instead of implicitly running them + in the (second) buildout run. + + * Patch the SOAPpy package provided by Nexedi so it doesn't fail with a + SyntaxError on Python 2.6. Right now we're using a SOAPpy repackaging from + http://ibid.omnia.za.net/eggs/ . + + * Synchronize the buildout behaviour for Zope 2.8 and 2.12 (i.e. allows Zope + 2.8 to work with a single buildout check-out). + + * See if we can use http://pypi.python.org/pypi/zc.sourcerelease/ to generate + a single (humongous) tarball with all needed software components for fully + offline operation. diff --git a/buildout/buildout-2.12.cfg b/buildout/buildout-2.12.cfg index 1a2a086e69407b6eefb966f1629b09f632d027a2..c6fe872cf5b081c81274fd69faac43b26a1e5be6 100644 --- a/buildout/buildout-2.12.cfg +++ b/buildout/buildout-2.12.cfg @@ -1,2 +1,3 @@ [buildout] +extends-cache = instance-profiles/extends-cache extends = profiles/official-2.12.cfg diff --git a/buildout/instance-profiles/zope-2.12.cfg b/buildout/instance-profiles/zope-2.12.cfg new file mode 100644 index 0000000000000000000000000000000000000000..70c4b909585985f585474eae0bbab1a8f1fc477e --- /dev/null +++ b/buildout/instance-profiles/zope-2.12.cfg @@ -0,0 +1,77 @@ +# adapt https://svn.erp5.org/repos/public/experimental/erp5.buildout +[buildout] +parts = zope-instance + +[zope-instance-template] +recipe = erp5.recipe.standaloneinstance +user = zope:zope + +# modify this to reflect your local mysql configuration +# Format: +# database[@host[:port]] [user [password [unix_socket]]] +# e.g "erp5db erp5user somepassword" or "erp5db erp5user" +erp5_sql_connection_string = ${:mysql_database_name}@${:mysql_host}:${:mysql_port} ${:mysql_user} ${:mysql_password} + +zope_conf_template = + ${buildout:software_home}/templates/default-erp5-standalone-zope.conf.in + +portal_id = erp5 +force-zodb-update = false +# zope.conf template part BEGIN +# instancehome and http-address have to be defined +# User modifable keys: +environment = +debug-mode = off +ip-address = 0.0.0.0 +# Good defaults, which do not have to be touched: +clienthome = ${:instancehome}/var +lib_path = ${:instancehome}/lib/python +effective-user = +enable-product-installation = on +locale = +datetime-format = us +zserver-threads = 4 +python-check-interval = 500 +zserver-read-only-mode = off +pid-filename = ${:clienthome}/Z2.pid +lock-filename = ${:clienthome}/Z2.lock +mime-types = +structured-text-header-level = 3 +rest-input-encoding = +rest-output-encoding = +rest-header-level = 3 +rest-language-code = en +cgi-environment = +dns-server = +http-realm = Zope +cgi-maxlen = 0 +http-header-max-length = 8192 +automatically-quote-dtml-request-data = on +trusted-proxy = +publisher-profile-file = +security-policy-implementation = C +skip-authentication-checking = off +skip-ownership-checking = off +verbose-security = off +maximum-number-of-session-objects = 1000 +session-add-notify-script-path = +session-delete-notify-script-path = +session-timeout-minutes = 20 +session-resolution-seconds = 20 +suppress-all-access-rules = off +suppress-all-site-roots = off +database-quota-size = +read-only-database = off +eventlog-level = info +eventlog-path = ${:instancehome}/log/event.log +access-level = WARN +access-path = ${:instancehome}/log/Z2.log +access-format = %(message)s +warnfilter = +max-listen-sockets = 100 +port-base = 0 +large-file-threshold = +default-zpublisher-encoding = utf8 +http-force-connection-close = off +zodb-path = ${:instancehome}/var/Data.fs +# zope.conf template part ENDS diff --git a/buildout/profiles/common.cfg b/buildout/profiles/common.cfg index 80a71a69c51f020ca2fc4c50421338428e046084..50559feec3e147cb3a3ca84168a31572578b8309 100644 --- a/buildout/profiles/common.cfg +++ b/buildout/profiles/common.cfg @@ -1,9 +1,9 @@ [buildout] develop += - local-eggs/erp5.recipe.standaloneinstance - local-eggs/erp5.recipe.bt5checkout - local-eggs/erp5.recipe.mysqldatabase - local-eggs/erp5.recipe.mysqlserver + ${:software_home}/local-eggs/erp5.recipe.standaloneinstance + ${:software_home}/local-eggs/erp5.recipe.bt5checkout + ${:software_home}/local-eggs/erp5.recipe.mysqldatabase + ${:software_home}/local-eggs/erp5.recipe.mysqlserver find-links = http://download.zope.org/distribution/ @@ -11,6 +11,7 @@ find-links = http://download.hforge.org/itools/ http://dist.repoze.org http://www.nexedi.org/static/packages/source/ + http://ibid.omnia.za.net/eggs/ [create-directories] recipe = plone.recipe.command diff --git a/buildout/profiles/deployment-2.12.cfg b/buildout/profiles/deployment-2.12.cfg new file mode 100644 index 0000000000000000000000000000000000000000..228cde2c243f3a7e3c8b38ea6da14f80bc92b858 --- /dev/null +++ b/buildout/profiles/deployment-2.12.cfg @@ -0,0 +1,37 @@ +[buildout] +extends = + ../instance-profiles/mysql-database.cfg + ../instance-profiles/mysql.cfg + ../instance-profiles/oood.cfg + ../instance-profiles/supervisor.cfg + ../instance-profiles/zope-2.12.cfg + ../profiles/software-definition.cfg + ../profiles/versions-2.12.cfg + ../software-profiles/erp5-2.12.cfg + ../profiles/common.cfg + +parts = + eggs + mysql-instance + oood-instance + supervisor-instance + +python = software_definition +versions = versions + +[eggs] +scripts += + mkzopeinstance + runzope + +[configuration] +supervisor_programs = + 10 mysql (autorestart=true) ${buildout:data-bin-directory}/pidproxy [ ${configuration:mysql_pid} ${buildout:data-bin-directory}/mysqld_safe ] + 20 oood (autorestart=true) ${buildout:data-bin-directory}/pidproxy [ ${configuration:oood_pid} ${buildout:data-bin-directory}/oood-runserw --start ] ${buildout:etc-directory} + +#[software-links] +#recipe = plone.recipe.command +#command = +# ln -sf ${software_definition:software_home}/bin/python2.4 ${buildout:bin-directory}/python2.4 +# ln -sf ${software_definition:software_home}/bin/ipython2.4 ${buildout:bin-directory}/ipython2.4 +#update-command = ${:command} diff --git a/buildout/profiles/development-2.12.cfg b/buildout/profiles/development-2.12.cfg new file mode 100644 index 0000000000000000000000000000000000000000..bdc6565c8d296a508dd6e55ff70486add48d45fa --- /dev/null +++ b/buildout/profiles/development-2.12.cfg @@ -0,0 +1,72 @@ +[buildout] +extends = + ../profiles/deployment-2.12.cfg + +parts += + development-site + +[software_definition] +software_home = ${buildout:directory} + +[runUnitTest] +recipe = erp5.recipe.testrunner +eggs = ${development-site:eggs} +extra-paths = ${development-site:extra-paths} +products = + ${development-site:products} + +# XXX have this list be populated by software_definition keys. Maybe adjust +# erp5.recipe.testrunner to accept this parameter in multiple lines and convert +# to this comma separated list. +bt5_path = ${development-site:bt5-path},${buildout:software_home}/parts/bt5-erp5,${buildout:software_home}/parts/bt5-erp5-extra + +instance-home = ${development-site:instancehome} +initialization = + import os + os.environ['PATH'] = ':'.join(['${buildout:data-bin-directory}','${buildout:bin-directory}'] + os.environ.get('PATH','').split(':')) + +[development-site] +<= zope-instance-template +# this is provided by the enhanced [eggs] in deployment-2.12.cfg +mkzopeinstance = ${buildout:directory}/bin/mkzopeinstance +# developer by default want to have updatable Data.fs +force-zodb-update = true +bt5-path = +bt5 = +eggs = + Products.ExternalEditor + ${eggs:eggs} +extra-paths = ${eggs:extra-paths} + +# MySQL +mysql_database_name = development_site +mysql_user = development_user +mysql_password = development_password +mysql_host = ${configuration:mysql_host} +mysql_port = ${configuration:mysql_port} + +# create database +mysql_create_database = true +# below could be set in configuration +mysql_superuser = root +mysql_superpassword = + +# zope.conf template part BEGIN +debug-mode = on +instancehome = ${buildout:var-directory}/development-site +http-address = 18080 +# until we will have more control over products we need to redefine to insert +# local products to be able to override +products = + ${:instancehome}/Products/ + ${software_definition:products-erp5} + ${software_definition:products-erp5-development} + ${software_definition:products-deps} +# zope.conf template part ENDS + +[requirements] +binary += + svn + svnadmin +development += + /usr/include.*/svn_client.h diff --git a/buildout/profiles/official-2.12.cfg b/buildout/profiles/official-2.12.cfg index 041bbc6c20d9c97de46fe8668134d2e4dbdf072c..16de2b17ad798dd371de8d78ec1df88241734b2c 100644 --- a/buildout/profiles/official-2.12.cfg +++ b/buildout/profiles/official-2.12.cfg @@ -22,10 +22,11 @@ extends = ../software-profiles/varnish.cfg ../profiles/common.cfg -python = python2.6 +python = software_definition versions = versions parts = + itools-build eggs apache memcached @@ -42,7 +43,9 @@ parts = products # omelette bootstrap2.6 -# software_home + bt5-erp5 + bt5-erp5-extra + software_home [show-requirements] recipe = plone.recipe.command @@ -53,9 +56,10 @@ update-command = ${:command} [software_definition] software_home = ${buildout:software_home} +executable = ${python2.6:executable} [software_home] recipe = collective.recipe.template input = ${buildout:software_home}/templates/software-home.inc.in -output = ${buildout:directory}/instance-profiles/software-home.inc +output = ${buildout:software_home}/instance-profiles/software-home.inc diff --git a/buildout/profiles/versions-2.12.cfg b/buildout/profiles/versions-2.12.cfg index b21491a00bfcaad203c10161a0beb7df9b442328..36129831c43464481e118dd95474aefbeb921cde 100644 --- a/buildout/profiles/versions-2.12.cfg +++ b/buildout/profiles/versions-2.12.cfg @@ -1,7 +1,7 @@ [buildout] extends = - http://svn.zope.org/repos/main/Zope/branches/2.12/versions.cfg + http://svn.zope.org/repos/main/Zope/tags/2.12.4/versions.cfg [versions] # XXX refactor parts shared with versions.cfg into a common file @@ -15,7 +15,14 @@ numpy = 1.3.0 plone.recipe.zope2instance = 3.6 erp5.recipe.standaloneinstance >= 0.4 pysvn = 1.7.2 -Zope2 = 2.12.3 +# Use SOAPpy from http://ibid.omnia.za.net/eggs/ instead of +# http://www.nexedi.org/static/packages/source/ due to SyntaxError on +# Python 2.6 +SOAPpy = 0.12.0 + +# clear the Acquisition version pin so our development checkout in local-eggs +# is used instead +Acquisition = erp5_products_revision = erp5_bt5_revision = ${:erp5_products_revision} diff --git a/buildout/software-profiles/erp5-2.12.cfg b/buildout/software-profiles/erp5-2.12.cfg index 6a6b97c4527d45597f1c741e1a9d773acad26eb8..072fef5ddb6ac489ecba4101ea99a54138a5899d 100644 --- a/buildout/software-profiles/erp5-2.12.cfg +++ b/buildout/software-profiles/erp5-2.12.cfg @@ -1,5 +1,9 @@ # XXX refactor parts shared with erp5.cfg into a common file [buildout] +extends = + mysql-tritonn-5.0.cfg + python-2.6.cfg + parts = itools-build mysql-python @@ -8,34 +12,46 @@ parts = products-erp5 develop += + ${:software_home}/local-eggs/Products.DCWorkflow # The patched Acquisition - local-eggs/Acquisition - local-eggs/Products.DCWorkflow + ${:software_home}/local-eggs/Acquisition + +# ugly itools building sections until itools and setuptools learn to get +# along: [itools] -recipe = hexagonit.recipe.download -extra-paths = ${buildout:directory}/parts/site-packages +# This part should not contain contain dependencies to other parts and should +# contain only paths relative to software_home as it will be used directly by +# instances. It should probably be moved to a .cfg file inside +# "instance-profiles" pkgname = itools-0.50.8 -url = http://download.hforge.org/itools/0.50/${:pkgname}.tar.gz +location = ${buildout:software_home}/parts/itools +lib = ${:location}/lib +source = ${:location}/src + +[itools-download] +recipe = hexagonit.recipe.download +strip-top-level-dir = true +url = http://download.hforge.org/itools/0.50/${itools:pkgname}.tar.gz +destination = ${itools:source} [itools-build] recipe = collective.recipe.cmd -lib = ${itools:location}/lib -source = ${itools:location}/${itools:pkgname} on_install = True on_update = False +# reference itools-download:destination instead of itools:source to trigger its +# execution cmds = - cd ${:source} -# ${python2.6:executable} setup.py install_lib --install-lib=${:lib} - ${python2.6:executable} setup.py install_lib -d ${:lib} + cd ${itools-download:destination} + ${python2.6:executable} setup.py install_lib -d ${itools:lib} uninstall_cmds = - cd ${:source} - ${buildout:executable} setup.py clean - rm -rf ${:lib} + cd ${itools-download:destination} + ${python2.6:executable} setup.py clean + rm -rf ${itools:lib} [mysql-python-env] -PATH =${mysql-tritonn-5.0:location}/bin:%(PATH)s +PATH =${software_definition:mysql_software}/bin:%(PATH)s [mysql-python] recipe = zc.recipe.egg:custom @@ -105,18 +121,27 @@ eggs = # Other products Products.DCWorkflowGraph Products.MimetypesRegistry + Products.ExternalEditor # Currently forked in our repository # Products.PortalTransforms +# Dependency for our fork of PortalTransforms + StructuredText extra-paths = - ${itools-build:lib} + ${itools:lib} dependent-scripts = true -python = python2.6 +# XXX getting python from [buildout] for the benefit of instance recipes +# which should not attempt to compile python again +#python = python2.6 + +# parameterizing the version of the generated python interpreter name by the +# python section version causes dependency between this egg section and the +# installation of python, which we don't want on an instance interpreter = python2.6 scripts = - python=python${python2.6:version} - ipython=ipython${python2.6:version} + python=${:interpreter} + ipython=i${:interpreter} [omelette] # XXX don't use this part until this omelette bug is fixed: @@ -124,7 +149,7 @@ scripts = recipe = collective.recipe.omelette eggs = ${eggs:eggs} packages = - ${itools-build:lib} . + ${itools:lib} . products = ${products:paths} diff --git a/buildout/templates/software-home.inc.in b/buildout/templates/software-home.inc.in index a99d2d47d0331e984fd3e4971bcbf86b00f0957d..1c2855569db5fa922f002a3032d20b6a7fb617a4 100644 --- a/buildout/templates/software-home.inc.in +++ b/buildout/templates/software-home.inc.in @@ -1,3 +1,8 @@ [buildout] software_home = ${buildout:software_home} eggs-directory = ${buildout:eggs-directory} +develop-eggs-directory = ${buildout:develop-eggs-directory} + +[software_definition] +software_home = ${buildout:software_home} +executable = ${software_definition:executable}