Add incpozo, the incremental repozo restorer
-
Maintainer
@kirr: What do you think of this change ? Should I try to push upstream (meaning ZODB's repozo) ? If so, do you have ideas abou the XXX left (or about this code in general) ?
I'm hesitant to include it in "zodb" main command, but mainly because it allows copying just one file to the server I need to run it on, and getting a working executable (modulo a working python interpreter with ZODB egg), instead of having to install it as an egg. So it's not a strong opposition anyway (more like a side effect of how I developped and bootstraped for first use).
-
@vpelletier thanks for heads up on a useful change.
Since this is an improvement to repozo, I would suggest to try to push it to ZODB upstream first (with tests) - as imho this way it would be the most useful for everyone. Repozo however seems to be long without maintainer and this way if upstreaming won't work we can keep the tool under zodbtools umbrella. However in such case I suggest we don't add just
incpozo
but fork full repozo into zodbtools and add our patches on top - the same way we did for analyze (see nexedi/zodbtools!1 (closed), ab17cf2d, 1e506a81).If we go zodbtools way my preference would be not to create another top level command but to have only
zodb
and keep everything under this toolbox as subcommands. By the way for tools that are working not with general ZODB databases (like e.g.zodb dump
andzodb cmp
do) I would prefer we create a subcommand covering particular topic - e.g.zodb fs1 repozo
. We already have zodbtools egg installed as part of ERP5 SR so from deployment point of view it should not be a problem to depend on zodbtools being available:https://lab.nexedi.com/nexedi/slapos/blob/04c27ca1/software/neoppod/software-common.cfg#L51
About code itself, and particular about places marked as XXX I suggest by default
recover --incremental
to treat situations that catch inconsistencies as error. However with e.g.--increment-full-fallback
upon hitting an error it should fallback to recovering file from scratch.By the way on recovering I suggest repozo to automatically check checksums of backup parts. I quickly grasped repozo code and while we have them in .dat files they are not checked. The files are read while recovering anyway so it should be cheap to check.
-
BTW, I don't agree with Jim on https://github.com/zopefoundation/ZODB/pull/128
If such tool is maintained by ZODB developers, and I think it should be the case, then the ZODB repository is the right place. I see no reason to split the repository. It's important for ZODB and FileStorage to come with maintenance tools (similarly, who'd use a FS without a fsck tool?). And the lack of tests is irrelevant (splitting the repository won't create them magically, or help creating them).
Scripts that have no place in ZODB.git would be those that are big and rarely used.
-
@jm that was my thinking when I filed above-mentioned pull request. Jim's rejections
https://github.com/zopefoundation/ZODB/pull/128#issuecomment-260970932
was the only reason to start zodbtools.
-
mentioned in merge request nexedi/zodbtools!4