This commit was manufactured by cvs2svn to create tag 'r0-9-2'.

git-svn-id: http://svn.savannah.nongnu.org/svn/rdiff-backup@151 2b77aa54-bcbc-44c9-a7ec-4f6cf2b41109

tags/r0-9-2/rdiff-backup/FAQ-body.html

+<h3>Table of contents</h3>
+
+<ol><li><a href="#__future__">When I try to run rdiff-backup it says
+"ImportError: No module named __future__" or "SyntaxError: invalid
+syntax".  What's happening?</a></li>
+
+<li><a href="#verbosity">What do the different verbosity levels mean?</a></li>
+
+<li><a href="#windows">Does rdiff-backup run under Windows?</a></li>
+
+<li><a href="#remove_dir">My backup set contains some files that I just realized I don't want/need backed up.  How do I remove them from the backup volume to save space?</li>
+
+<li><a href="#redhat">How do I install the RPMs on Redhat linux system?</a></li>
+
+<li><a href="#solaris">Does rdiff-backup work under Solaris?</a></li>
+
+</ol>
+
+<h3>Questions and Answers</h3>
+
+<ol>
+
+<a name="__future__">
+<li><strong>When I try to run rdiff-backup it says "ImportError: No
+module named __future__" or "SyntaxError: invalid syntax".  What's
+happening?</strong>
+
+<P>rdiff-backup versions 0.2.x require Python version 2.1 or later,
+and versions 0.3.x and later require Python version 2.2 or later.  If
+you don't know what version of python you are running, type in "python
+-V" from the shell.  I'm sorry if this is inconvenient, but
+rdiff-backup uses generators, iterators, nested scoping, and
+static/class methods extensively, and these were only added in version
+2.2.
+
+<P>If you have two versions of python installed, and running "python"
+defaults to an early version, you'll probably have to change the first
+line of the rdiff-backup script.  For instance, you could set it to:
+
+<pre>
+#/usr/bin/env python2.2
+</pre>
+</li>
+
+<a name="verbosity">
+<li><strong>What do the different verbosity levels mean?</strong>
+
+<P>There is no formal specification, but here is a rough description
+(settings are always cumulative, so 5 displays everything 4 does):
+
+<P>
+<table cellspacing="10">
+<tr><td>0</td><td>No information given</td></tr>
+<tr><td>1</td><td>Fatal Errors displayed</td></tr>
+<tr><td>2</td><td>Warnings</td></tr>
+<tr><td>3</td><td>Important messages, and maybe later some global statistics (default)</td></tr>
+<tr><td>4</td><td>Some global settings, miscellaneous messages</td></tr>
+<tr><td>5</td><td>Mentions which files were changed</td></tr>
+<tr><td>6</td><td>More information on each file processed</td></tr>
+<tr><td>7</td><td>More information on various things</td></tr>
+<tr><td>8</td><td>All logging is dated</td></tr>
+<tr><td>9</td><td>Details on which objects are moving across the connection</td></tr>
+</table>
+
+<a name="windows">
+<li><strong>Does rdiff-backup run under Windows?</strong>
+
+<P>Yes, apparently it is possible.  First, follow Jason Piterak's
+instructions:
+
+<pre>
+Subject: Cygwin rdiff-backup
+From: Jason  Piterak &lt;Jason_Piterak@c-i-s.com&gt;
+Date: Mon, 4 Feb 2002 16:54:24 -0500 (13:54 PST)
+To: rdiff-backup@keywest.Stanford.EDU
+
+Hello all,
+  On a lark, I thought I would attempt to get rdiff-backup to work under
+Windows98 under Cygwin. We have a number of NT/Win2K servers in the field
+that I'd love to be backing up via rdiff-backup, and this was the start of
+getting that working. 
+
+SUMMARY: 
+  o You can get all the pieces for rdiff-backup working under Cygwin.
+  o The backup process works up to the point of writing any files with
+timestamps.
+      ... This is because the ':' character is reserved for Alternate Data
+Stream (ADS) file designations under NTFS.
+
+HOW TO GET IT WORKING (to a point, anyway):
+  o Install Cygwin
+  o Download the Python 2.2 update through the Cygwin installer and install.
+  o Download the librsync libraries from the usual place, but before
+compiling...
+  o Cygwin does not use/provide glibc. Because of this, you have to repoint
+some header files in the Makefile:
+
+   -- Make sure that you have /usr/include/inttypes.h
+      redirected to /usr/include/sys/types.h. Do this by:
+
+      create a file /usr/include/inttypes.h with the contents:
+      #include &lt;sys/types.h&gt;
+  o Put rdiff-backup in your PATH, as you normally would.
+
+</pre>
+
+Then, whenever you use rdiff-backup (or at least if you are backing up
+to or restoring from a Windows system), use the
+<strong>--windows-time-format</strong> switch, which will tell
+rdiff-backup not to put a colon (":") in a filename (this option was
+added after Jason posted his message).  Finally, as Michael Muegel
+points out, you have to exclude all files from the source directory
+which have colons in them, so add something like the --exclude ".*:.*"
+option.  In the near future some quoting facility may be added to deal
+with these issues.
+</li>
+
+<P>
+<a name="remove_dir">
+<li><strong>My backup set contains some files that I just realized I
+don't want/need backed up.  How do I remove them from the backup
+volume to save space?</strong>
+
+<P>Let's take an example.  Suppose you ran
+<pre>rdiff-backup /usr /backup</pre>
+and now realize that you don't want /usr/local backed up on /backup.
+Next time you back up, you run
+<pre>rdiff-backup --exclude /usr/local /usr /backup</pre>
+so that /usr/local is no longer copied to /backup/usr/local.
+
+However, old information about /usr/local is still present in
+/backup/rdiff-backup-data/increments/usr/local.  You could wait for
+this information to expire and then run rdiff-backup with the
+--remove-older-than option, or you could remove the increments
+manually by typing:
+<pre>rm -rf /backup/rdiff-backup-data/increments/usr/local
+rm /backup/rdiff-backup-data/increments/usr/local.*.dir</pre>
+
+</li>
+
+<P>
+<a name="redhat">
+<li><strong>How do I install the RPMs on a Redhat linux system?</strong>
+
+<P>The problem is that the default version of python for Redhat 7.x is
+1.5.x, and rdiff-backup requires python >= 2.2.  Redhat/rawhide
+provides python 2.2 RPMs, but they are packaged under the "python2"
+name.
+
+<P>So, if you are running Redhat 7.x:
+
+<ol>
+<li>Make sure the python2 >= 2.2 package is installed,
+leaving python 1.5 the way it is
+<li>Install the rdiff-backup RPM, using --nodeps if it only complains
+    about python 2.2 missing.
+<li>Edit the first line of /usr/bin/rdiff-backup so it says<pre>
+#!/usr/bin/env python2
+</pre>
+so "python2" gets run instead of "python".
+</ol>
+
+<P>You can also upgrade using a non-Redhat python 2.2 RPM and avoid
+the above steps (this is what I did).  Because of all the dependencies
+it is usually easier to use source RPMs for this.
+</li>
+
+<P>
+<a name="solaris">
+<li><strong>Does rdiff-backup work under Solaris?</strong>
+
+<P>There may be a problem with rdiff-backup and Solaris' libthread.
+Adding "ulimit -n unlimited" may fix the problem though.  Here is a
+post by Kevin Spicer on the subject:
+
+<pre>
+Subject: RE: Crash report....still not^H^H^H working
+From: "Spicer, Kevin" <Kevin.Spicer@bmrb.co.uk>
+Date: Sat, 11 May 2002 23:36:42 +0100
+To: rdiff-backup@keywest.Stanford.EDU
+
+Quick mail to follow up on this.. 
+My rdiff backup (on Solaris 2.6 if you remember) has now worked
+reliably for nearly two weeks after I added...
+
+    ulimit -n unlimited 
+
+to the start of my cron job and created a wrapper script on the remote
+machine which looked like this...
+
+    #!/bin/sh 
+    ulimit -n unlimited 
+    rdiff-backup --server 
+    exit 
+
+And changed the remote schema on the command line of rdiff-backup to
+call the wrapper script rather than rdiff-backup itself on the remote
+machine.  As for the /dev/zero thing I've done a bit of Googleing and
+it seems that /dev/zero is used internally by libthread on Solaris
+(which doesn't really explain why its opening more than 64 files - but
+at least I think I've now got round it).
+</pre>
+</li>
+
+</ol>

tags/r0-9-2/rdiff-backup/README

+INSTALLATION:
+
+Thank you for trying rdiff-backup.  To install, run:
+
+	python setup.py install
+
+The default prefix is /usr, so files are put in /usr/bin,
+/usr/share/man/, etc.  An alternate prefix can be specified using the
+--prefix=<prefix> option.  For example:
+
+	python setup.py install --prefix=/usr/local
+
+
+REQUIREMENTS:
+
+Remember that you must have Python 2.2 or later and librsync 0.9.5.1
+or later installed.  To download, see http://www.python.org and
+http://sourceforge.net/projects/librsync/ respectively.
+
+For remote operation, rdiff-backup should be in installed and in the
+PATH on remote system(s) (see man page for more information).
+
+
+TROUBLESHOOTING:
+
+If you have everything installed properly, and it still doesn't work,
+see the enclosed FAQ.html, the web page at
+http://rdiff-backup.stanford.edu, and/or the mailing list.
+

tags/r0-9-2/rdiff-backup/TODO

+Fix bug with -m/--mirror-only switch and write test case
+
+Write some better selection test cases to test new Iterate_fast func.
+
+Considering rewriting Selection Iterate(_fast) code again.
+
+Fix remaining memory leaks in cmodule.c
+
+
+---------[ Long term ]---------------------------------------
+Security audit
+
+--read-only and --write-only /usr/foo switches to tighten security up some.
+
+
+Think about adding Gaudet's idea for keeping track of renamed files.
+
+
+
+If don't recover hardlink support and hardlink support on, don't resume.
+
+
+

tags/r0-9-2/rdiff-backup/dist/makedist

+#!/usr/bin/env python
+
+import os, re, shutil, time, sys
+
+SourceDir = "src"
+DistDir = "dist"
+
+# Various details about the files must also be specified by the rpm
+# spec template.
+spec_template = "dist/rdiff-backup.spec"
+#redhat_spec_template = "dist/rdiff-backup.rh7x.spec"
+
+def CopyMan(destination, version):
+	"""Create updated man page at the specified location"""
+	fp = open(destination, "w")
+	date = time.strftime("%B %Y", time.localtime(time.time()))
+	version = "Version "+version
+	firstline = ('.TH RDIFF-BACKUP 1 "%s" "%s" "User Manuals"\n' %
+				 (date, version))
+	fp.write(firstline)
+	infp = open("rdiff-backup.1", "r")
+	infp.readline()
+	fp.write(infp.read())
+	fp.close()
+	infp.close()
+
+def MakeFAQ():
+	"""Create FAQ.html and FAQ.wml files from FAQ-body.html"""
+	faqbody_fp = open("FAQ-body.html", "r")
+	faqbody_string = faqbody_fp.read()
+	faqbody_fp.close()
+
+	wml_fp = open("FAQ.wml", "w")
+	wml_fp.write(
+"""#include 'template.wml' curpage=faq title="rdiff-backup: FAQ"
+
+<divert body>
+<p><h2>FAQ:</h2>
+
+""")
+	wml_fp.write(faqbody_string)
+	wml_fp.write("\n</divert>\n")
+	wml_fp.close()
+
+	html_fp = open("FAQ.html", "w")
+	html_fp.write(
+"""<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+
+<html>
+  <head>
+    <title>rdiff-backup FAQ</title>
+  </head>
+
+  <body>
+  <h1>rdiff-backup FAQ</h1>
+""")
+	html_fp.write(faqbody_string)
+	html_fp.write("\n</body></html>")
+	html_fp.close()
+
+def VersionedCopy(source, dest):
+	"""Copy source to dest, substituting $version with version"""
+	fin = open(source, "rb")
+	inbuf = fin.read()
+	assert not fin.close()
+
+	outbuf = re.sub("\$version", Version, inbuf, 1)
+	if outbuf == inbuf: assert 0, "No $version string replaced"
+	assert not re.search("\$version", outbuf), \
+		   "Two $version strings found in the same file %s" % (source,)
+
+	fout = open(dest, "wb")
+	fout.write(outbuf)
+	assert not fout.close()
+
+def MakeTar():
+	"""Create rdiff-backup tar file"""
+	tardir = "rdiff-backup-%s" % Version
+	tarfile = "rdiff-backup-%s.tar.gz" % Version
+	try:
+		os.lstat(tardir)
+		os.system("rm -rf " + tardir)
+	except OSError: pass
+	os.mkdir(tardir)
+	for filename in ["CHANGELOG", "COPYING", "README", "FAQ.html",
+					 SourceDir + "/cmodule.c",
+					 SourceDir + "/_librsyncmodule.c",
+					 DistDir + "/setup.py"]:
+		assert not os.system("cp %s %s" % (filename, tardir)), filename
+
+	os.mkdir(tardir+"/rdiff_backup")
+	for filename in ["connection.py",	"destructive_stepping.py",
+					 "FilenameMapping.py", "Hardlink.py",
+					 "highlevel.py", "increment.py", "__init__.py",
+					 "iterfile.py", "lazy.py", "librsync.py",
+					 "log.py", "Main.py", "manage.py", "MiscStats.py",
+					 "Rdiff.py", "restore.py", "rlist.py",
+					 "robust.py", "rorpiter.py", "rpath.py",
+					 "selection.py", "SetConnections.py", "static.py",
+					 "statistics.py", "Time.py"]:
+		assert not os.system("cp %s/%s %s/rdiff_backup" %
+							 (SourceDir, filename, tardir)), filename
+
+	VersionedCopy("%s/Globals.py" % (SourceDir,),
+				  "%s/rdiff_backup/Globals.py" % (tardir,))
+	VersionedCopy("rdiff-backup", "%s/rdiff-backup" % (tardir,))
+	VersionedCopy(DistDir + "/setup.py", "%s/setup.py" % (tardir,))
+
+	os.chmod(os.path.join(tardir, "setup.py"), 0755)
+	os.chmod(os.path.join(tardir, "rdiff-backup"), 0755)
+	CopyMan(os.path.join(tardir, "rdiff-backup.1"), Version)
+	os.system("tar -cvzf %s %s" % (tarfile, tardir))
+	shutil.rmtree(tardir)
+	return tarfile
+
+def MakeSpecFile():
+	"""Create spec file using spec template"""
+	specfile = "rdiff-backup-%s-1.spec" % Version
+	VersionedCopy(spec_template, specfile)
+	return specfile
+
+def Main():
+	print "Making FAQ"
+	MakeFAQ()
+	print "Processing version " + Version
+	tarfile = MakeTar()
+	print "Made tar file " + tarfile
+	specfile = MakeSpecFile()
+	print "Made specfile ", specfile
+
+if __name__ == "__main__" and not globals().has_key('__no_execute__'):
+	if len(sys.argv) != 2:
+		print "Syntax: makedist [version_number]"
+		sys.exit(1)
+	Version = sys.argv[1]
+	Main()
+

tags/r0-9-2/rdiff-backup/dist/makerpm

+#!/usr/bin/env python
+
+import os, sys, re
+
+SourceDir = "src"
+
+if len(sys.argv) == 2:
+	version = sys.argv[1]
+	specfile = "rdiff-backup-%s-1.spec" % version
+	print "Using specfile %s" % specfile
+else:
+	print "Syntax:  %s version_number" % sys.argv[0]
+	sys.exit(1)
+
+base = ".".join(specfile.split(".")[:-1])
+srcrpm = base+".src.rpm"
+i386rpm = base+".i386.rpm"
+source_rpm = base+".src.rpm"
+tarfile = "-".join(base.split("-")[:-1]) + ".tar.gz"
+
+assert not os.system("install -o root -g root -m 644 %s "
+					 "/usr/src/redhat/SOURCES" % (tarfile,))
+assert not os.system("rpm -ba --sign -vv --target i386 " + specfile)
+assert not os.system("install -o ben -g ben -m 644 "
+					 "/usr/src/redhat/RPMS/i386/%s ." % i386rpm)
+assert not os.system("install -o ben -g ben -m 644 "
+					 "/usr/src/redhat/SRPMS/%s ." % source_rpm)

tags/r0-9-2/rdiff-backup/dist/makeweb

+#!/usr/bin/env python
+
+import sys, os
+
+def RunCommand(cmd):
+	print cmd
+	os.system(cmd)
+
+if not sys.argv[1:]:
+	print 'Call with version number, as in "./makeweb 0.3.1"'
+	sys.exit(1)
+
+version = sys.argv[1]
+webprefix = "/home/ben/misc/html/mirror/rdiff-backup/"
+
+
+RunCommand("cp *%s* %s" % (version, webprefix))
+RunCommand("rman -f html -r '' rdiff-backup.1 > %srdiff-backup.1.html"
+		   % webprefix)
+RunCommand("cp FAQ.wml CHANGELOG src/rdiff-backup %s" % webprefix)
+
+
+os.chdir(webprefix)
+print "cd ", webprefix
+RunCommand("rm latest latest.rpm latest.tar.gz")
+RunCommand("ln -s *%s*rpm latest.rpm" % (version,))
+RunCommand("ln -s *%s*tar.gz latest.tar.gz" % (version,))
+RunCommand("./Make")

tags/r0-9-2/rdiff-backup/dist/rdiff-backup-rh7x.patch

+--- rdiff-backup.old	Sat Apr  6 10:05:18 2002
++++ rdiff-backup	Sat Apr  6 10:05:25 2002
+@@ -1,4 +1,4 @@
+-#!/usr/bin/env python
++#!/usr/bin/env python2
+ #
+ # rdiff-backup -- Mirror files while keeping incremental changes
+ # Version 0.7.1 released March 25, 2002

tags/r0-9-2/rdiff-backup/dist/rdiff-backup.rh7x.spec

+Summary: Convenient and transparent local/remote incremental mirror/backup
+Name: rdiff-backup
+Release: 1
+URL: http://www.stanford.edu/~bescoto/rdiff-backup/
+Source: %{name}-%{version}.tar.gz
+Copyright: GPL
+Group: Applications/Archiving
+BuildRoot: %{_tmppath}/%{name}-root
+requires: librsync, python2 >= 2.2
+Patch: rdiff-backup-rh7x.patch
+
+%description
+rdiff-backup is a script, written in Python, that backs up one
+directory to another and is intended to be run periodically (nightly
+from cron for instance). The target directory ends up a copy of the
+source directory, but extra reverse diffs are stored in the target
+directory, so you can still recover files lost some time ago. The idea
+is to combine the best features of a mirror and an incremental
+backup. rdiff-backup can also operate in a bandwidth efficient manner
+over a pipe, like rsync. Thus you can use rdiff-backup and ssh to
+securely back a hard drive up to a remote location, and only the
+differences from the previous backup will be transmitted.
+
+%prep
+%setup
+%patch
+%build
+
+%install
+rm -rf $RPM_BUILD_ROOT
+mkdir -p $RPM_BUILD_ROOT/usr/bin
+mkdir -p $RPM_BUILD_ROOT/usr/share/man/man1
+
+install -m 755 rdiff-backup $RPM_BUILD_ROOT/usr/bin/rdiff-backup
+install -m 644 rdiff-backup.1 $RPM_BUILD_ROOT/usr/share/man/man1/rdiff-backup.1
+%clean
+
+%files
+%defattr(-,root,root)
+
+/usr/bin/rdiff-backup
+/usr/share/man/man1/rdiff-backup.1.gz
+%doc CHANGELOG COPYING README FAQ.html
+
+%changelog
+* Sat Apr 6 2002 Ben Escoto <bescoto@stanford.edu>
+- Made new version for Redhat 7.x series
+* Sun Nov 4 2001 Ben Escoto <bescoto@stanford.edu>
+- Initial RPM
+

tags/r0-9-2/rdiff-backup/dist/rdiff-backup.spec

+Version: $version
+Summary: Convenient and transparent local/remote incremental mirror/backup
+Name: rdiff-backup
+Release: 1
+URL: http://www.stanford.edu/~bescoto/rdiff-backup/
+Source: %{name}-%{version}.tar.gz
+Copyright: GPL
+Group: Applications/Archiving
+BuildRoot: %{_tmppath}/%{name}-root
+requires: librsync >= 0.9.5.1, python2 >= 2.2
+BuildPrereq: python2-devel >= 2.2
+
+%description
+rdiff-backup is a script, written in Python, that backs up one
+directory to another and is intended to be run periodically (nightly
+from cron for instance). The target directory ends up a copy of the
+source directory, but extra reverse diffs are stored in the target
+directory, so you can still recover files lost some time ago. The idea
+is to combine the best features of a mirror and an incremental
+backup. rdiff-backup can also operate in a bandwidth efficient manner
+over a pipe, like rsync. Thus you can use rdiff-backup and ssh to
+securely back a hard drive up to a remote location, and only the
+differences from the previous backup will be transmitted.
+
+%prep
+%setup
+
+%build
+python2 setup.py build
+
+%install
+python2 setup.py install --prefix=$RPM_BUILD_ROOT/usr
+%clean
+
+%files
+%defattr(-,root,root)
+/usr/bin/rdiff-backup
+/usr/share/doc/rdiff-backup-%{version}
+/usr/lib
+
+
+%changelog
+* Sun Nov 4 2001 Ben Escoto <bescoto@stanford.edu>
+- Initial RPM
+

tags/r0-9-2/rdiff-backup/dist/setup.py

+#!/usr/bin/env python
+
+import sys, os, getopt
+from distutils.core import setup, Extension
+
+version_string = "$version"
+
+if sys.version_info[:2] < (2,2):
+	print "Sorry, rdiff-backup requires version 2.2 or later of python"
+	sys.exit(1)
+
+setup(name="rdiff-backup",
+	  version=version_string,
+	  description="Local/remote mirroring+incremental backup",
+	  author="Ben Escoto",
+	  author_email="bescoto@stanford.edu",
+	  url="http://rdiff-backup.stanford.edu",
+	  packages = ['rdiff_backup'],
+	  ext_modules = [Extension("rdiff_backup.C", ["cmodule.c"]),
+					 Extension("rdiff_backup._librsync",
+							   ["_librsyncmodule.c"],
+							   libraries=["rsync"])],
+	  scripts = ['rdiff-backup'],
+	  data_files = [('share/man/man1', ['rdiff-backup.1']),
+					('share/doc/rdiff-backup-%s' % version_string,
+					 ['CHANGELOG', 'COPYING', 'README', 'FAQ.html'])])
+
+
+

tags/r0-9-2/rdiff-backup/misc/compress-rdiff-backup-increments

+#!/usr/bin/env python
+#
+# Compresses old rdiff-backup increments.  See
+# http://www.stanford.edu/~bescoto/rdiff-backup for information on
+# rdiff-backup.
+
+from __future__ import nested_scopes, generators
+import os, sys, getopt
+
+rdiff_backup_location = "/usr/bin/rdiff-backup"
+no_compression_regexp_string = None
+__no_execute__ = 1
+
+def print_help():
+	"""Print usage, exit"""
+	print """
+Usage: compress-rdiff-backup-increments [options] mirror_directory
+
+This script will compress the old rdiff-backup increments under
+mirror_directory, in a format compatible with rdiff-backup version
+0.7.1 and later.  So for instance if you were using an old version
+of rdiff-backup like this:
+
+rdiff-backup /foo /backup
+
+and now you want to take advantage of v0.7.1's space saving
+compression, you can run:
+
+compress-rdiff-backup-increments /backup
+
+
+Options:
+
+--rdiff-backup-location location
+    This script reads your rdiff-backup executable.  The default
+    is "/usr/bin/rdiff-backup", so if your rdiff-backup is in a
+    different location, you must use this switch.
+
+--no-compression-regexp regexp
+    Any increments whose base name match this regular expression
+    won't be compressed.  This is generally used to avoid
+    compressing already compressed files.  See the rdiff-backup
+    man page for the default.
+"""
+	sys.exit(1)
+	
+def parse_args(arglist):
+	"""Check and evaluate command line arguments, return dirname"""
+	global rdiff_backup_location
+	global no_compression_regexp_string
+	try: optlist, args = getopt.getopt(arglist, "v:",
+									   ["rdiff-backup-location=",
+										"no-compression-regexp="])
+	except getopt.error: print_help()
+
+	for opt, arg in optlist:
+		if opt == "--no-compression-regexp":
+			no_compression_regexp_string = arg
+		elif opt == "--rdiff-backup-location": rdiff_backup_location = arg
+		else:
+			print "Bad option: ", opt
+			print_help()
+
+	if len(args) != 1:
+		print "Wrong number of arguments"
+		print_help()
+	return args[0]
+
+def exec_rdiff_backup():
+	"""Execs rdiff-backup"""
+	try: execfile(rdiff_backup_location, globals())
+	except IOError:
+		print "Unable to read", rdiff_backup_location
+		print "You may need to use the --rdiff-backup-location argument"
+		sys.exit(1)
+
+	if not map(int, Globals.version.split(".")) >= [0, 7, 1]:
+		print "This script requires rdiff-backup version 0.7.1 or later,",
+		print "found version", Globals.version
+		sys.exit(1)
+
+def gzip_file(rp):
+	"""gzip rp, adding .gz to path and deleting original"""
+	newrp = RPath(rp.conn, rp.base, rp.index[:-1] + (rp.index[-1]+".gz",))
+	if newrp.lstat():
+		print "Warning: %s already exists, skipping" % newrp.path
+		return
+
+	print "gzipping ", rp.path
+	newrp.write_from_fileobj(rp.open("rb"), compress = 1)
+	RPath.copy_attribs(rp, newrp)
+	rp.delete()
+
+def Main():
+	dirname = parse_args(sys.argv[1:])
+	exec_rdiff_backup()
+	if no_compression_regexp_string is not None:
+		no_compression_regexp = re.compile(no_compression_regexp_string, re.I)
+	else: no_compression_regexp = \
+		  re.compile(Globals.no_compression_regexp_string, re.I)
+	Globals.change_source_perms = 1
+	Globals.change_ownership = (os.getuid() == 0)
+
+	# Check to make sure rbdir exists
+	root_rp = RPath(Globals.local_connection, dirname)
+	rbdir = root_rp.append("rdiff-backup-data")
+	if not rbdir.lstat():
+		print "Cannot find %s, exiting" % rbdir.path
+		sys.exit(1)
+
+	for dsrp in DestructiveStepping.Iterate_with_Finalizer(rbdir, 1):
+		if (dsrp.isincfile() and dsrp.isreg() and
+			not dsrp.isinccompressed() and
+			(dsrp.getinctype() == "diff" or dsrp.getinctype() == "snapshot")
+			and dsrp.getsize() != 0 and
+			not no_compression_regexp.match(dsrp.getincbase_str())):
+			gzip_file(dsrp)
+
+Main()
+

tags/r0-9-2/rdiff-backup/misc/find2dirs

+#!/usr/bin/env python
+
+from __future__ import generators
+import sys, os, stat
+
+def usage():
+	print "Usage: find2dirs dir1 dir2"
+	print
+	print "Given the name of two directories, list all the files in both, one"
+	print "per line, but don't repeat a file even if it is in both directories"
+	sys.exit(1)
+
+def getlist(base, ext = ""):
+	"""Return iterator yielding filenames from directory"""
+	if ext: yield ext
+	else: yield "."
+
+	fullname = os.path.join(base, ext)
+	if stat.S_ISDIR(stat.S_IFMT(os.lstat(fullname)[stat.ST_MODE])):
+		for subfile in os.listdir(fullname):
+			for fn in getlist(base, os.path.join(ext, subfile)): yield fn
+
+def main(dir1, dir2):
+	d = {}
+	for fn in getlist(dir1): d[fn] = 1
+	for fn in getlist(dir2): d[fn] = 1
+	for fn in d.keys(): print fn
+
+if not len(sys.argv) == 3: usage()
+else: main(sys.argv[1], sys.argv[2])

tags/r0-9-2/rdiff-backup/misc/init_files.py

+#!/usr/bin/env python
+
+"""init_smallfiles.py
+
+This program makes a number of files of the given size in the
+specified directory.
+
+"""
+
+import os, stat, sys, math
+
+if len(sys.argv) > 5 or len(sys.argv) < 4:
+	print "Usage: init_files [directory name] [file size] [file count] [base]"
+	print
+	print "Creates file_count files in directory_name of size file_size."
+	print "The created directory has a tree type structure where each level"
+	print "has at most base files or directories in it.  Default is 50."
+	sys.exit(1)
+
+dirname = sys.argv[1]
+filesize = int(sys.argv[2])
+filecount = int(sys.argv[3])
+block_size = 16384
+block = "." * block_size
+block_change = "." * (filesize % block_size)
+if len(sys.argv) == 4: base = 50
+else: base = int(sys.argv[4])
+
+def make_file(path):
+	"""Make the file at path"""
+	fp = open(path, "w")
+	for i in xrange(int(math.floor(filesize/block_size))): fp.write(block)
+	fp.write(block_change)
+	fp.close()
+
+def find_sublevels(count):
+	"""Return number of sublevels required for count files"""
+	return int(math.ceil(math.log(count)/math.log(base)))
+
+def make_dir(dir, count):
+	"""Make count files in the directory, making subdirectories if necessary"""
+	print "Making directory %s with %d files" % (dir, count)
+	os.mkdir(dir)
+	level = find_sublevels(count)
+	assert count <= pow(base, level)
+	if level == 1:
+		for i in range(count): make_file(os.path.join(dir, "file%d" %i))
+	else:
+		files_per_subdir = pow(base, level-1)
+		full_dirs = int(count/files_per_subdir)
+		assert full_dirs <= base
+		for i in range(full_dirs):
+			make_dir(os.path.join(dir, "subdir%d" % i), files_per_subdir)
+
+		change = count - full_dirs*files_per_subdir
+		assert change >= 0
+		if change > 0:
+			make_dir(os.path.join(dir, "subdir%d" % full_dirs), change)
+
+def start(dir):
+	try: os.stat(dir)
+	except os.error: pass
+	else:
+		print "Directory %s already exists, exiting." % dir
+		sys.exit(1)
+
+	make_dir(dirname, filecount)
+
+start(dirname)

tags/r0-9-2/rdiff-backup/misc/myrm

+#!/usr/bin/python
+
+import sys, os
+sys.path.insert(0, "../src")
+from rpath import *
+from connection import *
+import Globals
+
+lc = Globals.local_connection
+
+for filename in sys.argv[1:]:
+	#print "Deleting %s" % filename
+	rp = RPath(lc, filename)
+	if rp.lstat(): rp.delete()
+	#os.system("rm -rf " + rp.path)

tags/r0-9-2/rdiff-backup/misc/remove-comments.py

+#!/usr/bin/env python
+
+"""remove-comments.py
+
+Given a python program on standard input, spit one out on stdout that
+should work the same, but has blank and comment lines removed.
+
+"""
+
+import sys, re
+
+triple_regex = re.compile('"""')
+
+def eattriple(initial_line_stripped):
+	"""Keep reading until end of doc string"""
+	assert initial_line_stripped.startswith('"""')
+	if triple_regex.search(initial_line_stripped[3:]): return
+	while 1:
+		line = sys.stdin.readline()
+		if not line or triple_regex.search(line): break		
+
+while 1:
+	line = sys.stdin.readline()
+	if not line: break
+	stripped = line.strip()
+	if not stripped: continue
+	if stripped[0] == "#": continue
+	if stripped.startswith('"""'):
+		eattriple(stripped)
+		continue
+	sys.stdout.write(line)
+	

tags/r0-9-2/rdiff-backup/rdiff-backup

+#!/usr/bin/env python
+# rdiff-backup -- Mirror files while keeping incremental changes
+# Version $version released June 20, 2002
+# Copyright (C) 2001, 2002 Ben Escoto <bescoto@stanford.edu>
+#
+# This program is licensed under the GNU General Public License (GPL).
+# you can redistribute it and/or modify it under the terms of the GNU
+# General Public License as published by the Free Software Foundation,
+# Inc., 675 Mass Ave, Cambridge MA 02139, USA; either version 2 of the
+# License, or (at your option) any later version.  Distributions of
+# rdiff-backup should include a copy of the GPL in a file called
+# COPYING.  The GPL is also available online at
+# http://www.gnu.org/copyleft/gpl.html.
+#
+# See http://rdiff-backup.stanford.edu/ for more information.  Please
+# send mail to me or the mailing list if you find bugs or have any
+# suggestions.
+
+import sys
+import rdiff_backup.Main
+
+if __name__ == "__main__" and not globals().has_key('__no_execute__'):
+	rdiff_backup.Main.Main(sys.argv[1:])
+
+

tags/r0-9-2/rdiff-backup/rdiff_backup/FilenameMapping.py

+# Copyright 2002 Ben Escoto
+#
+# This file is part of rdiff-backup.
+#
+# rdiff-backup is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA
+# 02139, USA; either version 2 of the License, or (at your option) any
+# later version; incorporated herein by reference.
+
+"""Coordinate corresponding files with different names
+
+For instance, some source filenames may contain characters not allowed
+on the mirror end.  Also, if a source filename is very long (say 240
+characters), the extra characters added to related increments may put
+them over the usual 255 character limit.
+
+"""
+
+import re
+from log import *
+import Globals
+
+
+max_filename_length = 255
+
+# If true, enable character quoting, and set characters making
+# regex-style range.
+chars_to_quote = None
+
+# These compiled regular expressions are used in quoting and unquoting
+chars_to_quote_regexp = None
+unquoting_regexp = None
+
+# Use given char to quote.  Default is set in Globals.
+quoting_char = None
+
+
+def set_init_quote_vals():
+	"""Set quoting value from Globals on all conns"""
+	for conn in Globals.connections:
+		conn.FilenameMapping.set_init_quote_vals_local()
+
+def set_init_quote_vals_local():
+	"""Set value on local connection, initialize regexps"""
+	global chars_to_quote
+	chars_to_quote = Globals.chars_to_quote
+	if len(Globals.quoting_char) != 1:
+		Log.FatalError("Expected single character for quoting char,"
+					   "got '%s' instead" % (Globals.quoting_char,))
+	quoting_char = Globals.quoting_char
+	init_quoting_regexps()
+
+def init_quoting_regexps():
+	"""Compile quoting regular expressions"""
+	global chars_to_quote_regexp, unquoting_regexp
+	try:
+		chars_to_quote_regexp = \
+				 re.compile("[%s%s]" % (chars_to_quote, quoting_char), re.S)
+		unquoting_regexp = re.compile("%s[0-9]{3}" % quoting_char, re.S)
+	except re.error:
+		Log.FatalError("Error '%s' when processing char quote list %s" %
+					   (re.error, chars_to_quote))
+
+def quote(path):
+	"""Return quoted version of given path
+
+	Any characters quoted will be replaced by the quoting char and
+	the ascii number of the character.  For instance, "10:11:12"
+	would go to "10;05811;05812" if ":" were quoted and ";" were
+	the quoting character.
+
+	"""
+	return chars_to_quote_regexp.sub(quote_single, path)
+
+def quote_single(match):
+	"""Return replacement for a single character"""
+	return "%s%03d" % (quoting_char, ord(match.group()))
+
+def unquote(path):
+	"""Return original version of quoted filename"""
+	return unquoting_regexp.sub(unquote_single, path)
+
+def unquote_single(match):
+	"""Unquote a single quoted character"""
+	assert len(match.group()) == 4
+	return chr(int(match.group()[1:]))
+
+def get_quoted_dir_children(rpath):
+	"""For rpath directory, return list of quoted children in dir"""
+	if not rpath.isdir(): return []
+	dir_pairs = [(unquote(filename), filename)
+				 for filename in Robust.listrp(rpath)]
+	dir_pairs.sort() # sort by real index, not quoted part
+	child_list = []
+	for unquoted, filename in dir_pairs:
+		childrp = rpath.append(unquoted)
+		childrp.quote_path()
+		child_list.append(childrp)
+	return child_list
+
+
+

tags/r0-9-2/rdiff-backup/rdiff_backup/Globals.py

+# Copyright 2002 Ben Escoto
+#
+# This file is part of rdiff-backup.
+#
+# rdiff-backup is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA
+# 02139, USA; either version 2 of the License, or (at your option) any
+# later version; incorporated herein by reference.
+
+"""Hold a variety of constants usually set at initialization."""
+
+import re, os
+
+
+# The current version of rdiff-backup
+version = "$version"
+
+# If this is set, use this value in seconds as the current time
+# instead of reading it from the clock.
+current_time = None
+
+# This determines how many bytes to read at a time when copying
+blocksize = 32768
+
+# This is used by the BufferedRead class to determine how many
+# bytes to request from the underlying file per read().  Larger
+# values may save on connection overhead and latency.
+conn_bufsize = 98304
+
+# True if script is running as a server
+server = None
+
+# uid and gid of the owner of the rdiff-backup process.  This can
+# vary depending on the connection.
+process_uid = os.getuid()
+process_gid = os.getgid()
+
+# If true, when copying attributes, also change target's uid/gid
+change_ownership = None
+
+# If true, change the permissions of unwriteable mirror files
+# (such as directories) so that they can be written, and then
+# change them back.  This defaults to 1 just in case the process
+# is not running as root (root doesn't need to change
+# permissions).
+change_mirror_perms = (process_uid != 0)
+
+# If true, temporarily change permissions of unreadable files in
+# the source directory to make sure we can read all files.
+change_source_perms = None
+
+# If true, try to reset the atimes of the source partition.
+preserve_atime = None
+
+# This will be set as soon as the LocalConnection class loads
+local_connection = None
+
+# All connections should be added to the following list, so
+# further global changes can be propagated to the remote systems.
+# The first element should be Globals.local_connection.  For a
+# server, the second is the connection to the client.
+connections = []
+
+# Each process should have a connection number unique to the
+# session.  The client has connection number 0.
+connection_number = 0
+
+# Dictionary pairing connection numbers with connections.  Set in
+# SetConnections for all connections.
+connection_dict = {}
+
+# True if the script is the end that reads the source directory
+# for backups.  It is true for purely local sessions.
+isbackup_reader = None
+
+# Connection of the real backup reader (for which isbackup_reader
+# is true)
+backup_reader = None
+
+# True if the script is the end that writes to the increment and
+# mirror directories.  True for purely local sessions.
+isbackup_writer = None
+
+# Connection of the backup writer
+backup_writer = None
+
+# True if this process is the client invoked by the user
+isclient = None
+
+# Connection of the client
+client_conn = None
+
+# This list is used by the set function below.  When a new
+# connection is created with init_connection, its Globals class
+# will match this one for all the variables mentioned in this
+# list.
+changed_settings = []
+
+# rdiff-backup will try to checkpoint its state every
+# checkpoint_interval seconds.  Then when resuming, at most this
+# amount of time is lost.
+checkpoint_interval = 20
+
+# The RPath of the rdiff-backup-data directory.
+rbdir = None
+
+# Indicates if a resume or a lack of resume is forced.  This
+# should be None for the default.  0 means don't resume, and 1
+# means resume.
+resume = None
+
+# If there has been an aborted backup fewer than this many seconds
+# ago, attempt to resume it where it left off instead of starting
+# a new one.
+resume_window = 7200
+
+# This string is used when recognizing and creating time strings.
+# If the time_separator is ":", then W3 datetime strings like
+# 2001-12-07T04:22:01-07:00 are produced.  It can be set to "_" to
+# make filenames that don't contain colons, which aren't allowed
+# under MS windows NT.
+time_separator = ":"
+
+# quoting_enabled is true if we should quote certain characters in
+# filenames on the source side (see FilenameMapping for more
+# info).  chars_to_quote is a string whose characters should be
+# quoted, and quoting_char is the character to quote with.
+quoting_enabled = None
+chars_to_quote = ""
+quoting_char = ';'
+
+# If true, emit output intended to be easily readable by a
+# computer.  False means output is intended for humans.
+parsable_output = None
+
+# If true, then hardlinks will be preserved to mirror and recorded
+# in the increments directory.  There is also a difference here
+# between None and 0.  When restoring, None or 1 means to preserve
+# hardlinks iff can find a hardlink dictionary.  0 means ignore
+# hardlink information regardless.
+preserve_hardlinks = 1
+
+# If this is false, then rdiff-backup will not compress any
+# increments.  Default is to compress based on regexp below.
+compression = 1
+
+# Increments based on files whose names match this
+# case-insensitive regular expression won't be compressed (applies
+# to .snapshots and .diffs).  The second below will be the
+# compiled version of the first.
+no_compression_regexp_string = "(?i).*\\.(gz|z|bz|bz2|tgz|zip|rpm|deb|" \
+						"jpg|gif|png|jp2|mp3|ogg|avi|wmv|mpeg|mpg|rm|mov)$"
+no_compression_regexp = None
+
+# If true, filelists and directory statistics will be split on
+# nulls instead of newlines.
+null_separator = None
+
+# Determines whether or not ssh will be run with the -C switch
+ssh_compression = 1
+
+# If true, print statistics after successful backup
+print_statistics = None
+
+# On the reader and writer connections, the following will be
+# replaced by the source and mirror Select objects respectively.
+select_source, select_mirror = None, None
+
+# On the backup writer connection, holds the root incrementing branch
+# object.  Access is provided to increment error counts.
+ITRB = None
+
+def get(name):
+	"""Return the value of something in this module"""
+	return globals()[name]
+
+def is_not_None(name):
+	"""Returns true if value is not None"""
+	return globals()[name] is not None
+
+def set(name, val):
+	"""Set the value of something in this module
+
+	Use this instead of writing the values directly if the setting
+	matters to remote sides.  This function updates the
+	changed_settings list, so other connections know to copy the
+	changes.
+
+	"""
+	changed_settings.append(name)
+	globals()[name] = val
+
+def set_integer(name, val):
+	"""Like set, but make sure val is an integer"""
+	try: intval = int(val)
+	except ValueError:
+		Log.FatalError("Variable %s must be set to an integer -\n"
+					   "received %s instead." % (name, val))
+	set(name, intval)
+
+def get_dict_val(name, key):
+	"""Return val from dictionary in this class"""
+	return globals()[name][key]
+
+def set_dict_val(name, key, val):
+	"""Set value for dictionary in this class"""
+	globals()[name][key] = val
+
+def postset_regexp(name, re_string, flags = None):
+	"""Compile re_string on all existing connections, set to name"""
+	for conn in connections:
+		conn.Globals.postset_regexp_local(name, re_string, flags)
+
+def postset_regexp_local(name, re_string, flags):
+	"""Set name to compiled re_string locally"""
+	if flags: globals()[name] = re.compile(re_string, flags)
+	else: globals()[name] = re.compile(re_string)
+
+def set_select(dsrpath, tuplelist, quote_mode, *filelists):
+	"""Initialize select object using tuplelist
+
+	Note that each list in filelists must each be passed as
+	separate arguments, so each is recognized as a file by the
+	connection.  Otherwise we will get an error because a list
+	containing files can't be pickled.
+
+	"""
+	global select_source, select_mirror
+	if dsrpath.source:
+		select_source = Select(dsrpath, quote_mode)
+		select_source.ParseArgs(tuplelist, filelists)
+	else:
+		select_mirror = Select(dsrpath, quote_mode)
+		select_mirror.ParseArgs(tuplelist, filelists)
+
+
+from rpath import * # kludge to avoid circularity - not needed in this module
+from selection import *

tags/r0-9-2/rdiff-backup/rdiff_backup/Hardlink.py

+# Copyright 2002 Ben Escoto
+#
+# This file is part of rdiff-backup.
+#
+# rdiff-backup is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA
+# 02139, USA; either version 2 of the License, or (at your option) any
+# later version; incorporated herein by reference.
+
+"""Preserve and restore hard links
+
+If the preserve_hardlinks option is selected, linked files in the
+source directory will be linked in the mirror directory.  Linked files
+are treated like any other with respect to incrementing, but a
+database of all links will be recorded at each session, so linked
+files can still be restored from the increments.
+
+All these functions are meant to be executed on the destination
+side.  The source side should only transmit inode information.
+
+"""
+
+from __future__ import generators
+import cPickle
+
+
+# In all of these lists of indicies are the values.  The keys in
+# _inode_ ones are (inode, devloc) pairs.
+_src_inode_indicies = {}
+_dest_inode_indicies = {}
+
+# The keys for these two are just indicies.  They share values
+# with the earlier dictionaries.
+_src_index_indicies = {}
+_dest_index_indicies = {}
+
+# When a linked file is restored, its path is added to this dict,
+# so it can be found when later paths being restored are linked to
+# it.
+_restore_index_path = {}
+
+def get_inode_key(rorp):
+	"""Return rorp's key for _inode_ dictionaries"""
+	return (rorp.getinode(), rorp.getdevloc())
+
+def get_indicies(rorp, source):
+	"""Return a list of similarly linked indicies, using rorp's index"""
+	if source: dict = _src_index_indicies
+	else: dict = _dest_index_indicies
+	try: return dict[rorp.index]
+	except KeyError: return []
+
+def add_rorp(rorp, source):
+	"""Process new rorp and update hard link dictionaries
+
+	First enter it into src_inode_indicies.  If we have already
+	seen all the hard links, then we can delete the entry.
+	Everything must stay recorded in src_index_indicies though.
+
+	"""
+	if not rorp.isreg() or rorp.getnumlinks() < 2: return
+
+	if source:
+		inode_dict, index_dict = _src_inode_indicies, _src_index_indicies
+	else: inode_dict, index_dict = _dest_inode_indicies, _dest_index_indicies
+
+	rp_inode_key = get_inode_key(rorp)
+	if inode_dict.has_key(rp_inode_key):
+		index_list = inode_dict[rp_inode_key]
+		index_list.append(rorp.index)
+		if len(index_list) == rorp.getnumlinks():
+			del inode_dict[rp_inode_key]
+	else: # make new entry in both src dicts
+		index_list = [rorp.index]
+		inode_dict[rp_inode_key] = index_list
+	index_dict[rorp.index] = index_list
+
+def add_rorp_iter(iter, source):
+	"""Return new rorp iterator like iter that add_rorp's first"""
+	for rorp in iter:
+		add_rorp(rorp, source)
+		yield rorp
+
+def rorp_eq(src_rorp, dest_rorp):
+	"""Compare hardlinked for equality
+
+	Two files may otherwise seem equal but be hardlinked in
+	different ways.  This function considers them equal enough if
+	they have been hardlinked correctly to the previously seen
+	indicies.
+
+	"""
+	assert src_rorp.index == dest_rorp.index
+	if (not src_rorp.isreg() or not dest_rorp.isreg() or
+		src_rorp.getnumlinks() == dest_rorp.getnumlinks() == 1):
+		return 1 # Hard links don't apply
+
+	src_index_list = get_indicies(src_rorp, 1)
+	dest_index_list = get_indicies(dest_rorp, None)
+
+	# If a list only has one element, then it is only hardlinked
+	# to itself so far, so that is not a genuine difference yet.
+	if not src_index_list or len(src_index_list) == 1:
+		return not dest_index_list or len(dest_index_list) == 1
+	if not dest_index_list or len(dest_index_list) == 1: return None
+
+	# Both index lists exist and are non-empty
+	return src_index_list == dest_index_list # they are always sorted
+
+def islinked(rorp):
+	"""True if rorp's index is already linked to something on src side"""
+	return len(get_indicies(rorp, 1)) >= 2
+
+def restore_link(index, rpath):
+	"""Restores a linked file by linking it
+
+	When restoring, all the hardlink data is already present, and
+	we can only link to something already written.  In either
+	case, add to the _restore_index_path dict, so we know later
+	that the file is available for hard
+	linking.
+
+	Returns true if succeeded in creating rpath, false if must
+	restore rpath normally.
+
+	"""
+	if index not in _src_index_indicies: return None
+	for linked_index in _src_index_indicies[index]:
+		if linked_index in _restore_index_path:
+			srcpath = _restore_index_path[linked_index]
+			Log("Restoring %s by hard linking to %s" %
+				(rpath.path, srcpath), 6)
+			rpath.hardlink(srcpath)
+			return 1
+	_restore_index_path[index] = rpath.path
+	return None
+
+def link_rp(src_rorp, dest_rpath, dest_root = None):
+	"""Make dest_rpath into a link analogous to that of src_rorp"""
+	if not dest_root: dest_root = dest_rpath # use base of dest_rpath
+	dest_link_rpath = RPath(dest_root.conn, dest_root.base,
+							get_indicies(src_rorp, 1)[0])
+	dest_rpath.hardlink(dest_link_rpath.path)
+
+def write_linkdict(rpath, dict, compress = None):
+	"""Write link data to the rbdata dir
+
+	It is stored as the a big pickled dictionary dated to match
+	the current hardlinks.
+
+	"""
+	assert (Globals.isbackup_writer and
+			rpath.conn is Globals.local_connection)
+	tf = TempFileManager.new(rpath)
+	def init():
+		fp = tf.open("wb", compress)
+		cPickle.dump(dict, fp)
+		assert not fp.close()
+		tf.setdata()
+	Robust.make_tf_robustaction(init, (tf,), (rpath,)).execute()
+
+def get_linkrp(data_rpath, time, prefix):
+	"""Return RPath of linkdata, or None if cannot find"""
+	for rp in map(data_rpath.append, data_rpath.listdir()):
+		if (rp.isincfile() and rp.getincbase_str() == prefix and
+			(rp.getinctype() == 'snapshot' or rp.getinctype() == 'data')
+			and Time.stringtotime(rp.getinctime()) == time):
+			return rp
+	return None
+
+def get_linkdata(data_rpath, time, prefix = 'hardlink_data'):
+	"""Return index dictionary written by write_linkdata at time"""
+	rp = get_linkrp(data_rpath, time, prefix)
+	if not rp: return None
+	fp = rp.open("rb", rp.isinccompressed())
+	index_dict = cPickle.load(fp)
+	assert not fp.close()
+	return index_dict
+
+def final_writedata():
+	"""Write final checkpoint data to rbdir after successful backup"""
+	global final_inc
+	if _src_index_indicies:
+		Log("Writing hard link data", 6)
+		if Globals.compression:
+			final_inc = Globals.rbdir.append("hardlink_data.%s.data.gz" %
+											 Time.curtimestr)
+		else: final_inc = Globals.rbdir.append("hardlink_data.%s.data" %
+											   Time.curtimestr)
+		write_linkdict(final_inc, _src_index_indicies, Globals.compression)
+	else: # no hardlinks, so writing unnecessary
+		final_inc = None
+
+def retrieve_final(time):
+	"""Set source index dictionary from hardlink_data file if avail"""
+	global _src_index_indicies
+	hd = get_linkdata(Globals.rbdir, time)
+	if hd is None: return None
+	_src_index_indicies = hd
+	return 1
+
+def final_checkpoint(data_rpath):
+	"""Write contents of the four dictionaries to the data dir
+
+	If rdiff-backup receives a fatal error, it may still be able
+	to save the contents of the four hard link dictionaries.
+	Because these dictionaries may be big, they are not saved
+	after every 20 seconds or whatever, but just at the end.
+
+	"""
+	Log("Writing intermediate hard link data to disk", 2)
+	src_inode_rp = data_rpath.append("hardlink_source_inode_checkpoint."
+									 "%s.data" % Time.curtimestr)
+	src_index_rp = data_rpath.append("hardlink_source_index_checkpoint."
+									 "%s.data" % Time.curtimestr)
+	dest_inode_rp = data_rpath.append("hardlink_dest_inode_checkpoint."
+									  "%s.data" % Time.curtimestr)
+	dest_index_rp = data_rpath.append("hardlink_dest_index_checkpoint."
+									  "%s.data" % Time.curtimestr)
+	for (rp, dict) in ((src_inode_rp, _src_inode_indicies),
+					   (src_index_rp, _src_index_indicies),
+					   (dest_inode_rp, _dest_inode_indicies),
+					   (dest_index_rp, _dest_index_indicies)):
+		write_linkdict(rp, dict)
+
+def retrieve_checkpoint(data_rpath, time):
+	"""Retrieve hardlink data from final checkpoint
+
+	Return true if the retrieval worked, false otherwise.
+
+	"""
+	global _src_inode_indicies, _src_index_indicies
+	global _dest_inode_indicies, _dest_index_indicies
+	try:
+		src_inode = get_linkdata(data_rpath, time,
+								 "hardlink_source_inode_checkpoint")
+		src_index = get_linkdata(data_rpath, time,
+								 "hardlink_source_index_checkpoint")
+		dest_inode = get_linkdata(data_rpath, time,
+								  "hardlink_dest_inode_checkpoint")
+		dest_index = get_linkdata(data_rpath, time,
+								  "hardlink_dest_index_checkpoint")
+	except cPickle.UnpicklingError:
+		Log("Unpickling Error", 2)
+		return None
+	if (src_inode is None or src_index is None or
+		dest_inode is None or dest_index is None): return None
+	_src_inode_indicies, _src_index_indicies = src_inode, src_index
+	_dest_inode_indicies, _dest_index_indicies = dest_inode, dest_index
+	return 1
+
+def remove_all_checkpoints():
+	"""Remove all hardlink checkpoint information from directory"""
+	prefix_list = ["hardlink_source_inode_checkpoint",
+				   "hardlink_source_index_checkpoint",
+				   "hardlink_dest_inode_checkpoint",
+				   "hardlink_dest_index_checkpoint"]
+	for rp in map(Globals.rbdir.append, Globals.rbdir.listdir()):
+		if (rp.isincfile() and rp.getincbase_str() in prefix_list and
+			(rp.getinctype() == 'snapshot' or rp.getinctype() == 'data')):
+			rp.delete()
+
+
+from log import *
+from robust import *
+from rpath import *
+import Globals, Time

tags/r0-9-2/rdiff-backup/rdiff_backup/Make.old

+#!/usr/bin/env python
+
+"""Read component files of rdiff-backup, and glue them together after
+removing unnecessary bits."""
+
+import os
+
+def mystrip(filename):
+	"""Open filename, read input, strip appropriately, and return contents"""
+	fp = open(filename, "r")
+	lines = fp.readlines()
+	fp.close()
+
+	i = 0
+	while(lines[i][:60] !=
+		  "############################################################"):
+		i = i+1
+		
+	return "".join(lines[i:]).strip() + "\n\n\n"
+
+
+
+files = ["globals.py", "static.py", "lazy.py", "log.py", "ttime.py",
+		 "iterfile.py", "rdiff.py", "connection.py", "rpath.py",
+		 "hardlink.py", "robust.py", "rorpiter.py",
+		 "destructive_stepping.py", "selection.py",
+		 "filename_mapping.py", "statistics.py", "increment.py",
+		 "restore.py", "manage.py", "highlevel.py",
+		 "setconnections.py", "main.py"]
+
+os.system("cp header.py rdiff-backup")
+
+outfp = open("rdiff-backup", "a")
+for file in files:
+	outfp.write(mystrip(file))
+outfp.close()
+
+os.system("chmod 755 rdiff-backup")

tags/r0-9-2/rdiff-backup/rdiff_backup/MiscStats.py

+# Copyright 2002 Ben Escoto
+#
+# This file is part of rdiff-backup.
+#
+# rdiff-backup is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA
+# 02139, USA; either version 2 of the License, or (at your option) any
+# later version; incorporated herein by reference.
+
+"""Misc statistics methods, pertaining to dir and session stat files"""
+
+from statistics import *
+
+
+# This is the RPath of the directory statistics file, and the
+# associated open file.  It will hold a line of statistics for
+# each directory that is backed up.
+_dir_stats_rp = None
+_dir_stats_fp = None
+
+# This goes at the beginning of the directory statistics file and
+# explains the format.
+_dir_stats_header = """# rdiff-backup directory statistics file
+#
+# Each line is in the following format:
+# RelativeDirName %s
+""" % " ".join(StatsObj.stat_file_attrs)
+
+def open_dir_stats_file():
+	"""Open directory statistics file, write header"""
+	global _dir_stats_fp, _dir_stats_rp
+	assert not _dir_stats_fp, "Directory file already open"
+
+	if Globals.compression: suffix = "data.gz"
+	else: suffix = "data"
+	_dir_stats_rp = Inc.get_inc(Globals.rbdir.append("directory_statistics"),
+								Time.curtime, suffix)
+
+	if _dir_stats_rp.lstat():
+		Log("Warning, statistics file %s already exists, appending" %
+			_dir_stats_rp.path, 2)
+		_dir_stats_fp = _dir_stats_rp.open("ab", Globals.compression)
+	else: _dir_stats_fp = _dir_stats_rp.open("wb", Globals.compression)
+	_dir_stats_fp.write(_dir_stats_header)
+
+def write_dir_stats_line(statobj, index):
+	"""Write info from statobj about rpath to statistics file"""
+	if Globals.null_separator:
+		_dir_stats_fp.write(statobj.get_stats_line(index, None) + "\0")
+	else: _dir_stats_fp.write(statobj.get_stats_line(index) + "\n")
+
+def close_dir_stats_file():
+	"""Close directory statistics file if its open"""
+	global _dir_stats_fp
+	if _dir_stats_fp:
+		_dir_stats_fp.close()
+		_dir_stats_fp = None
+
+def write_session_statistics(statobj):
+	"""Write session statistics into file, log"""
+	stat_inc = Inc.get_inc(Globals.rbdir.append("session_statistics"),
+						   Time.curtime, "data")
+	statobj.StartTime = Time.curtime
+	statobj.EndTime = time.time()
+
+	# include hardlink data and dir stats in size of increments
+	if Globals.preserve_hardlinks and Hardlink.final_inc:
+		# include hardlink data in size of increments
+		statobj.IncrementFiles += 1
+		statobj.IncrementFileSize += Hardlink.final_inc.getsize()
+	if _dir_stats_rp and _dir_stats_rp.lstat():
+		statobj.IncrementFiles += 1
+		statobj.IncrementFileSize += _dir_stats_rp.getsize()
+
+	statobj.write_stats_to_rp(stat_inc)
+	if Globals.print_statistics:
+		message = statobj.get_stats_logstring("Session statistics")
+		Log.log_to_file(message)
+		Globals.client_conn.sys.stdout.write(message)
+
+
+from increment import *
+import Hardlink

tags/r0-9-2/rdiff-backup/rdiff_backup/Rdiff.py

+# Copyright 2002 Ben Escoto
+#
+# This file is part of rdiff-backup.
+#
+# rdiff-backup is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA
+# 02139, USA; either version 2 of the License, or (at your option) any
+# later version; incorporated herein by reference.
+
+"""Invoke rdiff utility to make signatures, deltas, or patch
+
+All these operations should be done in a relatively safe manner using
+RobustAction and the like.
+
+"""
+
+import os, librsync
+
+
+class RdiffException(Exception): pass
+
+def get_signature(rp):
+	"""Take signature of rpin file and return in file object"""
+	Log("Getting signature of %s" % rp.path, 7)
+	return librsync.SigFile(rp.open("rb"))
+
+def get_delta_sigfileobj(sig_fileobj, rp_new):
+	"""Like get_delta but signature is in a file object"""
+	Log("Getting delta of %s with signature stream" % (rp_new.path,), 7)
+	return librsync.DeltaFile(sig_fileobj, rp_new.open("rb"))
+
+def get_delta_sigrp(rp_signature, rp_new):
+	"""Take signature rp and new rp, return delta file object"""
+	Log("Getting delta of %s with signature %s" %
+		(rp_new.path, rp_signature.path), 7)
+	return librsync.DeltaFile(rp_signature.open("rb"), rp_new.open("rb"))
+
+def write_delta_action(basis, new, delta, compress = None):
+	"""Return action writing delta which brings basis to new
+
+	If compress is true, the output of rdiff will be gzipped
+	before written to delta.
+
+	"""
+	delta_tf = TempFileManager.new(delta)
+	def init(): write_delta(basis, new, delta_tf, compress)
+	return Robust.make_tf_robustaction(init, delta_tf, delta)
+
+def write_delta(basis, new, delta, compress = None):
+	"""Write rdiff delta which brings basis to new"""
+	Log("Writing delta %s from %s -> %s" %
+		(basis.path, new.path, delta.path), 7)
+	sigfile = librsync.SigFile(basis.open("rb"))
+	deltafile = librsync.DeltaFile(sigfile, new.open("rb"))
+	delta.write_from_fileobj(deltafile, compress)
+
+def patch_action(rp_basis, rp_delta, rp_out = None, out_tf = None,
+				 delta_compressed = None):
+	"""Return RobustAction which patches rp_basis with rp_delta
+
+	If rp_out is None, put output in rp_basis.  Will use TempFile
+	out_tf it is specified.  If delta_compressed is true, the
+	delta file will be decompressed before processing with rdiff.
+
+	"""
+	if not rp_out: rp_out = rp_basis
+	if not out_tf: out_tf = TempFileManager.new(rp_out)
+	def init():
+		rp_basis.conn.Rdiff.patch_local(rp_basis, rp_delta,
+										out_tf, delta_compressed)
+		out_tf.setdata()
+	return Robust.make_tf_robustaction(init, out_tf, rp_out)
+
+def patch_local(rp_basis, rp_delta, outrp, delta_compressed = None):
+	"""Patch routine that must be run on rp_basis.conn
+
+	This is because librsync may need to seek() around in rp_basis,
+	and so needs a real file.  Other rpaths can be remote.
+
+	"""
+	assert rp_basis.conn is Globals.local_connection
+	if delta_compressed: deltafile = rp_delta.open("rb", 1)
+	else: deltafile = rp_delta.open("rb")
+
+	sigfile = librsync.SigFile(rp_basis.open("rb"))
+	patchfile = librsync.PatchedFile(rp_basis.open("rb"), deltafile)
+	outrp.write_from_fileobj(patchfile)
+
+def patch_with_attribs_action(rp_basis, rp_delta, rp_out = None):
+	"""Like patch_action, but also transfers attributs from rp_delta"""
+	if not rp_out: rp_out = rp_basis
+	tf = TempFileManager.new(rp_out)
+	return Robust.chain_nested(patch_action(rp_basis, rp_delta, rp_out, tf),
+							   Robust.copy_attribs_action(rp_delta, tf))
+
+def copy_action(rpin, rpout):
+	"""Use rdiff to copy rpin to rpout, conserving bandwidth"""
+	if not rpin.isreg() or not rpout.isreg() or rpin.conn is rpout.conn:
+		# rdiff not applicable, fallback to regular copying
+		return Robust.copy_action(rpin, rpout)
+
+	Log("Rdiff copying %s to %s" % (rpin.path, rpout.path), 6)		
+	out_tf = TempFileManager.new(rpout)
+	def init(): rpout.conn.Rdiff.copy_local(rpin, rpout, out_tf)
+	return Robust.make_tf_robustaction(init, out_tf, rpout)
+
+def copy_local(rpin, rpout, rpnew):
+	"""Write rpnew == rpin using rpout as basis.  rpout and rpnew local"""
+	assert rpnew.conn is rpout.conn is Globals.local_connection
+	sigfile = librsync.SigFile(rpout.open("rb"))
+	deltafile = rpin.conn.librsync.DeltaFile(sigfile, rpin.open("rb"))
+	rpnew.write_from_fileobj(librsync.PatchedFile(rpout.open("rb"), deltafile))
+	
+
+from log import *
+from robust import *
+

tags/r0-9-2/rdiff-backup/rdiff_backup/SetConnections.py

+# Copyright 2002 Ben Escoto
+#
+# This file is part of rdiff-backup.
+#
+# rdiff-backup is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA
+# 02139, USA; either version 2 of the License, or (at your option) any
+# later version; incorporated herein by reference.
+
+"""Parse args and setup connections
+
+The functions in this module are used once by Main to parse file
+descriptions like bescoto@folly.stanford.edu:/usr/bin/ls and to set up
+the related connections.
+
+"""
+
+# This is the schema that determines how rdiff-backup will open a
+# pipe to the remote system.  If the file is given as A::B, %s will
+# be substituted with A in the schema.
+__cmd_schema = 'ssh -C %s rdiff-backup --server'
+__cmd_schema_no_compress = 'ssh %s rdiff-backup --server'
+
+# This is a list of remote commands used to start the connections.
+# The first is None because it is the local connection.
+__conn_remote_cmds = [None]
+
+class SetConnectionsException(Exception): pass
+
+def InitRPs(arglist, remote_schema = None, remote_cmd = None):
+	"""Map the given file descriptions into rpaths and return list"""
+	global __cmd_schema
+	if remote_schema: __cmd_schema = remote_schema
+	elif not Globals.ssh_compression: __cmd_schema = __cmd_schema_no_compress
+
+	if not arglist: return []
+	desc_pairs = map(parse_file_desc, arglist)
+
+	if filter(lambda x: x[0], desc_pairs): # True if any host_info found
+		if remote_cmd:
+			Log.FatalError("The --remote-cmd flag is not compatible "
+						   "with remote file descriptions.")
+	elif remote_schema:
+		Log("Remote schema option ignored - no remote file "
+			"descriptions.", 2)
+
+	cmd_pairs = map(desc2cmd_pairs, desc_pairs)
+	if remote_cmd: # last file description gets remote_cmd
+		cmd_pairs[-1] = (remote_cmd, cmd_pairs[-1][1])
+	return map(cmdpair2rp, cmd_pairs)
+
+def cmdpair2rp(cmd_pair):
+	"""Return RPath from cmd_pair (remote_cmd, filename)"""
+	cmd, filename = cmd_pair
+	if cmd: conn = init_connection(cmd)
+	else: conn = Globals.local_connection
+	return RPath(conn, filename)
+
+def desc2cmd_pairs(desc_pair):
+	"""Return pair (remote_cmd, filename) from desc_pair"""
+	host_info, filename = desc_pair
+	if not host_info: return (None, filename)
+	else: return (fill_schema(host_info), filename)
+
+def parse_file_desc(file_desc):
+	"""Parse file description returning pair (host_info, filename)
+
+	In other words, bescoto@folly.stanford.edu::/usr/bin/ls =>
+	("bescoto@folly.stanford.edu", "/usr/bin/ls").  The
+	complication is to allow for quoting of : by a \.  If the
+	string is not separated by :, then the host_info is None.
+
+	"""
+	def check_len(i):
+		if i >= len(file_desc):
+			raise SetConnectionsException(
+				"Unexpected end to file description %s" % file_desc)
+
+	host_info_list, i, last_was_quoted = [], 0, None
+	while 1:
+		if i == len(file_desc):
+			return (None, file_desc)
+
+		if file_desc[i] == '\\':
+			i = i+1
+			check_len(i)
+			last_was_quoted = 1
+		elif (file_desc[i] == ":" and i > 0 and file_desc[i-1] == ":"
+			  and not last_was_quoted):
+			host_info_list.pop() # Remove last colon from name
+			break
+		else: last_was_quoted = None
+		host_info_list.append(file_desc[i])
+		i = i+1
+
+	check_len(i+1)
+	return ("".join(host_info_list), file_desc[i+1:])
+
+def fill_schema(host_info):
+	"""Fills host_info into the schema and returns remote command"""
+	return __cmd_schema % host_info
+
+def init_connection(remote_cmd):
+	"""Run remote_cmd, register connection, and then return it
+
+	If remote_cmd is None, then the local connection will be
+	returned.  This also updates some settings on the remote side,
+	like global settings, its connection number, and verbosity.
+
+	"""
+	if not remote_cmd: return Globals.local_connection
+
+	Log("Executing " + remote_cmd, 4)
+	stdin, stdout = os.popen2(remote_cmd)
+	conn_number = len(Globals.connections)
+	conn = PipeConnection(stdout, stdin, conn_number)
+
+	check_connection_version(conn, remote_cmd)
+	Log("Registering connection %d" % conn_number, 7)
+	init_connection_routing(conn, conn_number, remote_cmd)
+	init_connection_settings(conn)
+	return conn
+
+def check_connection_version(conn, remote_cmd):
+	"""Log warning if connection has different version"""
+	try: remote_version = conn.Globals.get('version')
+	except ConnectionReadError, exception:
+		Log.FatalError("""%s
+
+Couldn't start up the remote connection by executing
+
+    %s
+
+Remember that, under the default settings, rdiff-backup must be
+installed in the PATH on the remote system.  See the man page for more
+information.""" % (exception, remote_cmd))
+		
+	if remote_version != Globals.version:
+		Log("Warning: Local version %s does not match remote version %s."
+			% (Globals.version, remote_version), 2)
+
+def init_connection_routing(conn, conn_number, remote_cmd):
+	"""Called by init_connection, establish routing, conn dict"""
+	Globals.connection_dict[conn_number] = conn
+
+	conn.SetConnections.init_connection_remote(conn_number)
+	for other_remote_conn in Globals.connections[1:]:
+		conn.SetConnections.add_redirected_conn(
+			other_remote_conn.conn_number)
+		other_remote_conn.SetConnections.add_redirected_conn(conn_number)
+
+	Globals.connections.append(conn)
+	__conn_remote_cmds.append(remote_cmd)
+
+def init_connection_settings(conn):
+	"""Tell new conn about log settings and updated globals"""
+	conn.Log.setverbosity(Log.verbosity)
+	conn.Log.setterm_verbosity(Log.term_verbosity)
+	for setting_name in Globals.changed_settings:
+		conn.Globals.set(setting_name, Globals.get(setting_name))
+
+def init_connection_remote(conn_number):
+	"""Run on server side to tell self that have given conn_number"""
+	Globals.connection_number = conn_number
+	Globals.local_connection.conn_number = conn_number
+	Globals.connection_dict[0] = Globals.connections[1]
+	Globals.connection_dict[conn_number] = Globals.local_connection
+
+def add_redirected_conn(conn_number):
+	"""Run on server side - tell about redirected connection"""
+	Globals.connection_dict[conn_number] = \
+		   RedirectedConnection(conn_number)
+
+def UpdateGlobal(setting_name, val):
+	"""Update value of global variable across all connections"""
+	for conn in Globals.connections:
+		conn.Globals.set(setting_name, val)
+
+def BackupInitConnections(reading_conn, writing_conn):
+	"""Backup specific connection initialization"""
+	reading_conn.Globals.set("isbackup_reader", 1)
+	writing_conn.Globals.set("isbackup_writer", 1)
+	UpdateGlobal("backup_reader", reading_conn)
+	UpdateGlobal("backup_writer", writing_conn)
+
+def CloseConnections():
+	"""Close all connections.  Run by client"""
+	assert not Globals.server
+	for conn in Globals.connections: conn.quit()
+	del Globals.connections[1:] # Only leave local connection
+	Globals.connection_dict = {0: Globals.local_connection}
+	Globals.backup_reader = Globals.isbackup_reader = \
+		  Globals.backup_writer = Globals.isbackup_writer = None
+
+def TestConnections():
+	"""Test connections, printing results"""
+	if len(Globals.connections) == 1: print "No remote connections specified"
+	else:
+		for i in range(1, len(Globals.connections)): test_connection(i)
+
+def test_connection(conn_number):
+	"""Test connection.  conn_number 0 is the local connection"""
+	print "Testing server started by: ", __conn_remote_cmds[conn_number]
+	conn = Globals.connections[conn_number]
+	try:
+		assert conn.pow(2,3) == 8
+		assert conn.os.path.join("a", "b") == "a/b"
+		version = conn.reval("lambda: Globals.version")
+	except:
+		sys.stderr.write("Server tests failed\n")
+		raise
+	if not version == Globals.version:
+		print """Server may work, but there is a version mismatch:
+Local version: %s
+Remote version: %s""" % (Globals.version, version)
+	else: print "Server OK"
+
+
+from log import *
+from rpath import *
+from connection import *
+import Globals

tags/r0-9-2/rdiff-backup/rdiff_backup/Time.py

+# Copyright 2002 Ben Escoto
+#
+# This file is part of rdiff-backup.
+#
+# rdiff-backup is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, Inc., 675 Mass Ave, Cambridge MA
+# 02139, USA; either version 2 of the License, or (at your option) any
+# later version; incorporated herein by reference.
+
+"""Provide time related exceptions and functions"""
+
+import time, types, re
+import Globals
+
+
+class TimeException(Exception): pass
+
+_interval_conv_dict = {"s": 1, "m": 60, "h": 3600, "D": 86400,
+					   "W": 7*86400, "M": 30*86400, "Y": 365*86400}
+_integer_regexp = re.compile("^[0-9]+$")
+_interval_regexp = re.compile("^([0-9]+)([smhDWMY])")
+_genstr_date_regexp1 = re.compile("^(?P<year>[0-9]{4})[-/]"
+					   "(?P<month>[0-9]{1,2})[-/](?P<day>[0-9]{1,2})$")
+_genstr_date_regexp2 = re.compile("^(?P<month>[0-9]{1,2})[-/]"
+					   "(?P<day>[0-9]{1,2})[-/](?P<year>[0-9]{4})$")
+curtime = curtimestr = None
+
+def setcurtime(curtime = None):
+	"""Sets the current time in curtime and curtimestr on all systems"""
+	t = curtime or time.time()
+	for conn in Globals.connections:
+		conn.Time.setcurtime_local(t, timetostring(t))
+
+def setcurtime_local(timeinseconds, timestr):
+	"""Only set the current time locally"""
+	global curtime, curtimestr
+	curtime, curtimestr = timeinseconds, timestr
+
+def setprevtime(timeinseconds):
+	"""Sets the previous inc time in prevtime and prevtimestr"""
+	assert timeinseconds > 0, timeinseconds
+	timestr = timetostring(timeinseconds)
+	for conn in Globals.connections:
+		conn.Time.setprevtime_local(timeinseconds, timestr)
+
+def setprevtime_local(timeinseconds, timestr):
+	"""Like setprevtime but only set the local version"""
+	global prevtime, prevtimestr
+	prevtime, prevtimestr = timeinseconds, timestr
+
+def timetostring(timeinseconds):
+	"""Return w3 datetime compliant listing of timeinseconds"""
+	return time.strftime("%Y-%m-%dT%H" + Globals.time_separator +
+						 "%M" + Globals.time_separator + "%S",
+						 time.localtime(timeinseconds)) + gettzd()
+
+def stringtotime(timestring):
+	"""Return time in seconds from w3 timestring
+
+	If there is an error parsing the string, or it doesn't look
+	like a w3 datetime string, return None.
+
+	"""
+	try:
+		date, daytime = timestring[:19].split("T")
+		year, month, day = map(int, date.split("-"))
+		hour, minute, second = map(int,
+								   daytime.split(Globals.time_separator))
+		assert 1900 < year < 2100, year
+		assert 1 <= month <= 12
+		assert 1 <= day <= 31
+		assert 0 <= hour <= 23
+		assert 0 <= minute <= 59
+		assert 0 <= second <= 61  # leap seconds
+		timetuple = (year, month, day, hour, minute, second, -1, -1, -1)
+		if time.daylight:
+			utc_in_secs = time.mktime(timetuple) - time.altzone
+		else: utc_in_secs = time.mktime(timetuple) - time.timezone
+
+		return long(utc_in_secs) + tzdtoseconds(timestring[19:])
+	except (TypeError, ValueError, AssertionError): return None
+
+def timetopretty(timeinseconds):
+	"""Return pretty version of time"""
+	return time.asctime(time.localtime(timeinseconds))
+
+def stringtopretty(timestring):
+	"""Return pretty version of time given w3 time string"""
+	return timetopretty(stringtotime(timestring))
+
+def inttopretty(seconds):
+	"""Convert num of seconds to readable string like "2 hours"."""
+	partlist = []
+	hours, seconds = divmod(seconds, 3600)
+	if hours > 1: partlist.append("%d hours" % hours)
+	elif hours == 1: partlist.append("1 hour")
+
+	minutes, seconds = divmod(seconds, 60)
+	if minutes > 1: partlist.append("%d minutes" % minutes)
+	elif minutes == 1: partlist.append("1 minute")
+
+	if seconds == 1: partlist.append("1 second")
+	elif not partlist or seconds > 1:
+		if isinstance(seconds, int) or isinstance(seconds, long):
+			partlist.append("%s seconds" % seconds)
+		else: partlist.append("%.2f seconds" % seconds)
+	return " ".join(partlist)
+
+def intstringtoseconds(interval_string):
+	"""Convert a string expressing an interval (e.g. "4D2s") to seconds"""
+	def error():
+		raise TimeException("""Bad interval string "%s"
+
+Intervals are specified like 2Y (2 years) or 2h30m (2.5 hours).  The
+allowed special characters are s, m, h, D, W, M, and Y.  See the man
+page for more information.
+""" % interval_string)
+	if len(interval_string) < 2: error()
+
+	total = 0
+	while interval_string:
+		match = _interval_regexp.match(interval_string)
+		if not match: error()
+		num, ext = int(match.group(1)), match.group(2)
+		if not ext in _interval_conv_dict or num < 0: error()
+		total += num*_interval_conv_dict[ext]
+		interval_string = interval_string[match.end(0):]
+	return total
+
+def gettzd():
+	"""Return w3's timezone identification string.
+
+	Expresed as [+/-]hh:mm.  For instance, PST is -08:00.  Zone is
+	coincides with what localtime(), etc., use.
+
+	"""
+	if time.daylight: offset = -1 * time.altzone/60
+	else: offset = -1 * time.timezone/60
+	if offset > 0: prefix = "+"
+	elif offset < 0: prefix = "-"
+	else: return "Z" # time is already in UTC
+
+	hours, minutes = map(abs, divmod(offset, 60))
+	assert 0 <= hours <= 23
+	assert 0 <= minutes <= 59
+	return "%s%02d%s%02d" % (prefix, hours,
+							 Globals.time_separator, minutes)
+
+def tzdtoseconds(tzd):
+	"""Given w3 compliant TZD, return how far ahead UTC is"""
+	if tzd == "Z": return 0
+	assert len(tzd) == 6 # only accept forms like +08:00 for now
+	assert (tzd[0] == "-" or tzd[0] == "+") and \
+		   tzd[3] == Globals.time_separator
+	return -60 * (60 * int(tzd[:3]) + int(tzd[4:]))
+
+def cmp(time1, time2):
+	"""Compare time1 and time2 and return -1, 0, or 1"""
+	if type(time1) is types.StringType:
+		time1 = stringtotime(time1)
+		assert time1 is not None
+	if type(time2) is types.StringType:
+		time2 = stringtotime(time2)
+		assert time2 is not None
+
+	if time1 < time2: return -1
+	elif time1 == time2: return 0
+	else: return 1
+
+def genstrtotime(timestr, curtime = None):
+	"""Convert a generic time string to a time in seconds"""
+	if curtime is None: curtime = globals()['curtime']
+	if timestr == "now": return curtime
+
+	def error():
+		raise TimeException("""Bad time string "%s"
+
+The acceptible time strings are intervals (like "3D64s"), w3-datetime
+strings, like "2002-04-26T04:22:01-07:00" (strings like
+"2002-04-26T04:22:01" are also acceptable - rdiff-backup will use the
+current time zone), or ordinary dates like 2/4/1997 or 2001-04-23
+(various combinations are acceptable, but the month always precedes
+the day).""" % timestr)
+
+	# Test for straight integer
+	if _integer_regexp.search(timestr): return int(timestr)
+
+	# Test for w3-datetime format, possibly missing tzd
+	t = stringtotime(timestr) or stringtotime(timestr+gettzd())
+	if t: return t
+
+	try: # test for an interval, like "2 days ago"
+		return curtime - intstringtoseconds(timestr)
+	except TimeException: pass
+
+	# Now check for dates like 2001/3/23
+	match = _genstr_date_regexp1.search(timestr) or \
+			_genstr_date_regexp2.search(timestr)
+	if not match: error()
+	timestr = "%s-%02d-%02dT00:00:00%s" % (match.group('year'),
+			     int(match.group('month')), int(match.group('day')), gettzd())
+	t = stringtotime(timestr)
+	if t: return t
+	else: error()
+
+

tags/r0-9-2/rdiff-backup/rdiff_backup/compilec.py

+#!/usr/bin/env python
+
+import sys, os
+from distutils.core import setup, Extension
+
+assert len(sys.argv) == 1
+sys.argv.append("build")
+
+setup(name="CModule",
+	  version="0.9.0",
+	  description="rdiff-backup's C component",
+	  ext_modules=[Extension("C", ["cmodule.c"])])
+
+assert not os.system("mv build/lib.linux-i686-2.2/C.so .")
+assert not os.system("rm -rf build")

tags/r0-9-2/rdiff-backup/rdiff_backup/filelist.py

+from __future__ import generators
+execfile("manage.py")
+
+#######################################################################
+#
+# filelist - Some routines that help with operations over files listed
+#            in standard input instead of over whole directories.
+#
+
+class FilelistError(Exception): pass
+
+class Filelist:
+	"""Many of these methods have analogs in highlevel.py"""
+	def File2Iter(fp, baserp):
+		"""Convert file obj with one pathname per line into rpiter
+
+		Closes fp when done.  Given files are added to baserp.
+
+		"""
+		while 1:
+			line = fp.readline()
+			if not line: break
+			if line[-1] == "\n": line = line[:-1] # strip trailing newline
+			if not line: continue # skip blank lines
+			elif line[0] == "/": raise FilelistError(
+				"Read in absolute file name %s." % line)
+			yield baserp.append(line)
+		assert not fp.close(), "Error closing filelist fp"
+
+	def Mirror(src_rpath, dest_rpath, rpiter):
+		"""Copy files in fileiter from src_rpath to dest_rpath"""
+		sigiter = dest_rpath.conn.Filelist.get_sigs(dest_rpath, rpiter)
+		diffiter = Filelist.get_diffs(src_rpath, sigiter)
+		dest_rpath.conn.Filelist.patch(dest_rpath, diffiter)
+		dest_rpath.setdata()
+
+	def Mirror_and_increment(src_rpath, dest_rpath, inc_rpath):
+		"""Mirror + put increment in tree based at inc_rpath"""
+		sigiter = dest_rpath.conn.Filelist.get_sigs(dest_rpath, rpiter)
+		diffiter = Filelist.get_diffs(src_rpath, sigiter)
+		dest_rpath.conn.Filelist.patch_and_increment(dest_rpath, diffiter,
+													 inc_rpath)
+		dest_rpath.setdata()
+
+	def get_sigs(dest_rpbase, rpiter):
+		"""Get signatures of file analogs in rpiter
+
+		This is meant to be run on the destination side.  Only the
+		extention part of the rps in rpiter will be used; the base is
+		ignored.
+
+		"""
+		def dest_iter(src_iter):
+			for src_rp in src_iter: yield dest_rpbase.new_index(src_rp.index)
+		return RORPIter.Signatures(dest_iter())
+
+	def get_diffs(src_rpbase, sigiter):
+		"""Get diffs based on sigiter and files in src_rpbase
+
+		This should be run on the local side.
+
+		"""
+		for sig_rorp in sigiter:
+			new_rp = src_rpbase.new_index(sig_rorp.index)
+			yield RORPIter.diffonce(sig_rorp, new_rp)
+
+	def patch(dest_rpbase, diffiter):
+		"""Process diffs in diffiter and update files in dest_rbpase.
+
+		Run remotely.
+
+		"""
+		for diff_rorp in diffiter:
+			basisrp = dest_rpbase.new_index(diff_rorp.index)
+			if basisrp.lstat(): Filelist.make_subdirs(basisrp)
+			Log("Processing %s" % basisrp.path, 7)
+			RORPIter.patchonce(dest_rpbase, basisrp, diff_rorp)
+
+	def patch_and_increment(dest_rpbase, diffiter, inc_rpbase):
+		"""Apply diffs in diffiter to dest_rpbase, and increment to inc_rpbase
+
+		Also to be run remotely.
+
+		"""
+		for diff_rorp in diffiter:
+			basisrp = dest_rpbase.new_index(diff_rorp.index)
+			if diff_rorp.lstat(): Filelist.make_subdirs(basisrp)
+			Log("Processing %s" % basisrp.path, 7)
+			# XXX This isn't done yet...
+
+	def make_subdirs(rpath):
+		"""Make sure that all the directories under the rpath exist
+
+		This function doesn't try to get the permissions right on the
+		underlying directories, just do the minimum to make sure the
+		file can be created.
+
+		"""
+		dirname = rpath.dirsplit()[0]
+		if dirname == '.' or dirname == '': return
+		dir_rp = RPath(rpath.conn, dirname)
+		Filelist.make_subdirs(dir_rp)
+		if not dir_rp.lstat(): dir_rp.mkdir()
+
+
+MakeStatic(Filelist)

tags/r0-9-2/rdiff-backup/rdiff_backup/header.py

+#!/usr/bin/env python
+#
+# rdiff-backup -- Mirror files while keeping incremental changes
+# Version 0.8.0 released June 14, 2002
+# Copyright (C) 2001, 2002 Ben Escoto <bescoto@stanford.edu>
+#
+# This program is licensed under the GNU General Public License (GPL).
+# Distributions of rdiff-backup usually include a copy of the GPL in a
+# file called COPYING.  The GPL is also available online at
+# http://www.gnu.org/copyleft/gpl.html.
+#
+# See http://www.stanford.edu/~bescoto/rdiff-backup for more
+# information.  Please send mail to me or the mailing list if you find
+# bugs or have any suggestions.
+
+from __future__ import nested_scopes, generators
+import os, stat, time, sys, getopt, re, cPickle, types, shutil, sha, marshal, traceback, popen2, tempfile, gzip, UserList, errno, signal
+
+

tags/r0-9-2/rdiff-backup/rdiff_backup/profiled_rdb.py

+#!/usr/bin/env python
+
+"""Run rdiff-backup with profiling on
+
+Same as rdiff-backup but runs profiler, and prints profiling
+statistics afterwards.
+
+"""
+
+__no_execute__ = 1
+import sys, rdiff_backup.Main, profile, pstats
+profile.run("rdiff_backup.Main.Main(%s)" % repr(sys.argv[1:]),
+			"profile-output")
+p = pstats.Stats("profile-output")
+p.sort_stats('time')
+p.print_stats(40)
+#p.print_callers(20)
+
+