Skip to content

R
re6stnet
Commit 4f9f9c37 authored by Guillaume Bury's avatar Guillaume Bury
Browse files
Options

Egg packaging

parent 52a42554
Hide whitespace changes
Inline Side-by-side
Showing with 352 additions and 230 deletions
.gitignore 0 → 100644
View file @ 4f9f9c37
*.pyc
*.pyo
*.swp
*~
/build/
/dist/
/re6stnet.egg-info/
CHANGES 0 → 100644
View file @ 4f9f9c37
Change History
==============
0.1 (2012-08-??)
----------------
Initial release.
MANIFEST.in 0 → 100644
View file @ 4f9f9c37
graft docs
include TODO
README
View file @ 4f9f9c37
re6stnet is a daemon setting up a resilient virtual private network over the
internet
==========
re6stnet
==========
HOW TO:
re6stnet has three separate components : a setup (re6stnet-conf), a
server (re6stnet-registry) and a client (re6stnet).
Lambda users only have to launch the setup and then their client.
The server is meant to be started once on a node which also will be running
a client instance.
---------------------------------------------
Resilient, Scalable, IPv6 Network application
---------------------------------------------
The organisation of the code
re6stnet.py Client : Init and main loop
re6st/plib.py Launch server/client/routing processes
re6st/utils.py Small functions to do useful jobs
re6st/db.py Manage the peer database
re6st/tunnel.py Manage the tunnels
re6st/upnpigd.py Forward ports
re6st-conf.py Get certificates from server
re6st-registry.py Server : deliver certificates and distribute peers
Note: On certain version of python (e.g. 2.7.3~rc2-2.1 ) dns lookup is
performed for each request, and cause a delay in response.
To avoid this, one can either upgrade python, fix their resolv.conf or use
the fix at the end of this file.
OPTIONS : RE6ST-REGISTRY
usage : ./registry port [options...]
port
The port on which the server will listen
--private ip
Ipv6 address of the re6stnet client running on the machine. This
address will be advertised only to nodes having a valid
certificate.
--db path
Path to the server Database file. A new DB file will be created
and correctly initialized if the file doesn't exists.
One can give ":memory" as path, the database is then temporary
--ca path
Path to the certificate authority file. The certificate authority
MUST contain the VPN network prefix in its serial number. To
generate correct ca and key files for the 2001:db8:42:: prefix,
the following command can be used :
openssl req -nodes -new -x509 -key ca.key -set_serial \
0x120010db80042 -days 365 -out ca.crt
--key path
Path to the server key file. To generate a key file, see the --ca
option
--bootstrap prefix ip port proto
Connection informations of a node given to other as a bootstrap
node to initiate connection with the network.
Prefix should be the prefix number of a node, given in binary and
with correct length. For instance the VPN address
2001:db8:42:1::/64 ( asusming a network prefix 2001:db8:42::/48 )
corresponds to a prefix 1/16 i.e 0000000000000001.
--mailhost mailhost
Mailhost to be used to send email containing token for registration
:Author: Nexedi <re6stnet@erp5.org>
OPTIONS : RE6ST-SETUP
usage : ./setup [options...]
--server address
Ip address of the machine running the re6stnet server. Both ipv4
and ipv6 addresses are supported.
Overview
========
--port port
Port to connect to on the machine running the re6stnet server.
re6stnet creates a resilient, scalable, ipv6 network on top of an existing ipv4
network, by creating tunnels on the fly, and then routing targeted traffic
through these tunnels.
-d, --dir directory
Path of a directory where will be stored the files generated by the
setup. The Setup genereates the following files, in the explicit
order :
- ca.pem : certificate authority file downloaded from the server
- peers.db : peers database initialized for re6stnet.py
- cert.key : private key generated by the script
- cert.crt : individual certificate file generated by the server
- dh2048.pem : dh file for oenvpn server
re6stnet can be used to :
- guarantee connectedness between computers connected to the
internet, for which there exists a working route ( in case the direct route
isn't available ).
- create very large networks
- give ipv6 addresses to machines with only ipv4 available
-r, --req name value
Specify an attribute to add to the certificate request sent to the
server. Can be used multiple times.
Each use of the --req name value, will add the attribute name with
the associated value in the sugbject of the certificate request.
--ca-only
Stop the script after downloading the certificate authority file
from the server
HOW IT WORKS
============
--db-only
Stop the script after creating the peers DB and downloading the
connection information of a bootstrap node of the VPN.
A re6stnet networks consists of at least one server ( re6st-registry ) and many
clients or nodes ( re6stnet ). The server delivers certificates for secure
authentification in establishing tunnels, and - for now - store the complete
list of nodes running.
re6stnet detects and take into accounts nodes present on the local network.
--no-boot
Does not re'quest a bootstrap peer to the peer discovery server
(useful in debug when the server does not have any peer in his
database). When requesting a bootstrap peer to a server whoch does
not have any, an execption will occur, and the script will stop
Resilience
----------
re6stnet can guarantee that if there exists a route between two machines,
traffic will be correctly routed between these two machines.
Since the destination of established connection are random, there can be no
guarantee that such a route exists given which tunnels are established, but
for big enough networks ( more than a hundred nodes ), the probability that
the network isn't connected is very low.
Scalability
-----------
OPTIONS : RES6NET
usage : ./re6stnet.py [options...]
--ip address port proto
Specify connection information to be advertised to other nodes.
address MUST be a ipv4 address since as of now openvpn does not
support ipv6 addresses.
Proto should be either udp or tcp-client
Since each node has very few data about the network, re6stnet is easily
scalable to tens of thousand of nodes
-i, --interface interface
Give one interface name for each use of the argument. The interface
will be used to detect other nodes on the local network.
Requirements
============
--peers-db-refresh duration
Duration in seconds of the peers DB refresh interval.
Default : 3600 ( 1 hour )
- Python 2.7 or later
-l, --log directory
Path to the directory used for log files. Will create one file
for babel logging and one file for each openvpn server and client
started.
Default : /var/log
- OpenSSL binary and development libraries
-s, --state directory
Path to the directory used for state files. State files include :
- peers.db : the peers db used to establish connection
- re6stnet.babeld.state : babeld state file ( created if does not
exists, overriden if exists )
There must be a valid peers db file ( named peers.db ) in the
directory. A valid peers db file can be created with setup.py
Default : /var/lib/re6stnet
- OpenVPN
-v, --verbose level
Defines the verbose level, level should be an integer between 0
and 3 ( including ). There is no precise convention for verbode
level for now, except an increased number means more log messages.
This parameter is also given to babel for its log.
To adjust verbose level for openvpn, add an openvpn optional
arguments at the end of the command line
Default : 0
- Babel ( http://www.pps.univ-paris-diderot.fr/~jch/software/babel/ )
--registry address
Complete public ( reachable from the internet ) address of the machine
running a registry. Will be used to get the pirvate address of the
registry and/or bootstrap peers
Examples : http://192.0.2.42:80, http://[2001:db8:42::1]:80
Installation
============
--hello duration
Set hello interval, in seconds, for both wired and wireless
connections. Openvpn ping-exit option is set to 4 times the hello
interval. Argument passed down to the babel daemon, equivalent
to :
-h duration -H duration
in babeld ( for more information, see babeld man page )
It takes between 3 times and 4 times the hello interval for babel
to re-establish connection with a node for which the direct
connection has been cut
Default : 15
Let's explain how to install the egg
-w, --wireless
Consider all interfaces as being wireless interfaces. Argument
directly passed down to the babeld daemon
Commercial Support
==================
--pp port proto
Port and protocol used by the openvpn server(s). Start one openvpn
server for each couple port/protocol specified.
Additionally, if no external configuration is given in the command
line, re6stnet will attempt to forward a port with upnp for each
couple port/proto given.
Protocols should be either udp or tcp-server.
Default : (1194, udp), (1194, tcp-server)
Later....
--tunnel-refresh duration
Interval in seconds between two tunnel refresh. Refreshing tunnels
mean :
- killing all dead tunnels ( detected via the ping-exit option
if openvpn )
- killing the 'worst' tunnels, so that at least the ratio of
tunnels set by the --refresh-rate option have been killed
- creating new tunnels to other clients randomly choosen in the
peers database, to reach the number of connection specified by
the connection-count option ( There can be less tunnels if the
peers DB does not contain enough peers )
Default : 300
--dh path
Path to the dh file to be used by the openvpn server
(for more information see the openvpn man page)
--ca path
Path to the certificate authority file delivered by the re6stnet
server. The prefix of the VPN network is included in the serial
number of the file.
--cert path
Path to the individual certificate file delivered by the re6stnet
server. The prefix of the machine is included in the certificate's
subject common name.
--connection-count number
The maximum number of openvpn clients to start.
Default : 20
--refresh-rate ratio
The ratio of connection to kill each time we refresh tunnels.
For more information see the --tunnel-refresh option
ratio should be a float between 0 and 1 ( included )
Default : 0.05
openvpn_args
Additional arguments to be passed down to all openvpn processes
can be given at the end of the command line.
In that case, insert '--' to delimit re6stnet regular options
from the additional openvpn arguments. The list of arguments will
be passed down to ALL openvpn processes ( including servers )
exactly as they are given
One SHOULD give a --key argument with the key file delivered by the
re6stnet server
@file
You can give to re6stnet a config file as a regular argument
(meaning before giving optional openvpn arguments)
The file should contain one option per line, possibly ommitting
the '--'. Only long option are allowed (i.e "v 3" will not work
while "verbose 3" will)
You can give a file ( with the @ prefix ) as an argument within a
file
.. The organisation of the code
re6stnet.py Client : Init and main loop
re6st/plib.py Launch server/client/routing processes
re6st/utils.py Small functions to do useful jobs
re6st/db.py Manage the peer database
re6st/tunnel.py Manage the tunnels
re6st/upnpigd.py Forward ports
re6st-conf.py Get certificates from server
re6st-registry.py Server : deliver certificates and distribute peers
.. XXX: write monkey-patch in __init__.py
If you are using a version of python < 2.7.3-2, then you should include this at
the beggining of registry.py
Note: On certain version of python (e.g. 2.7.3~rc2-2.1 ) dns lookup is
performed for each request, and cause a delay in response.
To avoid this, one can either upgrade python, fix their resolv.conf or use
the fix at the end of this file.
--------------------------------------------------------------------------------
# Fix for librpcxml to avoid doing reverse dns on each request
......@@ -245,17 +98,3 @@ def not_insane_address_string(self):
BaseHTTPServer.BaseHTTPRequestHandler.address_string = not_insane_address_string
--------------------------------------------------------------------------------
New log system :
we use the logging module now. There are three levels for log messages :
- info : give basic information about what re6stnet is doing
- debug : display internal work of the script (finished action, detailed
information about tunnels, etc... )
- trace : for intensive debug, display configuration, arguments given to
processes, all information pertinent to debug but not required
mot of the time
Additionally, warning, error and exception can be used.
Note : logging.exception prints informations similar to pdb.set_trace()
info, which is pretty heavy, so for exception we expect ( for
instance, connection problems to the registry ), one can print the
exception as warning. ( see db.refresh() ).
docs/re6st-conf 0 → 100644
View file @ 4f9f9c37
OPTIONS : RE6ST-SETUP
usage : ./setup [options...]
--server address
Ip address of the machine running the re6stnet server. Both ipv4
and ipv6 addresses are supported.
--port port
Port to connect to on the machine running the re6stnet server.
-d, --dir directory
Path of a directory where will be stored the files generated by the
setup. The Setup genereates the following files, in the explicit
order :
- ca.pem : certificate authority file downloaded from the server
- peers.db : peers database initialized for re6stnet.py
- cert.key : private key generated by the script
- cert.crt : individual certificate file generated by the server
- dh2048.pem : dh file for oenvpn server
-r, --req name value
Specify an attribute to add to the certificate request sent to the
server. Can be used multiple times.
Each use of the --req name value, will add the attribute name with
the associated value in the sugbject of the certificate request.
--ca-only
Stop the script after downloading the certificate authority file
from the server
--db-only
Stop the script after creating the peers DB and downloading the
connection information of a bootstrap node of the VPN.
--no-boot
Does not re'quest a bootstrap peer to the peer discovery server
(useful in debug when the server does not have any peer in his
database). When requesting a bootstrap peer to a server whoch does
not have any, an execption will occur, and the script will stop
docs/re6st-registry 0 → 100644
View file @ 4f9f9c37
OPTIONS : RE6ST-REGISTRY
usage : ./registry port [options...]
port
The port on which the server will listen
--private ip
Ipv6 address of the re6stnet client running on the machine. This
address will be advertised only to nodes having a valid
certificate.
--db path
Path to the server Database file. A new DB file will be created
and correctly initialized if the file doesn't exists.
One can give ":memory" as path, the database is then temporary
--ca path
Path to the certificate authority file. The certificate authority
MUST contain the VPN network prefix in its serial number. To
generate correct ca and key files for the 2001:db8:42:: prefix,
the following command can be used :
openssl req -nodes -new -x509 -key ca.key -set_serial \
0x120010db80042 -days 365 -out ca.crt
--key path
Path to the server key file. To generate a key file, see the --ca
option
--bootstrap prefix ip port proto
Connection informations of a node given to other as a bootstrap
node to initiate connection with the network.
Prefix should be the prefix number of a node, given in binary and
with correct length. For instance the VPN address
2001:db8:42:1::/64 ( asusming a network prefix 2001:db8:42::/48 )
corresponds to a prefix 1/16 i.e 0000000000000001.
--mailhost mailhost
Mailhost to be used to send email containing token for registration
docs/re6stnet 0 → 100644
View file @ 4f9f9c37
OPTIONS : RES6NET
usage : ./re6stnet.py [options...]
--ip address port proto
Specify connection information to be advertised to other nodes.
address MUST be a ipv4 address since as of now openvpn does not
support ipv6 addresses.
Proto should be either udp or tcp-client
-i, --interface interface
Give one interface name for each use of the argument. The interface
will be used to detect other nodes on the local network.
--peers-db-refresh duration
Duration in seconds of the peers DB refresh interval.
Default : 3600 ( 1 hour )
-l, --log directory
Path to the directory used for log files. Will create one file
for babel logging and one file for each openvpn server and client
started.
Default : /var/log
-s, --state directory
Path to the directory used for state files. State files include :
- peers.db : the peers db used to establish connection
- re6stnet.babeld.state : babeld state file ( created if does not
exists, overriden if exists )
There must be a valid peers db file ( named peers.db ) in the
directory. A valid peers db file can be created with setup.py
Default : /var/lib/re6stnet
-v, --verbose level
Defines the verbose level, level should be an integer between 0
and 3 ( including ). There is no precise convention for verbode
level for now, except an increased number means more log messages.
This parameter is also given to babel for its log.
To adjust verbose level for openvpn, add an openvpn optional
arguments at the end of the command line
Default : 0
--registry address
Complete public ( reachable from the internet ) address of the machine
running a registry. Will be used to get the pirvate address of the
registry and/or bootstrap peers
Examples : http://192.0.2.42:80, http://[2001:db8:42::1]:80
--hello duration
Set hello interval, in seconds, for both wired and wireless
connections. Openvpn ping-exit option is set to 4 times the hello
interval. Argument passed down to the babel daemon, equivalent
to :
-h duration -H duration
in babeld ( for more information, see babeld man page )
It takes between 3 times and 4 times the hello interval for babel
to re-establish connection with a node for which the direct
connection has been cut
Default : 15
-w, --wireless
Consider all interfaces as being wireless interfaces. Argument
directly passed down to the babeld daemon
--pp port proto
Port and protocol used by the openvpn server(s). Start one openvpn
server for each couple port/protocol specified.
Additionally, if no external configuration is given in the command
line, re6stnet will attempt to forward a port with upnp for each
couple port/proto given.
Protocols should be either udp or tcp-server.
Default : (1194, udp), (1194, tcp-server)
--tunnel-refresh duration
Interval in seconds between two tunnel refresh. Refreshing tunnels
mean :
- killing all dead tunnels ( detected via the ping-exit option
if openvpn )
- killing the 'worst' tunnels, so that at least the ratio of
tunnels set by the --refresh-rate option have been killed
- creating new tunnels to other clients randomly choosen in the
peers database, to reach the number of connection specified by
the connection-count option ( There can be less tunnels if the
peers DB does not contain enough peers )
Default : 300
--dh path
Path to the dh file to be used by the openvpn server
(for more information see the openvpn man page)
--ca path
Path to the certificate authority file delivered by the re6stnet
server. The prefix of the VPN network is included in the serial
number of the file.
--cert path
Path to the individual certificate file delivered by the re6stnet
server. The prefix of the machine is included in the certificate's
subject common name.
--connection-count number
The maximum number of openvpn clients to start.
Default : 20
--refresh-rate ratio
The ratio of connection to kill each time we refresh tunnels.
For more information see the --tunnel-refresh option
ratio should be a float between 0 and 1 ( included )
Default : 0.05
openvpn_args
Additional arguments to be passed down to all openvpn processes
can be given at the end of the command line.
In that case, insert '--' to delimit re6stnet regular options
from the additional openvpn arguments. The list of arguments will
be passed down to ALL openvpn processes ( including servers )
exactly as they are given
One SHOULD give a --key argument with the key file delivered by the
re6stnet server
@file
You can give to re6stnet a config file as a regular argument
(meaning before giving optional openvpn arguments)
The file should contain one option per line, possibly ommitting
the '--'. Only long option are allowed (i.e "v 3" will not work
while "verbose 3" will)
You can give a file ( with the @ prefix ) as an argument within a
file
If you are using a version of python < 2.7.3-2, then you should include this at
the beggining of registry.py
--------------------------------------------------------------------------------
# Fix for librpcxml to avoid doing reverse dns on each request
# it was causing a 10s delay on each request when no reverse DNS was avalaible
import BaseHTTPServer
def not_insane_address_string(self):
host, port = self.client_address[:2]
return '%s (reverse DNS disabled)' % host # used to call: socket.getfqdn(host)
BaseHTTPServer.BaseHTTPRequestHandler.address_string = not_insane_address_string
--------------------------------------------------------------------------------
re6st/plib.py
View file @ 4f9f9c37
......@@ -2,7 +2,9 @@ import os, subprocess, logging
import utils
verbose = 0
here = os.path.realpath(os.path.dirname(__file__))
ovpn_server = os.path.join(here, 'ovpn-server')
ovpn_client = os.path.join(here, 'ovpn-client')
def openvpn(hello_interval, *args, **kw):
args = ['openvpn',
......@@ -23,9 +25,9 @@ def server(server_ip, ip_length, max_clients, dh_path, pipe_fd, port, proto, hel
return openvpn(hello_interval,
'--tls-server',
'--mode', 'server',
'--up', 'ovpn-server %s/%u' % (server_ip, 64),
'--client-connect', 'ovpn-server ' + str(pipe_fd),
'--client-disconnect', 'ovpn-server ' + str(pipe_fd),
'--up', '%s %s/%u' % (ovpn_server, server_ip, 64),
'--client-connect', ovpn_server + ' ' + str(pipe_fd),
'--client-disconnect', ovpn_server + ' ' + str(pipe_fd),
'--dh', dh_path,
'--max-clients', str(max_clients),
'--port', str(port),
......@@ -37,8 +39,8 @@ def client(server_address, pipe_fd, hello_interval, *args, **kw):
logging.debug('Starting client...')
remote = ['--nobind',
'--client',
'--up', 'ovpn-client',
'--route-up', 'ovpn-client ' + str(pipe_fd)]
'--up', ovpn_client,
'--route-up', ovpn_client + ' ' + str(pipe_fd)]
try:
for ip, port, proto in utils.address_list(server_address):
remote += '--remote', ip, port, proto
......
setup.py 0 → 100644
View file @ 4f9f9c37
"""Resilient, Scalable, IPv6 Network
"""
from setuptools import setup, find_packages
classifiers = """\
Environment :: Console
License :: OSI Approved :: GNU General Public License (GPL)
Natural Language :: English
Operating System :: POSIX :: Linux
Programming Language :: Python :: 2.7
Topic :: Internet
Topic :: System :: Networking
"""
setup(
name = 're6stnet',
version = '0.1',
description = __doc__.strip(),
author = 'Nexedi',
author_email = 're6stnet@erp5.org',
url = 'http://re6st.net',
license = 'GPL 2+',
platforms = ["any"],
classifiers=classifiers.splitlines(),
long_description = ".. contents::\n\n" + open('README').read()
+ "\n" + open('CHANGES').read(),
packages = find_packages(),
scripts = [
're6stnet',
're6st-conf',
're6st-registry',
],
package_data = {
're6st': [
'ovpn-server',
'ovpn-client',
],
},
install_requires = ['pyOpenSSL', 'miniupnpc'],
#dependency_links = [
# "http://miniupnp.free.fr/files/download.php?file=miniupnpc-1.7.20120714.tar.gz#egg=miniupnpc-1.7",
# ],
zip_safe = False,
)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7