Skip to content

R
re6stnet
Commit 77ffa9cd authored by Ulysse Beaugnon's avatar Ulysse Beaugnon
Browse files
Options

Merge branch 'master' of https://git.erp5.org/repos/vifibnet 

Conflicts:
	TODO
	tunnel.py
	vifibnet.py
parents d4ec57c5 5965813b
Hide whitespace changes
Inline Side-by-side
Showing with 107 additions and 56 deletions
README
View file @ 77ffa9cd
Note (for U): logs haven't changed. I just added some logs messages to notify
when a task has been finished
Vifibnet is a daemon setting up a resilient virtual private network over the
internet
HOW TO:
Vifibnet ( sic ) has three separate components : a setup (setup.py), a
Vifibnet ( sic ) has three separate components : a setup (setup.py), a
server (registry.py) and a client (vifibnet.py.
Lambda users only have to launch the setup and then their client.
The server is meant to be started once on a node which also will be running
......@@ -31,7 +35,7 @@ OPTIONS : REGISTRY.PY
MUST contain the VPN network prefix in its serial number. To
generate correct ca and key files for the 2001:db8:42:: prefix,
the following command can be used :
openssl req -nodes -new -x509 -key ca.key -set_serial
openssl req -nodes -new -x509 -key ca.key -set_serial \
0x120010db80042 -days 365 -out ca.crt
--key path
......@@ -87,12 +91,11 @@ OPTIONS : VIFIBNET.PY
Specify connection information to be advertised to other nodes.
address MUST be a ipv4 address since as of now openvpn does not
support ipv6 addresses.
proto should be either udp or tcp-client
Proto should be either udp or tcp-client
--internal-port port
Specify the port on which will be launched the openvpn server(s)
Can differ from port given in the --ip option.
Default : 1194
-i, --interface interface
Give one interface name for each use of the argument. The interface
will be used to detect other nodes on the local network.
--peers-db-refresh duration
Duration in seconds of the peers DB refresh interval.
......@@ -138,17 +141,20 @@ OPTIONS : VIFIBNET.PY
It takes between 3 times and 4 times the hello interval for babel
to re-establish connection with a node for which the direct
connection has been cut
Default : 30
Default : 15
-w, --wireless
Consider all interfaces as being wireless interfaces. Argument
directly passed down to the babeld daemon
--proto p [p']
Protocol used by the openvpn server(s). Start one openvpn server
for each protocl specified.
p (and p') should be either udp or tcp-server
Default : udp
--pp port proto
Port and protocol used by the openvpn server(s). Start one openvpn
server for each couple port/protocol specified.
Additionally, if no external configuration is given in the command
line, vifibnet will attempt to forward a port with upnp for each
couple port/proto given.
Protocols should be either udp or tcp-server.
Default : (1194, udp)
--tunnel-refresh duration
Interval in seconds between two tunnel refresh. Refreshing tunnels
......@@ -203,3 +209,5 @@ OPTIONS : VIFIBNET.PY
The file should contain one option per line, possibly ommitting
the '--'. Only long option are allowed (i.e "v 3" will not work
while "verbose 3" will)
You can give a file ( with the @ prefix ) as an argument within a
file
TODO
View file @ 77ffa9cd
Bugs:
The address of the client is declared while it should only be the address
of the server
To be done :
use the server as a bootstrap node -> switch peer discovery to be done
by vifibnet directly ?
......@@ -17,14 +13,15 @@ To be done :
provide false informations. Needs of signature
If it is not sufficient, we could use avahi (dm-dns for linux)
Replace comments at the beginning of functions with docstrings & give all
fn docstrings
Write docstrings for all class/methods/functions
Upgrade the logging function in order to be able to log message like
"Refreshing peers DB ... done", or add log messages to specify that an
action advertised by a previous log message has been completed
To be discussed:
G : There is a blacklist system now ( blacklisted prefixes are deleted from
the peers database ). Since all nodes whose packets are routed through
the local network are blacklisted, I think we should reset the blacklist
from time to time....
U : Babel seems to be very long to establish the routes : maybe we should
tell him thant we are not on a wired network but on a mobile network ?
G : babel establish routes quickly enough i'd say. There are two new
......@@ -52,3 +49,4 @@ To be discussed:
reestablish connection, if a direct link is cut
U : So we have to reduce the hello interval. 2min to detect a dead link is
far too much.
G : k
db.py
View file @ 77ffa9cd
......@@ -15,28 +15,40 @@ class PeerManager:
self._server_port = server_port
self._db_size = db_size
self._pp = pp
self._blacklist = [(prefix,)]
self._manual = manual
self._proxy = xmlrpclib.ServerProxy('http://%s:%u'
% (server, server_port))
utils.log('Connectiong to peers database', 4)
utils.log('Connectiong to peers database...', 4)
self._db = sqlite3.connect(os.path.join(db_dir_path, 'peers.db'),
isolation_level=None)
utils.log('Preparing peers database', 4)
utils.log('Database opened', 5)
utils.log('Preparing peers database...', 4)
try:
self._db.execute("UPDATE peers SET used = 0")
except sqlite3.OperationalError, e:
if e.args[0] == 'no such table: peers':
raise RuntimeError
utils.log('Database prepared', 5)
self.next_refresh = time.time()
def reset_blacklist(self):
self._blacklist = [(self._prefix)]
def blacklist(self, prefix):
utils.log('Blacklisting %s' % (prefix,), 4)
self._db.execute("DELETE FROM peers WHERE prefix = ?", (prefix,))
self._blacklist = list(set(self._blacklist + [(prefix,)]))
def refresh(self):
utils.log('Refreshing the peers DB', 2)
utils.log('Refreshing the peers DB...', 2)
try:
self._declare()
self._populate()
utils.log('DB refreshed', 3)
self.next_refresh = time.time() + self._refresh_time
except socket.error, e:
utils.log(str(e), 4)
......@@ -45,14 +57,15 @@ class PeerManager:
def _declare(self):
if self._address != None:
utils.log('Sending connection info to server', 3)
utils.log('Sending connection info to server...', 3)
self._proxy.declare((self._internal_ip,
utils.address_list(self._address)))
utils.log('Info sent', 5)
else:
utils.log("Warning : couldn't send ip, unknown external config", 4)
def _populate(self):
utils.log('Populating the peers DB', 2)
utils.log('Populating the peers DB...', 2)
new_peer_list = self._proxy.getPeerList(self._db_size,
self._internal_ip)
self._db.execute("""DELETE FROM peers WHERE used <= 0 ORDER BY used,
......@@ -61,7 +74,9 @@ class PeerManager:
(str(len(new_peer_list) - self._db_size),))
self._db.executemany("""INSERT OR IGNORE INTO peers (prefix, address)
VALUES (?,?)""", new_peer_list)
self._db.execute("DELETE FROM peers WHERE prefix = ?", (self._prefix,))
self._db.executemany("DELETE FROM peers WHERE prefix = ?",
self._blacklist)
utils.log('DB populated', 3)
utils.log('New peers : %s' % ', '.join(map(str, new_peer_list)), 5)
def getUnusedPeers(self, peer_count):
......@@ -73,16 +88,19 @@ class PeerManager:
utils.log('Updating peers database : using peer ' + str(prefix), 5)
self._db.execute("UPDATE peers SET used = 1 WHERE prefix = ?",
(prefix,))
utils.log('DB updated', 5)
def unusePeer(self, prefix):
utils.log('Updating peers database : unusing peer ' + str(prefix), 5)
self._db.execute("UPDATE peers SET used = 0 WHERE prefix = ?",
(prefix,))
utils.log('DB updated', 5)
def flagPeer(self, prefix):
utils.log('Updating peers database : flagging peer ' + str(prefix), 5)
self._db.execute("UPDATE peers SET used = -1 WHERE prefix = ?",
(prefix,))
utils.log('DB updated', 5)
def handle_message(self, msg):
script_type, arg = msg.split()
......
plib.py
View file @ 77ffa9cd
......@@ -18,7 +18,7 @@ def openvpn(hello_interval, *args, **kw):
return subprocess.Popen(args, **kw)
def server(server_ip, ip_length, max_clients, dh_path, pipe_fd, port, proto, hello_interval, *args, **kw):
utils.log('Starting server', 3)
utils.log('Starting server...', 3)
return openvpn(hello_interval,
'--tls-server',
'--mode', 'server',
......@@ -32,7 +32,7 @@ def server(server_ip, ip_length, max_clients, dh_path, pipe_fd, port, proto, hel
*args, **kw)
def client(server_address, pipe_fd, hello_interval, *args, **kw):
utils.log('Starting client', 5)
utils.log('Starting client...', 5)
remote= ['--nobind',
'--client',
'--up', 'ovpn-client',
......@@ -44,16 +44,16 @@ def client(server_address, pipe_fd, hello_interval, *args, **kw):
def router(network, internal_ip, interface_list,
wireless, hello_interval, state_path, **kw):
utils.log('Starting babel', 3)
utils.log('Starting babel...', 3)
args = ['babeld',
'-C', 'redistribute local ip %s' % (internal_ip),
'-C', 'redistribute local deny',
# Route VIFIB ip adresses
'-C', 'in ip %s::/%u' % (utils.ipFromBin(network), len(network)),
# Route only addresse in the 'local' network,
# or other entire networks
#'-C', 'in ip %s' % (config.internal_ip),
#'-C', 'in ip ::/0 le %s' % network_mask,
# Route only addresse in the 'local' network,
# or other entire networks
#'-C', 'in ip %s' % (config.internal_ip),
#'-C', 'in ip ::/0 le %s' % network_mask,
# Don't route other addresses
'-C', 'in deny',
'-d', str(verbose),
......
tunnel.py
View file @ 77ffa9cd
import os, random, traceback, time, struct, operator
import os, random, traceback, time, struct, subprocess, operator
import plib, utils, db
log = None
......@@ -15,7 +15,8 @@ class Connection:
self.process = plib.client(address, write_pipe, hello, '--dev', iface,
*ovpn_args, stdout=os.open(os.path.join(log,
'vifibnet.client.%s.log' % (prefix,)),
os.O_WRONLY | os.O_CREAT | os.O_TRUNC))
os.O_WRONLY | os.O_CREAT | os.O_TRUNC),
stderr=subprocess.STDOUT)
self.iface = iface
self.routes = 0
......@@ -66,7 +67,7 @@ class Connection:
class TunnelManager:
def __init__(self, write_pipe, peer_db, openvpn_args, hello_interval,
refresh, connection_count, refresh_rate):
refresh, connection_count, refresh_rate, iface_list, network):
self._write_pipe = write_pipe
self._peer_db = peer_db
self._connection_dict = {}
......@@ -74,6 +75,10 @@ class TunnelManager:
self._ovpn_args = openvpn_args
self._hello = hello_interval
self._refresh_time = refresh
self._network = network
self._net_len = len(network)
self._iface_list = iface_list
self.__indirect_connect = []
self.free_interface_set = set(('client1', 'client2', 'client3',
'client4', 'client5', 'client6',
'client7', 'client8', 'client9',
......@@ -84,11 +89,12 @@ class TunnelManager:
self._refresh_count = refresh_rate * self._client_count
def refresh(self):
utils.log('Refreshing the tunnels', 2)
utils.log('Refreshing the tunnels...', 2)
self._cleanDeads()
self._countRoutes()
self._removeSomeTunnels()
self._makeNewTunnels()
utils.log('Tunnels refreshed', 2)
self.next_refresh = time.time() + self._refresh_time
def _cleanDeads(self):
......@@ -107,7 +113,7 @@ class TunnelManager:
self._kill(prefix)
def _kill(self, prefix):
utils.log('Killing the connection with ' + prefix, 2)
utils.log('Killing the connection with %s...' % (prefix,), 2)
connection = self._connection_dict.pop(prefix)
try:
connection.process.kill()
......@@ -117,9 +123,11 @@ class TunnelManager:
self.free_interface_set.add(connection.iface)
self._peer_db.unusePeer(prefix)
del self._iface_to_prefix[connection.iface]
utils.log('Connection with %s killed' % (prefix,), 2)
def _makeNewTunnels(self):
utils.log('Trying to make %i new tunnels' %
i = 0
utils.log('Trying to make %i new tunnels...' %
(self._client_count - len(self._connection_dict)), 5)
try:
for prefix, address in self._peer_db.getUnusedPeers(
......@@ -131,6 +139,8 @@ class TunnelManager:
prefix, self._ovpn_args)
self._iface_to_prefix[iface] = prefix
self._peer_db.usePeer(prefix)
i += 1
utils.log('%u new tunnels established' % (i,), 3)
except KeyError:
utils.log("""Can't establish connection with %s
: no available interface""" % prefix, 2)
......@@ -138,7 +148,8 @@ class TunnelManager:
traceback.print_exc()
def _countRoutes(self):
utils.log('Starting to count the routes on each interface', 3)
utils.log('Starting to count the routes on each interface...', 3)
self._indirect_connect = []
for iface in self._iface_to_prefix.keys():
self._connection_dict[self._iface_to_prefix[iface]].routes = 0
f = open('/proc/net/ipv6_route', 'r')
......@@ -146,9 +157,21 @@ class TunnelManager:
ip, subnet_size, iface = struct.unpack('32s x 2s 106x %ss x'
% (len(line) - 142), line)
iface = iface.replace(' ', '')
utils.log('Route on iface %s detected to %s/%s'
% (iface, ip, subnet_size), 8)
if iface in self._iface_to_prefix.keys():
self._connection_dict[self._iface_to_prefix[iface]].routes += 1
if iface in self._iface_list:
subnet_size = int(subnet_size, 16)
ip = bin(int(ip, 16))[2:].rjust(128, '0')
if self._net_len < subnet_size < 128 and ip.startswith(self._network):
prefix = ip[self._net_len:subnet_size]
utils.log('A route to %s has been discovered on the LAN'
% (prefix,), 3)
self._peer_db.blacklist(prefix)
utils.log("Routes have been counted", 3)
for p in self._connection_dict.keys():
utils.log('Routes on iface %s : %s' % (
self._connection_dict[p].iface,
self._connection_dict[p].routes), 5)
vifibnet.py
View file @ 77ffa9cd
......@@ -3,17 +3,18 @@ import argparse, errno, os, select, subprocess, time
from argparse import ArgumentParser
import db, plib, upnpigd, utils, tunnel
class ArgParser(ArgumentParser):
def convert_arg_line_to_args(self, arg_line):
arg_line = arg_line.split('#')[0].rstrip()
if arg_line:
if arg_line.startswith('@'):
yield arg_line
return
for arg in ('--' + arg_line.lstrip('--')).split():
if arg.strip():
yield arg
def ovpnArgs(optional_args, ca_path, cert_path):
# Treat openvpn arguments
if optional_args[0] == "--":
......@@ -39,19 +40,17 @@ def getConfig():
help='Path to vifibnet logs directory')
_('-s', '--state', default='/var/lib/vifibnet',
help='Path to VPN state directory')
_('--verbose', '-v', default=0, type=int,
_('-v', '--verbose', default=0, type=int,
help='Defines the verbose level')
#_('--babel-state', default='/var/lib/vifibnet/babel_state',
# help='Path to babeld state-file')
#_('--db', default='/var/lib/vifibnet/peers.db',
# help='Path to peers database')
_('-i', '--interface', action='append', dest='iface_list', default=[],
help='Extra interface for LAN discovery')
_('--server', required=True,
help="VPN address of the discovery peer server")
_('--server-port', required=True, type=int,
help="VPN port of the discovery peer server")
# Routing algorithm options
_('--hello', type=int, default=30,
_('--hello', type=int, default=15,
help='Hello interval for babel, in seconds')
_('-w', '--wireless', action='store_true',
help='''Set all interfaces to be treated as wireless interfaces
......@@ -94,17 +93,20 @@ def main():
tunnel.log = config.log
utils.verbose = plib.verbose = config.verbose
utils.log("Configuration :\n" + str(config), 5)
# Create and open read_only pipe to get server events
utils.log('Creating pipe for server events', 3)
utils.log('Creating pipe for server events...', 3)
r_pipe, write_pipe = os.pipe()
read_pipe = os.fdopen(r_pipe)
utils.log('Pipe created', 5)
# Init db and tunnels
forwarder = None
if manual:
utils.log('Manual external configuration', 3)
utils.log('Detected manual external configuration', 3)
else:
utils.log('Attempting automatic configuration via UPnP', 4)
utils.log('Attempting automatic configuration via UPnP...', 4)
try:
forwarder = upnpigd.Forwarder()
config.address = []
......@@ -120,10 +122,11 @@ def main():
manual, config.pp, 200)
tunnel_manager = tunnel.TunnelManager(write_pipe, peer_db, openvpn_args,
config.hello, config.tunnel_refresh, config.connection_count,
config.refresh_rate)
config.refresh_rate, config.iface_list, network)
# Launch routing protocol. WARNING : you have to be root to start babeld
interface_list = ['vifibnet'] + list(tunnel_manager.free_interface_set)
interface_list = ['vifibnet'] + list(tunnel_manager.free_interface_set) \
+ config.iface_list
router = plib.router(network, internal_ip, interface_list, config.wireless,
config.hello, os.path.join(config.state, 'vifibnet.babeld.state'),
stdout=os.open(os.path.join(config.log, 'vifibnet.babeld.log'),
......@@ -135,7 +138,8 @@ def main():
proto, config.hello, '--dev', 'vifibnet', *openvpn_args,
stdout=os.open(os.path.join(config.log,
'vifibnet.server.%s.log' % (proto,)),
os.O_WRONLY | os.O_CREAT | os.O_TRUNC))
os.O_WRONLY | os.O_CREAT | os.O_TRUNC),
stderr=subprocess.STDOUT)
for port, proto in config.pp)
tunnel_manager.refresh()
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7