Commit 50003a3d authored by Guillaume Bury's avatar Guillaume Bury

Merge branch 'master' of https://git.erp5.org/repos/vifibnet

Conflicts:
	vifibnet.py
parents 356e1468 3368dc9b
......@@ -2,8 +2,8 @@ Vifibnet is a daemon setting up a resilient virtual private network over the
internet
HOW TO:
Vifibnet ( sic ) has three separate components : a setup, a server and
a client.
Vifibnet ( sic ) has three separate components : a setup (setup.py), a
server (registry.py) and a client (vifibnet.py.
Lambda users only have to launch the setup and then their client.
The server is meant to be started once on a node which also will be running
a client instance.
......
G : We should focus on clearing the TODO List, then go to testing phase,
since the end is nearing... ( I finish on august, 3 )
To be done :
Catch a more precise exception thant Exception at line 108 in vifibnet.py
( UPnP forwarding )
Upgrade the logging function in order to be able to log message like
"Refreshing peers DB ... done", or add log messages to specify that an
action advertised by a previous log message has been completed
......@@ -14,6 +8,8 @@ To be done :
Use an algorithm to choose which connections to keep and/or establish
instead of pure randomness
|-> number of routes / tunnel : doesn't work
|-> favorise most used roads ?
Replace comments at the beginning of functions with docstrings & give all
fn docstrings
......@@ -24,27 +20,12 @@ To be done :
Use a timeout for the server peersDB so we can flag unreachable peers and
remove the peers whose certificate is no longer valid
Specify a lease duration in ForwardViaUPnP
Specify a lease duration in ForwardViaUPnP and also forward a tcp port
Handle LAN internally in order not to have catastrophic results ....
( avahi could be used )
To be discussed:
Ideas for the name :
CON ( cloud overlay network ), cloudnet. From J.P -> already taken
G, J : To get traffic stats ( bytes in/out ), you can use
/sys/class/net/interface/statistics/rx_bytes, etc...
or /proc/net/dev/snmp6/interface ( all in one file ). This can be enough
if used as follows: get traffic diff from last time we checked in order
to choose which connection is significantly unused compared to others,
and close it. Of course, too recent connections (i.e. those for which we
have no previous stat) would be always kept.
This should be combined with routing table (i.e. how many nodes are
served by each tunnel), which is possibly redundant.
ip6tables should be avoided if possible.
U : Great !!!
U : Babel seems to be very long to establish the routes : maybe we should
tell him thant we are not on a wired network but on a mobile network ?
G : babel establish routes quickly enough i'd say. There are two new
......@@ -69,28 +50,13 @@ To be discussed:
on nexedi's server downtime ? it could be useful for the internship
rapport
U : The peer DB size should depend on the number of connection and the
refresh time
G : ?! I don't agree, the db size should be proportional ( with a factor
like 0.01 or less ) to the total number of peers in the entire network,
with maybe a max size.
U : what we need to do is to keep the randomness. For this, we need a big
enought DB to ensure we can still choose a peer as if it was choosen
directly from the server. The requiered db size can be calculated from
the number of connections and the refresh time.
G : ok, you can erase this talk
U : Why are --ip and internal-port mutually exclusive ?
Currently upnp only forward via UDP. Should he also forward via TCP ?
Why dont we only use UDP ?
No error should be raised when no upnp is detected : we should allow
machines having public IP to do an automatic configuration using the
discovery by an other peer
G : Actually, i was wrong, --ip and internal-port are no longer exclusive
Julien said udp might not be used by some people because of
restrictions imposed by the ISP ( FAI in french ), so we should
allow both, and act according to the options specifying which servers
to start (upd, tcp-server)
G : I think the number of route going through an interface should be a
Connection attribute, not a dict in tunnelManager
U : Yes, it was planned, just wait for me to finish implementing it
U : '--up', 'ovpn-server %s/%u' % (server_ip, len(network)) in plib.py
if you use len(network), this means that all our network is on the
same LAN and that the interface of the server is connected to it
wich means that any packet should be routed to this interface
an interface should only advertise the /64 (or less) which has been
attributed to it
......@@ -22,7 +22,7 @@ def server(server_ip, ip_length, max_clients, dh_path, pipe_fd, port, proto, hel
return openvpn(hello_interval,
'--tls-server',
'--mode', 'server',
'--up', 'ovpn-server %s/%u' % (server_ip, ip_length),
'--up', 'ovpn-server %s/%u' % (server_ip, 64),
'--client-connect', 'ovpn-server ' + str(pipe_fd),
'--client-disconnect', 'ovpn-server ' + str(pipe_fd),
'--dh', dh_path,
......
......@@ -2,31 +2,64 @@
#include <cmath>
#include <map>
#include <queue>
#include <stack>
void erase(vector<int>& v, int value)
{
for(int i=0;;i++)
if(v[i] == value)
{
v[i] = v.back();
v.pop_back();
break;
}
}
Graph::Graph(int size, int k, int maxPeers, mt19937& rng) :
distrib(uniform_int_distribution<int>(0, size-1)),
size(size), generator(rng)
size(size), generator(rng), k(k), maxPeers(maxPeers)
{
adjacency = new vector<int>[size];
generated = new vector<int>[size];
for(int i=0; i<size; i++)
{
int otherNode;
unordered_set<int> alreadyConnected;
alreadyConnected.insert(i);
for(int j=0; j<k; j++)
{
int otherNode;
while(alreadyConnected.count(otherNode = distrib(rng)) == 1
|| otherNode > i && adjacency[otherNode].size() > maxPeers-10
while(alreadyConnected.count(otherNode = distrib(generator)) >= 1
|| otherNode > i && adjacency[otherNode].size() > maxPeers-k
|| adjacency[otherNode].size() > maxPeers)
{ }
adjacency[i].push_back(otherNode);
generated[i].push_back(otherNode);
adjacency[otherNode].push_back(i);
}
}
}
int Graph::AddEdge(int from, unordered_set<int>& alreadyConnected)
{
int otherNode;
while(alreadyConnected.count(otherNode = distrib(generator)) >= 1
|| (adjacency[otherNode].size() > maxPeers ))
{ }
adjacency[from].push_back(otherNode);
generated[from].push_back(otherNode);
adjacency[otherNode].push_back(from);
return otherNode;
}
int Graph::RemoveEdge(int from, int to)
{
erase(generated[from], to);
erase(adjacency[from], to);
erase(adjacency[to], from);
}
void Graph::GetDistancesFrom(int node, int* distance)
{
for(int j=0; j<size; j++)
......@@ -50,6 +83,54 @@ void Graph::GetDistancesFrom(int node, int* distance)
}
}
void Graph::GetRoutesFrom(int node, int* distance, float* routesCount)
{
unordered_set<int> prevs[size];
for(int i=0; i<size; i++)
{
distance[i] = -1;
routesCount[i] = 1;
}
distance[node] = 0;
queue<int> remainingNodes;
remainingNodes.push(node);
stack<int> order;
// Get the order
while(!remainingNodes.empty())
{
int node = remainingNodes.front();
int d = distance[node];
remainingNodes.pop();
order.push(node);
for(int neighbor : adjacency[node])
if(distance[neighbor] == -1)
{
distance[neighbor] = d+1;
prevs[neighbor].insert(node);
remainingNodes.push(neighbor);
}
else if(distance[neighbor] == d+1)
prevs[neighbor].insert(node);
}
// get the BC
while(!order.empty())
{
int node = order.top();
order.pop();
float w = routesCount[node];
for(int i : prevs[node])
routesCount[i] += w;
}
}
int Graph::CountUnreachableFrom(int node)
{
bool accessibility[size];
......
// To compile : g++ -std=c++0x results.cpp graph.cpp main.cpp -lpthread
// To compile : g++ -std=c++0x results.cpp graph.cpp main.cpp -lpthread
#include "main.h"
#include <future>
#include <sstream>
......@@ -14,8 +14,8 @@ Results Simulate(int seed, int n, int k, int maxPeer, int maxDistanceFrom, floa
for(int r=0; r<runs; r++)
{
Graph graph(n, k, maxPeer, rng);
graph.KillMachines(alivePercent);
results.AddAccessibilitySample(((double)graph.CountUnreachableFrom(0))/((double)n));
//graph.KillMachines(alivePercent);
//results.AddAccessibilitySample(((double)graph.CountUnreachableFrom(0))/((double)n));
//int minCut = graph.GetMinCut();
//if(results.minKConnexity == -1 || results.minKConnexity > minCut)
//results.minKConnexity = minCut;
......@@ -28,6 +28,56 @@ Results Simulate(int seed, int n, int k, int maxPeer, int maxDistanceFrom, floa
graph.GetDistancesFrom(i, distance);
results.UpdateDistance(distance, graph.size);
}*/
int distance[graph.size];
float routesCount[graph.size];
int nRefresh = 1;
graph.GetDistancesFrom(0, distance);
double moy = 0;
for(int i=0; i<graph.size; i++)
moy += distance[i];
moy /= graph.size;
cout << "Avg distance : " << moy << endl; cout.flush();
for(int i = 0; i<100; i++)
{
for(int j=0; j<graph.size; j++)
{
graph.GetRoutesFrom(j, distance, routesCount);
unordered_set<int> alreadyConnected;
// erase some edge
for(int k=0; k<nRefresh; k++)
{
int minNode = -1;
int minimum = -1;
for(int index = 0; index < graph.generated[j].size(); index++)
if(minNode == -1 || routesCount[graph.generated[j][index]] < minimum)
{
minNode = graph.generated[j][index];
minimum = routesCount[minNode];
}
graph.RemoveEdge(j, minNode);
}
// Add new edges
alreadyConnected.insert(j);
for(int k : graph.adjacency[j])
alreadyConnected.insert(k);
for(int k=0; k<nRefresh; k++)
alreadyConnected.insert(graph.AddEdge(j, alreadyConnected));
}
graph.GetDistancesFrom(0, distance);
moy = 0;
for(int i=0; i<graph.size; i++)
moy += distance[i];
moy /= graph.size;
cout << "Avg distance : " << moy << endl;
}
}
results.Finalise();
......@@ -46,12 +96,12 @@ int main(int argc, char** argv)
vector<future<string>> outputStrings;
for(int n=2000; n<=2000; n*=2)
for(int k=10; k<=10; k+=5)
for(float a=0.01; a<=1; a+=0.05)
for(float a=1; a<=1; a+=0.05)
{
int seed = rng();
outputStrings.push_back(async(launch::async, [seed, n, k, a]()
{
Results results = Simulate(seed, n, k, 3*k, 10000, a, 1);
Results results = Simulate(seed, n, k, 2.5*k, 10000, a, 1);
ostringstream out;
out << n << "," << k << "," << a << "," << 3*k << ","
<< results.avgDistance << ","
......
......@@ -35,22 +35,28 @@ class Graph
{
public:
Graph(int size, int k, int maxPeers, mt19937& rng);
~Graph() { delete[] adjacency; };
~Graph() { delete[] adjacency; delete[] generated; };
void GetDistancesFrom(int node, int* distance);
int AddEdge(int from, unordered_set<int>& alreadyConnected);
int RemoveEdge(int from, int to);
int GetMinCut();
int CountUnreachableFrom(int node);
void GetRoutesFrom(int node, int* distance, float* routesCount);
void KillMachines(float proportion);
//void SplitAS(float proportionAS1, float proportionAS2);
vector<int>* adjacency;
vector<int>* generated;
int size;
private:
int GetMinCut(MinCutGraph& graph);
uniform_int_distribution<int> distrib;
mt19937& generator;
int maxPeers;
int k;
};
class Results
......
......@@ -14,6 +14,7 @@ class Connection:
os.O_WRONLY|os.O_CREAT|os.O_TRUNC))
self.iface = iface
self.routes = 0
self._prefix = prefix
self._creation_date = time.time()
self._bandwidth = None
......@@ -64,7 +65,7 @@ class TunnelManager:
self._write_pipe = write_pipe
self._peer_db = peer_db
self._connection_dict = {}
self._route_count = {}
self._iface_to_prefix = {}
self._ovpn_args = openvpn_args
self._hello = hello_interval
self._refresh_time = refresh
......@@ -87,6 +88,7 @@ class TunnelManager:
def _cleanDeads(self):
for prefix in self._connection_dict.keys():
if not self._connection_dict[prefix].refresh():
self._kill(prefix)
......@@ -106,7 +108,7 @@ class TunnelManager:
pass
self.free_interface_set.add(connection.iface)
self._peer_db.unusePeer(prefix)
del self._route_count[connection.iface]
del self._iface_to_prefix[connection.iface]
def _makeNewTunnels(self):
utils.log('Trying to make %i new tunnels' %
......@@ -119,7 +121,7 @@ class TunnelManager:
self._connection_dict[prefix] = Connection(address,
self._write_pipe, self._hello, iface,
prefix, self._ovpn_args)
self._route_count[iface] = 0
self._iface_to_prefix[iface] = prefix
self._peer_db.usePeer(prefix)
except KeyError:
utils.log("""Can't establish connection with %s
......@@ -129,16 +131,18 @@ class TunnelManager:
def _countRoutes(self):
utils.log('Starting to count the routes on each interface', 3)
for iface in self._route_count.keys():
self._route_count[iface] = 0
for iface in self._iface_to_prefix.keys():
self._connection_dict[self._iface_to_prefix[iface]].routes = 0
f = open('/proc/net/ipv6_route', 'r')
for line in f:
ip, subnet_size, iface = struct.unpack("""32s x 2s x 32x x 2x x
32x x 8x x 8x x 8x x 8x x %ss x""" % (len(line)-142), line)
ip, subnet_size, iface = struct.unpack("""32s x 2s 106x
%ss x""" % (len(line)-142), line)
iface = iface.replace(' ', '')
if iface in self._route_count.keys():
self._route_count[iface] += 1
for iface in self._route_count.keys():
utils.log('Routes on iface %s : %s' % (iface,self._route_count[iface] ), 5)
if iface in self._iface_to_prefix.keys():
self._connection_dict[self._iface_to_prefix[iface]].routes += 1
for p in self._connection_dict.keys():
utils.log('Routes on iface %s : %s' % (
self._connection_dict[p].iface,
self._connection_dict[p].routes ), 5)
......@@ -2,17 +2,32 @@ import miniupnpc
import socket
# return (address, port)
def ForwardViaUPnP(local_port):
def ForwardViaUPnP(local_port, protos):
u = miniupnpc.UPnP()
u.discoverdelay = 200
u.discover()
u.selectigd()
external_port = 1194
external_port = 1000
while True:
while u.getspecificportmapping(external_port, 'UDP') != None:
external_port = max(externalPort + 1, 49152)
if external_port == 65536:
raise Exception
if u.addportmapping(external_port, 'UDP', u.lanaddr, local_port, 'Vifib openvpn server', ''):
return (u.externalipaddress(), external_port)
if 'udp' in protos:
while u.getspecificportmapping(external_port, 'UDP') != None :
external_port += 1
if external_port == 65536:
raise Exception
if 'tcp-server' in protos:
while u.getspecificportmapping(external_port, 'TCP') != None :
external_port += 1
if external_port == 65536:
raise Exception
if 'udp' in protos:
u.addportmapping(external_port, 'UDP', u.lanaddr, local_port,
'Vifib openvpn server', '')
if 'tcp-server' in protos:
u.addportmapping(external_port, 'TCP', u.lanaddr, local_port,
'Vifib openvpn server', '')
print (u.externalipaddress(), external_port)
return (u.externalipaddress(), external_port)
......@@ -101,8 +101,8 @@ def main():
else:
utils.log('Attempting automatic configuration via UPnP', 4)
try:
ext_ip, ext_port = upnpigd.ForwardViaUPnP(config.internal_port)
config.address = list([ext_ip, ext_port, proto]
ext_ip, ext_port = upnpigd.ForwardViaUPnP(config.internal_port, config.proto)
config.address = list([ext_ip, str(ext_port), proto]
for proto in config.proto)
except Exception:
utils.log('An atempt to forward a port via UPnP failed', 4)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment