- 27 Oct, 2020 8 commits
-
-
Michael Tremer authored
This patch removes the possibility that any IP address ranges might show up in multiple exported files. If this was content from the database: * 10.0.0.0/16 - DE * 10.0.1.0/24 - FR Then the IP address 10.0.1.1 would match for both countries. The algorithm will now break the larger /16 subnet apart into smaller subnets so that 10.0.1.0/24 is no longer overlapped. There was some time spent on this to make this as efficient as possible. Signed-off-by:Michael Tremer <michael.tremer@ipfire.org>
-
Peter Müller authored
Signed-off-by:
Peter Müller <peter.mueller@ipfire.org> Signed-off-by:
-
Peter Müller authored
Some RIRs include detailled information regarding networks not managed by or allocated to themselves, particually APNIC. We need to filter those networks (they usually have a characteristic network name) in order to prevent operational quirks or returning wrong country codes. Fixes: #12501 Partially fixes: #12499 Cc: Michael Tremer <michael.tremer@ipfire.org> Signed-off-by:
-
Peter Müller authored
Signed-off-by:
-
Peter Müller authored
Sanity checks for parsed networks have been put into a separate function to avoid boilerplate code for extended sources. This makes the location database less vulnerable to garbage written into RIR databases on purpose or by chance. Fixes: #12500 Signed-off-by:
-
Peter Müller authored
In contrast to the location database itself, the xt_geoip module consumes a list of IP networks for each country, and returns after the first match. We therefore need to... (a) sort IP networks by their size, allow as precise matches as possible (b) export _any_ IP networks - including inverted subnets - to prevent undefined overlaps (c) do the entire thing as fast as possible, consuming as less disk space as possible, which is why we can't just iterate over all /24 chunks Partially fixes: #12499 Signed-off-by: -
Peter Müller authored
This reverts commit 13f67f28. Signed-off-by:
-
Peter Müller authored
This reverts commit 44341478. Signed-off-by:
-
- 22 Oct, 2020 1 commit
-
-
Michael Tremer authored
Signed-off-by:
-
- 21 Oct, 2020 10 commits
-
-
Michael Tremer authored
There are no functional changes, this just runs quicker now. Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
loc_network_new_from_string() seem to have had some unexpected behaviour for invalid inputs. The function has been tidied up slightly and returns as soon as some invalid input was detected. Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
Signed-off-by:
-
- 20 Oct, 2020 1 commit
-
-
Michael Tremer authored
Reported-by:
Gisle Vanem <gisle.vanem@gmail.com> Signed-off-by:
-
- 16 Oct, 2020 3 commits
-
-
Michael Tremer authored
This reverts commit f532841e. Signed-off-by:
-
Michael Tremer authored
This reverts commit a36bc686. Signed-off-by:
-
Michael Tremer authored
Signed-off-by:
-
- 12 Oct, 2020 2 commits
-
-
Peter Müller authored
In contrast to ARIN and LACNIC, we are able to process more detailled feeds from those RIRs, avoiding storage of obviously unnecessary data. Thanks to various SQL optimisations, doing so now takes less time than the first version of this did. Signed-off-by:
-
Peter Müller authored
This reverts commit 92f6abf4. Signed-off-by:
-
- 24 Sep, 2020 2 commits
-
-
Michael Tremer authored
This reverts commit 64e95fa9. Signed-off-by:
-
Michael Tremer authored
This reverts commit c2cc55d5. The query stalls the database and therefore the automatic scripts are no longer able to generate a new version of the database. Signed-off-by:
-
- 21 Sep, 2020 4 commits
-
-
Michael Tremer authored
Signed-off-by: -
Peter Müller authored
When importing inetnums, we might import various small networks which are not relevant for us as long as they do not have a different country code than their parent network. Therefore we delete all these entries to keep the database smaller without losing any information. The second version of this patch introduces a SQL statement parallelised across all CPUs available, while the DELETE-statement of the first version literally took ages to complete. However, cleaning up those data still takes about 26 hours (!) on our location02 testing machine, making daily updates of the location database impossible to the current knowledge. real 1521m30.620s user 38m45.521s sys 9m6.027s Special thanks goes to Michael for spending numerous hours on this, setting up a testing environment, doing PostgreSQL magic and providing helpful advice while debugging. Partially fixes: #12458 Cc: Michael Tremer <michael.tremer@ipfire.org> Signed-off-by:
-
Peter Müller authored
The extended feeds do not have enough detailed information for us, so that we need to import inetnums from RIRs where possible. Filtering private networks is necessary as RIR data may contain 0.0.0.0/0 or similar entries for administrative purposes or due to misfilings. Special thanks goes to Michael for spending numerous hours on this, setting up a testing environment and providing helpful advice while debugging. Partially fixes: #12458 Cc: Michael Tremer <michael.tremer@ipfire.org> Signed-off-by:
-
Peter Müller authored
Signed-off-by:
-
- 18 Sep, 2020 6 commits
-
-
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
We can use this to avoid formatting the network in a string representation which is then parsed again later to determine the first and last addresses. Signed-off-by: -
Michael Tremer authored
This avoids recomputing the last address multiple times. Signed-off-by: -
Michael Tremer authored
Signed-off-by: -
Michael Tremer authored
It is unnecessary to write any subnets in the output, because they might already be included in a bigger parent network. Since the tree is ordered, we only need to check if the network to be written is a subnet of the previous one. If so, we skip it. Signed-off-by: -
Stefan Schantl authored
Signed-off-by:
Stefan Schantl <stefan.schantl@ipfire.org> Signed-off-by:
-
- 10 Sep, 2020 3 commits
-
-
Stefan Schantl authored
Signed-off-by:
-
Stefan Schantl authored
Signed-off-by:
-
Stefan Schantl authored
Signed-off-by:
-
