Skip to content

L
libloc
Switch branch/tag
  1. 03 Mar, 2022 1 commit
    • Michael Tremer's avatar
      export: Fix filtering logic · 964633d9
      Michael Tremer authored 
      It is possible to filter for what kind of network should be exported.

This worked well when the filter list only contained country codes, or
when it only contained ASNs. If there was a mix, only networks that
match both (i.e. virtually nothing) matched.

This patch fixes that we will use for either of them.
Signed-off-by: default avatarMichael Tremer <michael.tremer@ipfire.org>
      964633d9
  3. 02 Mar, 2022 6 commits
  5. 01 Mar, 2022 2 commits
    • Michael Tremer's avatar
      ipset: Fix hash type for IPv6 · 27dc4fa5
      Michael Tremer authored 
      IPv6 exports could not be loaded because sets were created as type
"inet" instead of "inet6" which is fixed by this patch.
Signed-off-by: default avatarMichael Tremer <michael.tremer@ipfire.org>
      27dc4fa5
    • Michael Tremer's avatar
      ipset: Optimise hash table size · 47de14b0
      Michael Tremer authored 
      ipset uses a hash table internally which can be dynamically sized to
chose whether more space efficiency or performance is required.

Previously to this patch, we always set the size of the hash table to
1024 buckets. Having large sets with almost half a million entries, this
is not performing well since we will spend a lot of time in searching
the linked list.

This will probably perform even slower on systems with smaller cache
sizes like the IPFire Mini Appliance.

Having more buckets that are sparesely filled, will result in less
memory fetches at the cost of more wastage. Throughout the whole IPv4
set, this ranges from about 50 MB for a factor of 4, to about 100 MB for
a factor of 0.75.

Since memory of this quantity is cheap and since we want to increase
throughput, I have chosen to set the fill factor to 0.75.

Logistically, it is a little bit complicated to know this in advance
when we have to write the header, so we will write the entire file
first, and then come back to write the header again. This is required to
keep memory consumption down during the export.
Signed-off-by: default avatarMichael Tremer <michael.tremer@ipfire.org>
      47de14b0
  7. 25 Feb, 2022 1 commit
  9. 16 Feb, 2022 1 commit
  11. 11 Feb, 2022 3 commits
  13. 02 Jan, 2022 4 commits
  15. 13 Dec, 2021 4 commits
  17. 23 Nov, 2021 3 commits
  19. 20 Nov, 2021 4 commits
  21. 02 Nov, 2021 2 commits
    • Peter Müller's avatar
      location-importer.in: Add Spamhaus DROP lists · 69b3d894
      Peter Müller authored 
      A while ago, it was discussed whether or not libloc should become an
"opinionated database", i. e. including any information on a network's
reputation.

In general, this idea was dismissed as libloc is neither intended nor
suitable for such tasks, and we do not want to make (political?)
decisions like these for various reasons. All we do is to provide a
useful location database in a neutral way, and leave it up to our users
on how to react on certain results.

However, there is a problematic area. Take AS55303 as an example: We
_know_ this is to be a dirty network, tampering with RIR data and
hijacking IP space, and strongly recommend against processing any
connection originating from or directed to it.

Since it appears to be loaded with proxies used by miscreants for
abusive purposes, all we can do at the time of writing is to flag it
as "anonymous proxy", but we lack possibility of telling our users
something like "this is not a safe area". The very same goes for known
bulletproof ISPs, IP hijackers, and so forth.

This patch therefore suggests to populate the "is_drop" flag introduced
in libloc 0.9.8 (albeit currently unused in production) with the
contents of Spamhaus' DROP lists (https://www.spamhaus.org/drop/), to
have at least the baddest of the bad covered. The very same lists are,
in fact, included in popular IPS rulesets as well - a decent amount of
IPFire users is therefore likely to have them already enabled, but in a
very costly way.

It is not planned to go further, partly because there is no other feed
publicly available, which would come with the same intention,
volatility, and FP rate.

The third version of this patch makes use of an auxiliary function to
sanitise ASNs, hence avoiding boilerplate code, and treats any line
starting with a semicolon as a comment, which should be sufficient.
Further, extracting ASNs from the ASN-DROP feed is done in a more clear
way, avoiding code snippets hard to read.
Signed-off-by: default avatarPeter Müller <peter.mueller@ipfire.org>
Signed-off-by: default avatarMichael Tremer <michael.tremer@ipfire.org>
      69b3d894
    • Peter Müller's avatar
      location-importer: Introduce auxiliary function to sanitise ASNs · 43fe570c
      Peter Müller authored 
      The third version of this patch does this in an even more Pythonic way.
Signed-off-by: default avatarPeter Müller <peter.mueller@ipfire.org>
Signed-off-by: default avatarMichael Tremer <michael.tremer@ipfire.org>
      43fe570c
  23. 25 Oct, 2021 1 commit
  25. 11 Oct, 2021 1 commit
  27. 30 Sep, 2021 7 commits
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7