Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

There is loc_network_has_flag which does exactly the same
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

UNRELEASED should not be left as-is when actually releasing.
The latest changelog entry now point at unstable instead.

The simple d/genchangelog.sh now does `dch -r ''` automatically
to ensure this distribution update doesn't get lost along the way
on future invocations.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

There was a discrepancy between the paths in the build environment and
the installed headers which could not be included in any third-party
software.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

These are used in multiple places, but there is no good place where
those should go, yet.

To make them usable in more places, I am temporarily moving them here.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This one apparently went down the drain between these two patches:
- https://patchwork.ipfire.org/project/location/patch/20210522125758.28770-1-peter.mueller@ipfire.org/
- https://patchwork.ipfire.org/project/location/patch/aefd1904-4b38-f5cf-ab1d-9d69636cf914@ipfire.org/

Due to other safeguards, the current damage in production is limited to:

location=# SELECT * FROM networks WHERE country = 'ZZ';
      network       | country | original_countries | source
--------------------+---------+--------------------+--------
 130.195.64.0/19    | ZZ      | {ZZ}               | APNIC
 130.195.96.0/19    | ZZ      | {ZZ}               | APNIC
 122.252.24.0/21    | ZZ      | {ZZ}               | APNIC
 2406:840:10::/48   | ZZ      | {ZZ}               | APNIC
 2406:840:fd0f::/48 | ZZ      | {ZZ}               | APNIC
(5 rows)

Fixes: #12673
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

A decent amount of autnum objects - especially, but not exclusively in
the APNIC sector - does not contain a link to an organisation handle.

In such cases, this patch is going to use the first description line of
the atunum object in question (if available) as a string for its name.
The overwhelming majority of affected ASNs contains a valuable
information there, so this is almost as good as having an organisation
handle linked to it.

Fixes: #12660
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

- Update for maintainer name and email address, as to reflect who
  actually prepared this release of the package. It was not me,
  but I was selected due to having the first commit on the package.

  To mitigate against this, when running the `debchange --release`
  (`dch -r`) command, environment variables DEBFULLNAME and DEBEMAIL
  should be configured properly for the current user.

- Removal of NMU comment on my name, as I am not really doing a
  non-maintainer upload. I would say the 'NMU' message is fairly
  useless on this repository, as it is self-maintained here.
Signed-off-by: Valters Jansons <valter.jansons@gmail.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This is what we do when we initialize the stringpool
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This commit further builds on historical changelog modifications,
to properly attribute all authors of the commits.

An additional d/genchangelog.sh script has been added. This allows
generation of changelog entries, internally using `debchange` (`dch`).
The script accepts an argument, which is the commit range to generate
entries for. Each commit's subject line (first line of body) is used,
along with author name and email. This information is added to the
changelog. Automatic detection (via `debchange` built-in functionality)
is used to determine whether these entries should be added to an
existing version number. If there is no UNRELEASED version, then a new
version is automatically tagged.

The new version tag will usually need to be modified, for example,
replacing an automatically generated 0.9.6-2 with 0.9.7-1.

The final release change (s/UNRELEASED/unstable/) needs to be done
manually as well, when the Git tag is actually being tagged.
`dch -r` can be useful for this particular purpose.
Signed-off-by: Valters Jansons <valter.jansons@gmail.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-import.in: optimise regular expression for filtering ASN allocations to other RIRs when parsing ARIN AS names file

https://lists.ipfire.org/pipermail/location/2021-June/000394.htmlReported-by: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

There are muiltiple standards of listing symbols throughout the Linux
ecosystem. For `dpkg`, a d/package.symbols file tracks symbols, and in
which version they were added in. This is then used to allow dependency
checks/resolution.

See man:dpkg-gensymbols(1) for details about the generation,
and man:dpkg-shlibdeps(1) for how the symbols file ends up being used.

This commit adds a d/libloc1.symbols file, containing the current state
of the symbols. There is now also a d/gensymbols.sh script, which
generates this symbols file. The script tries to determine what Git
tags need to be checked for changes in symbols, by looking at current
maximum version referenced in symbols file.

After checking tags, the current revision is also processed, to allow
building symbols file for a yet unreleased version (prior to tagging it).
This is to allow symbols changes to be included in a tag.

Do keep in mind, that for the workflow above, when running the script,
the d/changelog file should contain information about what version the
current revision will be released at (potentially tagged as UNRELEASED
in the d/changelog file). Otherwise, if there is no version tagged,
the `dpkg-gensymbols` tool will use the old version information,
in turn incorrectly attributing new symbols to an old version.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

When an invalid country code is entered, loc_country_new returns an
error which is interpreted as a match to the list since we check for a
non-zero return code.

Any invalid country codes are now silently ignored and not considered a
match.

Fixes: #12620 - "location list-networks-by-cc" returns garbage
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Amazon publishes information regarding some of their IP networks
primarily used for AWS cloud services in a machine-readable format. To
improve libloc lookup results for these, we have little choice other
than importing and parsing them.

Unfortunately, there seems to be no machine-readable list of the
locations of their data centers or availability zones available. If
there _is_ any, please let the author know.

The second version of this patch adds a meaningful description for the
"source" column in the overrides tables, to make introduced changes
less intransparent.

Fixes: #12594
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This allows us to track changes introduced by IP feeds from 3rd parties,
such as Amazon AWS, on the SQL server side.

In order not to break existing tables (which would required TRUNCATE),
there currently is no constraint set for the new column, but "NOT NULL"
is planned in the future.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

ARIN and LACNIC, unfortunately, do not seem to publish data containing
human readable AS names. For the former, we at least have a list of
tecnical names, which this patch fetches and inserts into the autnums
table.

While some of them do not seem to be suitable for human consumption (i.
e. being very cryptic), providing these data might be helpful
neverthelesss.

The second version of this patch contains some additional remarks on
efficient Python coding style from Michael, doing things more "pythonic".
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This patch implements an additional flag intended for networks and
Autonomous Systems being considered hostile. While libloc does not and
should not be an opinionated database, reality shows it is being used
this way.

Hereby, we assign "XD" (drop) as a custom country code for networks
being flagged this way. According to ISO, "XA" to "XZ" are reserved for
"user-assgined codes" (https://www.iso.org/glossary-for-iso-3166.html),
so this is a safe thing to do.

This patch does not interfere with "A1" to "A3", which we currently
assign outside standardised country code ranges for historical reasons.

Neither does it specify any policy or source for tagging networks with a
"drop" flag. Doing so is beyond the scope of this - technical -
approach.

To avoid confusions with the SQL "DROP" command, "is_drop" will be used
as a column name for database operations.

Thanks to Michael for his remarks and ideas during the run-up.

Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

https://lists.ipfire.org/pipermail/location/2021-May/000377.htmlReported-by: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This helps us to determine how many network objects have more than one
country set, and what their original country code set looked like.

The third version of this patch uses ALTER TABLE to add the column for
original countries, preventing existing SQL setups from breaking, and is
correctly based against the current "master" branch.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This allows us to trace back concrete changes or anomalies to their RIR
source, without having to parse everything again. Further, it enables
adding 3rd party sources such as IP feeds from Amazon, without loosing
track of the changes introduced by them.

The second version of this patchset uses ALTER TABLE to add the source
columns, avoiding breaking existing SQL setups.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Fixes: #12617
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

This reduces log spam in case of processing RIR database, checking for
networks with unknown country codes assigned. If we would not have
written into the database, there is no need to warn about them.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>