ipset: Optimise hash table size
ipset uses a hash table internally which can be dynamically sized to
chose whether more space efficiency or performance is required.
Previously to this patch, we always set the size of the hash table to
1024 buckets. Having large sets with almost half a million entries, this
is not performing well since we will spend a lot of time in searching
the linked list.
This will probably perform even slower on systems with smaller cache
sizes like the IPFire Mini Appliance.
Having more buckets that are sparesely filled, will result in less
memory fetches at the cost of more wastage. Throughout the whole IPv4
set, this ranges from about 50 MB for a factor of 4, to about 100 MB for
a factor of 0.75.
Since memory of this quantity is cheap and since we want to increase
throughput, I have chosen to set the fill factor to 0.75.
Logistically, it is a little bit complicated to know this in advance
when we have to write the header, so we will write the entire file
first, and then come back to write the header again. This is required to
keep memory consumption down during the export.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Showing
Please register or sign in to comment