• Michael Tremer's avatar
    ipset: Optimise hash table size · 47de14b0
    Michael Tremer authored
    ipset uses a hash table internally which can be dynamically sized to
    chose whether more space efficiency or performance is required.
    
    Previously to this patch, we always set the size of the hash table to
    1024 buckets. Having large sets with almost half a million entries, this
    is not performing well since we will spend a lot of time in searching
    the linked list.
    
    This will probably perform even slower on systems with smaller cache
    sizes like the IPFire Mini Appliance.
    
    Having more buckets that are sparesely filled, will result in less
    memory fetches at the cost of more wastage. Throughout the whole IPv4
    set, this ranges from about 50 MB for a factor of 4, to about 100 MB for
    a factor of 0.75.
    
    Since memory of this quantity is cheap and since we want to increase
    throughput, I have chosen to set the fill factor to 0.75.
    
    Logistically, it is a little bit complicated to know this in advance
    when we have to write the header, so we will write the entire file
    first, and then come back to write the header again. This is required to
    keep memory consumption down during the export.
    Signed-off-by: default avatarMichael Tremer <michael.tremer@ipfire.org>
    47de14b0
export.py 6.92 KB