• Coly Li's avatar
    bcache: avoid extra memory consumption in struct bbio for large bucket size · 4e4d4e09
    Coly Li authored
    Bcache uses struct bbio to do I/Os for meta data pages like uuids,
    disk_buckets, prio_buckets, and btree nodes.
    
    Example writing a btree node onto cache device, the process is,
    - Allocate a struct bbio from mempool c->bio_meta.
    - Inside struct bbio embedded a struct bio, initialize bi_inline_vecs
      for this embedded bio.
    - Call bch_bio_map() to map each meta data page to each bv from the
      inlined  bi_io_vec table.
    - Call bch_submit_bbio() to submit the bio into underlying block layer.
    - When the I/O completed, only release the struct bbio, don't touch the
      reference counter of the meta data pages.
    
    The struct bbio is defined as,
    738 struct bbio {
    739     unsigned int            submit_time_us;
    	[snipped]
    748     struct bio              bio;
    749 };
    
    Because struct bio is embedded at the end of struct bbio, therefore the
    actual size of struct bbio is sizeof(struct bio) + size of the embedded
    bio->bi_inline_vecs.
    
    Now all the meta data bucket size are limited to meta_bucket_pages(), if
    the bucket size is large than meta_bucket_pages()*PAGE_SECTORS, rested
    space in the bucket is unused. Therefore the most used space in meta
    bucket is (1<<MAX_ORDER) pages, or (1<<CONFIG_FORCE_MAX_ZONEORDER) if it
    is configured.
    
    Therefore for large bucket size, it is unnecessary to calculate the
    allocation size of mempool c->bio_meta as,
    	mempool_init_kmalloc_pool(&c->bio_meta, 2,
    			sizeof(struct bbio) +
    			sizeof(struct bio_vec) * bucket_pages(c))
    It is too large, neither the Linux buddy allocator cannot allocate so
    much continuous pages, nor the extra allocated pages are wasted.
    
    This patch replace bucket_pages() to meta_bucket_pages() in two places,
    - In bch_cache_set_alloc(), when initialize mempool c->bio_meta, uses
      sizeof(struct bbio) + sizeof(struct bio_vec) * bucket_pages(c) to set
      the allocating object size.
    - In bch_bbio_alloc(), when calling bio_init() to set inline bvec talbe
      bi_inline_bvecs, uses meta_bucket_pages() to indicate number of the
      inline bio vencs number.
    
    Now the maximum size of embedded bio inside struct bbio exactly matches
    the limit of meta_bucket_pages(), no extra page wasted.
    Signed-off-by: default avatarColy Li <colyli@suse.de>
    Reviewed-by: default avatarHannes Reinecke <hare@suse.de>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    4e4d4e09
io.c 4.22 KB