• Nicolai Stange's avatar
    lib/mpi: mpi_read_from_buffer(): return error code · 03cdfaad
    Nicolai Stange authored
    mpi_read_from_buffer() reads a MPI from a buffer into a newly allocated
    MPI instance. It expects the buffer's leading two bytes to contain the
    number of bits, followed by the actual payload.
    
    On failure, it returns NULL and updates the in/out argument ret_nread
    somewhat inconsistently:
    - If the given buffer is too short to contain the leading two bytes
      encoding the number of bits or their value is unsupported, then
      ret_nread will be cleared.
    - If the allocation of the resulting MPI instance fails, ret_nread is left
      as is.
    
    The only user of mpi_read_from_buffer(), digsig_verify_rsa(), simply checks
    for a return value of NULL and returns -ENOMEM if that happens.
    
    While this is all of cosmetic nature only, there is another error condition
    which currently isn't detectable by the caller of mpi_read_from_buffer():
    if the given buffer is too small to hold the number of bits as encoded in
    its first two bytes, the return value will be non-NULL and *ret_nread > 0.
    
    In preparation of communicating this condition to the caller, let
    mpi_read_from_buffer() return error values by means of the ERR_PTR()
    mechanism.
    
    Make the sole caller of mpi_read_from_buffer(), digsig_verify_rsa(),
    check the return value for IS_ERR() rather than == NULL. If IS_ERR() is
    true, return the associated error value rather than the fixed -ENOMEM.
    Signed-off-by: default avatarNicolai Stange <nicstange@gmail.com>
    Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
    03cdfaad
digsig.c 5.57 KB