Commit c5ce7c69 authored by Nicolai Stange's avatar Nicolai Stange Committed by Herbert Xu

lib/digsig: digsig_verify_rsa(): return -EINVAL if modulo length is zero

Currently, if digsig_verify_rsa() detects that the modulo's length is zero,
i.e. mlen == 0, it returns -ENOMEM which doesn't really fit here.

Make digsig_verify_rsa() return -EINVAL upon mlen == 0.
Signed-off-by: default avatarNicolai Stange <nicstange@gmail.com>
Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
parent 03cdfaad
...@@ -114,13 +114,15 @@ static int digsig_verify_rsa(struct key *key, ...@@ -114,13 +114,15 @@ static int digsig_verify_rsa(struct key *key,
datap += remaining; datap += remaining;
} }
err = -ENOMEM;
mblen = mpi_get_nbits(pkey[0]); mblen = mpi_get_nbits(pkey[0]);
mlen = DIV_ROUND_UP(mblen, 8); mlen = DIV_ROUND_UP(mblen, 8);
if (mlen == 0) if (mlen == 0) {
err = -EINVAL;
goto err; goto err;
}
err = -ENOMEM;
out1 = kzalloc(mlen, GFP_KERNEL); out1 = kzalloc(mlen, GFP_KERNEL);
if (!out1) if (!out1)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment