From: Heinrich Schuchardt Date: Wed, 31 Aug 2022 19:13:40 +0000 (+0200) Subject: lib: rsa: fix padding_pss_verify X-Git-Url: http://git.dujemihanovic.xyz/%22http:/www.sics.se/static/git-favicon.png?a=commitdiff_plain;h=0cd933bb4bd74084d942c42098ebf9e07d9e0f63;p=u-boot.git lib: rsa: fix padding_pss_verify Check the that the hash length is shorter than the message length. This avoids: ./tools/../lib/rsa/rsa-verify.c:275:11: warning: ‘*db’ may be used uninitialized [-Wmaybe-uninitialized] 275 | db[0] &= 0xff >> leftmost_bits; Fixes: 061daa0b61f0 ("rsa: add support of padding pss") Signed-off-by: Heinrich Schuchardt Reviewed-by: Simon Glass --- diff --git a/lib/rsa/rsa-verify.c b/lib/rsa/rsa-verify.c index 1d95cfbdee..9605c37639 100644 --- a/lib/rsa/rsa-verify.c +++ b/lib/rsa/rsa-verify.c @@ -215,6 +215,8 @@ out: * @msg_len: Message length * @hash: Pointer to the expected hash * @hash_len: Length of the hash + * + * Return: 0 if padding is correct, non-zero otherwise */ int padding_pss_verify(struct image_sign_info *info, const uint8_t *msg, int msg_len, @@ -234,6 +236,9 @@ int padding_pss_verify(struct image_sign_info *info, uint8_t leftmost_mask; struct checksum_algo *checksum = info->checksum; + if (db_len <= 0) + return -EINVAL; + /* first, allocate everything */ db_mask = malloc(db_len); db = malloc(db_len);