During code reviews several instances of duplicate module unloading checks
were found.

Remove the duplicate checks.

Link: https://lore.kernel.org/r/20200421203354.49420-1-jsmart2021@gmail.comReviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com>
Signed-off-by: James Smart <jsmart2021@gmail.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>