Fixed issue board label filtering not removing labels

## What does this MR do?

When trying to remove a label filter it would actually add it instead making it impossible to remove the label filter without modifying the URL. This fixes that by correctly removing the label from the filter.

## What are the relevant issue numbers?

Closes #21417

See merge request !6208