It doesn't make sense to have a per-device work queue, as the
scheduler should be called per file handler. Having a single
one causes failures if multiple streams are filtered by vim2m.

So, move it to be inside the context structure.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>