This simplifies the driver to stop using the deprecated attach interface.

While at it we also implement teardown properly and fix the refcounting
by using a kref.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>