New flush method.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GP102.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GV100.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with NV50.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GK20A.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GF100.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Things are a bit different here on Turing, and will require further changes
yet once I've investigated them more thoroughly.

For now though, the existing GP100 code is compatible enough with one small
hack to forward on fault buffer interrupts.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GM107.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GM200.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GK104.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GM200.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GK104.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

The GPU executes DEVINIT itself now, which makes our lives a bit easier.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

No real surprises here so far.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Appears to be compatible with GP100.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Will be required for Turing.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

The token will also contain runlist ID on Turing, so instead expose it as
an opaque value from NVKM so the client doesn't need to care.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

The GPU saves off some stuff to the address specified in this part of RAMFC
when the channel faults, so we should probably point it at a valid address.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

The trick we used (and still use for older GPUs) doesn't work on Turing.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Turing will require different code.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

We're about to be adding more of them.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

We will need to bash different registers on Turing.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Will be used by SVM code to allow direct (without going through MMU) memcpy
using the GPU copy engines.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Will be used to match fault buffer entries with a channel.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

This is needed for Turing, but we're supposed to wait for completion after
re-writing the value on older GPUs anyway.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Aside from being a nice cleanup, these will to allow the upcoming direct
page mapping interfaces to play nicely with normal mappings.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

The GPU will continually fire interrupts while a fault buffer GET != PUT,
and to stop the spurious interrupts while the handler does its thing, we
were disabling the fault buffer temporarily.

This is not actually a great idea to begin with, and made worse by Volta
resetting GET/PUT when it's reactivated.  So, let's not do that.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Will allow more shared fault buffer handling code between Pascal/Volta.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Various structures are accessed by the GPU through BAR2 for some reason
on newer GPUs.  This commit makes it more convenient to handle.

Will be used for GP100- fault buffers, and GV100- fault method buffers.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Will be used for Turing.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Turing GPUs can have more than one.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

This makes debugging with DP tracing a lot harder to interpret, so name
each i2c based off the name of the encoder that it's for
Signed-off-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Karol Herbst <karolherbst@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

We need to actually make sure we check this on resume since otherwise we
won't know whether or not the topology is still there once we've
resumed, which will cause us to still think the topology is connected
even after it's been removed if the removal happens mid-suspend.
Signed-off-by: Lyude Paul <lyude@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>