Instead of when we try to bind it check the usermm when
we try to use it in the IOCTLs.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>

Move that out of the main IOCTL function.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Non-scheduler code is longer supported.

v2: agd: rebased on upstream
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Ken Wang  <Qingqing.Wang@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

The other ones don't have any VAs assigned anyway or are uninteresting to us.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Make UVD/VCE VM emulation more efficient.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Stop copying that to the bo list entry, it doesn't change anyway.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Move that into the BO list. No functional change.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Rename it to amdgpu_cs_parser_bos and move validation and bo list init there.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Remove unused user_ptr field, group fields by usage.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

No need to actually check the current placement. Just use the allowed domains
when the threshold is reached.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

The threshold should only be computed once.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

This fixes a race condition in the error case: since the pt bos have not
necessarily been reserved in case of an error, we could move a pt bo that
is currently in the middle of being evicted/moved by another process,
which then resulted in a BUG_ON in ttm_bo_add_to_lru.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Most VM BOs end up in the duplicates list, validate it
first make -ENOMEM less likely.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <David1.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

This makes it less likely to run into an ENOMEM because
VM page tables are evicted last.

v2: move the BOs in the LRU tail after validation
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

This avoids allocating it on the fly.

v2: fix grammar in comment
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>

This way we avoid the extra allocation for the page directory entry.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>

They share the reservation object with the page directory anyway.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>

This fixes a random corruption under memory pressure. We need to fence
the BO for the user fence as well, otherwise it might be swapped out
and the GPU could write the fence value to an undesired location.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

No need for a GEM reference here.
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

Unify the two code path again, cause they do pretty much the same thing.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <davdi1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>

We need to clear parser.ibs and num_ibs before amd_sched_fence_create,
otherwise the IB could be freed twice if fence creates fails.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>

Before this patch the scheduler fence was created when we push the job
into the queue, so we could only get the fence after pushing it.

The mutex now was necessary to prevent the thread pushing the jobs to
the hardware from running faster than the thread pushing the jobs into
the queue.

Otherwise the thread pushing jobs into the queue would have accessed
possible freed up memory when it tries to get a reference to the fence.

So what you get in the end is thread A:
mutex_lock(&job->lock);
...
Kick of thread B.
...
mutex_unlock(&job->lock);

And thread B:
mutex_lock(&job->lock);
....
mutex_unlock(&job->lock);
kfree(job);

I'm actually not sure if I'm still up to date on this, but this usage
pattern used to be not allowed with mutexes. See here as well
https://lwn.net/Articles/575460/.

v2: remove unrelated changes, fix missing owner
v3: rebased, add more commit message
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

OGL needs these tracepoints to investigate performance issue.

Change-Id: I5e58187d061253f7d665dfce8e4e163ba91d3e2b
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>

No need any more to allocate that structure dynamically, just put it on the
stack. This is a start to cleanup some of the scheduler fallouts.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

Fixing a memory leak when the scheduler is enabled.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

We recently changed the locking in this function and now there is a
missing unlock on error.  Also there are some other resources that we
should probably release as well...

Fixes: f48b2659 ('drm/amdgpu: fix the broken vm->mutex V2')
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>

fix the vm->mutex and ww_mutex confilcts.
vm->mutex is always token first, then ww_mutex.

V2: remove unneccessary checking for pt bo.

Change-Id: Iea56e183752c02831126d06d2f5b7a474a6e4743
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

Finally getting rid of it.
Signed-off-by: Christian König <christian.koenig@amd.com>

This makes it much easier to find when userspace misses to send some buffers.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

Rework the sdma structures in the driver to
consolidate all of the sdma info into a single
structure and allow for asics that may have
different numbers of sdma instances.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

The new amdgpu driver passes a user space pointer in a 64-bit structure
member, which is the correct way to do it, but it attempts to
directly cast it to a __user pointer in the kernel, which causes
a warning in three places:

drm/amd/amdgpu/amdgpu_cs.c: In function 'amdgpu_cs_parser_init':
drm/amd/amdgpu/amdgpu_cs.c:180:21: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  chunk_array_user = (uint64_t __user *)(cs->in.chunks);

This changes all three to add an intermediate cast to 'unsigned long'
as other drivers do. This avoids the warning and works correctly on
both 32-bit and 64-bit architectures.

Fixes: e60b344f ("drm/amdgpu: optimize amdgpu_parser_init")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

The "i" variable should be signed or it leads to a crash in the error
handling code.

Fixes: 1d263474 ('drm/amdgpu: unwind properly in amdgpu_cs_parser_init()')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

The amdgpu_cs_parser_init() function doesn't clean up after itself but
instead the caller uses a free everything function amdgpu_cs_parser_fini()
on failure.  This style of error handling is often buggy.  In this
example, we call "drm_free_large(parser->chunks[i].kdata);" when it is
an unintialized pointer or when "parser->chunks" is NULL.

I fixed this bug by adding unwind code so that it frees everything that
it allocates.

I also mode some other very minor changes:
1) Renamed "r" to "ret".
2) Moved the chunk_array allocation to the start of the function.
3) Removed some initializers which are no longer needed.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reported-by: Ilja Van Sprundel <ivansprundel@ioactive.com>
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Embed the scheduler into the ring structure instead of allocating it.
Use the ring name directly instead of the id.

v2: rebased, whitespace cleanup
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>

Just to be consistent with the other members.

v2: rename the ring member as well.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> (v1)
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>

Use container_of rather than casting.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: David Zhou <david1.zhou@amd.com>
Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>

Use consistent naming across functions.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: David Zhou <david1.zhou@amd.com>
Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>

This allows for multiple BOs to have the same reservation object.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

Some buffers (UVD/VM page tables) must be placed in VRAM,
but the byte restriction for moving buffers didn't took this
into account.

Port of radeon commit 4b095566.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>