The pool parameter can be NULL if we free through the shrinker.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Reported-by: Andy Lavr <andy.lavr@gmail.com>
Tested-by: Andy Lavr <andy.lavr@gmail.com>
Link: https://patchwork.freedesktop.org/patch/399365/
Currently drivers get called to move a buffer, but if they have to
move it temporarily through another space (SYSTEM->VRAM via TT)
then they can end up with a lot of ttm->driver->ttm call stacks,
if the temprorary space moves requires eviction.
Instead of letting the driver do all the placement/space for the
temporary, allow it to report back (-EMULTIHOP) and a placement (hop)
to the move code, which will then do the temporary move, and the
correct placement move afterwards.
This removes a lot of code from drivers, at the expense of
adding some midlayering. I've some further ideas on how to turn
it inside out, but I think this is a good solution to the call
stack problems.
v2: separate out the driver patches, add WARN for getting
MULTHOP in paths we shouldn't (Daniel)
v3: use memset (Christian)
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: hristian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201109005432.861936-2-airlied@gmail.com
The new functions ttm_bo_{vmap,vunmap}() map and unmap a TTM BO in kernel
address space. The mapping's address is returned as struct dma_buf_map.
Each function is a simplified version of TTM's existing kmap code. Both
functions respect the memory's location ani/or writecombine flags.
On top TTM's functions, GEM TTM helpers got drm_gem_ttm_{vmap,vunmap}(),
two helpers that convert a GEM object into the TTM BO and forward the call
to TTM's vmap/vunmap. These helpers can be dropped into the rsp GEM object
callbacks.
v5:
* use size_t for storing mapping size (Christian)
* ignore premapped memory areas correctly in ttm_bo_vunmap()
* rebase onto latest TTM interfaces (Christian)
* remove BUG() from ttm_bo_vmap() (Christian)
v4:
* drop ttm_kmap_obj_to_dma_buf() in favor of vmap helpers (Daniel,
Christian)
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Tested-by: Sam Ravnborg <sam@ravnborg.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20201103093015.1063-6-tzimmermann@suse.de
The ttm_operation_ctx structure has a mixture of flags and bools. Drop the
flags and replace them with bools as well.
v2: fix typos, improve comments
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/398686/
During eviction we do want to trigger the OOM killer.
Only while doing new allocations we should try to avoid that and
return -ENOMEM to the application.
v2: rename the flag to gfp_retry_mayfail.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/398685/
Not used any more.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/397087/?series=83051&rev=1
Provide the necessary parameters by all drivers and use the new pool alloc
when no driver specific function is provided.
v2: fix the GEM VRAM helpers
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/397081/?series=83051&rev=1
This replaces the spaghetti code in the two existing page pools.
First of all depending on the allocation size it is between 3 (1GiB) and
5 (1MiB) times faster than the old implementation.
It makes better use of buddy pages to allow for larger physical contiguous
allocations which should result in better TLB utilization at least for
amdgpu.
Instead of a completely braindead approach of filling the pool with one
CPU while another one is trying to shrink it we only give back freed
pages.
This also results in much less locking contention and a trylock free MM
shrinker callback, so we can guarantee that pages are given back to the
system when needed.
Downside of this is that it takes longer for many small allocations until
the pool is filled up. We could address this, but I couldn't find an use
case where this actually matters. We also don't bother freeing large
chunks of pages any more since the CPU overhead in that path isn't really
that important.
The sysfs files are replaced with a single module parameter, allowing
users to override how many pages should be globally pooled in TTM. This
unfortunately breaks the UAPI slightly, but as far as we know nobody ever
depended on this.
Zeroing memory coming from the pool was handled inconsistently. The
alloc_pages() based pool was zeroing it, the dma_alloc_attr() based one
wasn't. For now the new implementation isn't zeroing pages from the pool
either and only sets the __GFP_ZERO flag when necessary.
The implementation has only 768 lines of code compared to the over 2600
of the old one, and also allows for saving quite a bunch of code in the
drivers since we don't need specialized handling there any more based on
kernel config.
Additional to all of that there was a neat bug with IOMMU, coherent DMA
mappings and huge pages which is now fixed in the new code as well.
v2: make ttm_pool_apply_caching static as reported by the kernel bot, add
some more checks
v3: fix some more checkpatch.pl warnings
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/397080/?series=83051&rev=1
It makes no difference to kmalloc if the structure
is 48 or 64 bytes in size.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/396950/
This is not related to allocating the backing store in any way.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/396947/
Neither page allocation backend nor the driver should mess with that.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Link: https://patchwork.freedesktop.org/patch/396948/
The move notify callback is only used in one place, this should
be removed in the future, but for now just rename it to the use
case which is to notify the driver that the GPU memory is to be
deleted.
Drivers can be cleaned up after this separately.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201021044031.1752624-2-airlied@gmail.com
This moves the to system move into the drivers, and moves all
the unbinds in the move path under driver control
Note: radeon/nouveau already wait so don't duplicate it.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201020010319.1692445-4-airlied@gmail.com
In all 3 drivers there is a case where the driver knows the
bo is in SYSTEM so don't call the api that checks that.
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201019071314.1671485-4-airlied@gmail.com
Changing the caching on the fly never really worked
flawlessly.
So stop this completely and just let drivers specific the
desired caching in the tt or bus object.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/394256/
Instead of the placement flags use the caching of the bus
mapping or tt object for the page protection flags.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/394255/
All drivers can determine the tt caching state at creation time,
no need to do this on the fly during every validation.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/394253/
This is not something drivers should use.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/393430/
It is the sole user of this.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/393498/
We can always access the global state.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/393499/
Make it more clear what the resource manager function
does and nuke the wrapper function.
v2: nuke the wrapper
v3: fix typo in radeon, rebased
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> (v2)
Link: https://patchwork.freedesktop.org/patch/393914/
Remove unused parameters, shorten the function name.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/393431/
Just check earlier if a BO can be page faulted in the first place.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/392321/
As an alternative to the placement flag add a
pin count to the ttm buffer object.
v2: add dma_resv_assert_help() calls
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/391596/?series=81973&rev=1
When we swapout/in a BO we try to change the caching
attributes of the pages before/after doing the copy.
On x86 this is done by calling set_pages_uc(),
set_memory_wc() or set_pages_wb() for not highmem pages
to update the linear mapping of the page.
On all other platforms we do exactly nothing.
Now on x86 this is unnecessary because copy_highpage() will
either create a temporary mapping of the page which is wb
anyway and destroyed immediately again or use the linear
mapping with the correct caching attributes.
So stop this nonsense and just keep the caching as it is and
return an error when a driver tries to change the caching of
an already populated TT object.
This is much more defensive since changing caching
attributes is platform and driver specific and usually
doesn't work after the page was initially allocated.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/391293/
Update kernel-doc line comments to fix warnings reported by make W=1.
drivers/gpu/drm/ttm/ttm_memory.c:271: warning: Function parameter or
member 'glob' not described in 'ttm_shrink'
drivers/gpu/drm/ttm/ttm_memory.c:271: warning: Function parameter or
member 'from_wq' not described in 'ttm_shrink'
drivers/gpu/drm/ttm/ttm_memory.c:271: warning: Function parameter or
member 'extra' not described in 'ttm_shrink'
drivers/gpu/drm/ttm/ttm_memory.c:271: warning: Function parameter or
member 'ctx' not described in 'ttm_shrink'
Signed-off-by: Tian Tao <tiantao6@hisilicon.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/391317/
Signed-off-by: Christian König <christian.koenig@amd.com>
This moves the generic tracking into the drivers and protects
against reentrancy in the drivers. It fixes up radeon and agp
to be able to query the bound status as that is required.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200917043040.146575-2-airlied@gmail.com
Unexport ttm_check_under_lowerlimit.
Make ttm_bo_acc_size static and unexport it.
Remove ttm_get_kernel_zone_memory_size.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/390515/
Move bound up into the bo object, and keep populated with the tt
object.
The ghost object handling needs to follow the flags at the bo
level now instead of it being part of the ttm tt object.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200915024007.67163-7-airlied@gmail.com
This adds 2 getters and 4 setters, however unbound and populated
are currently the same thing, this will change, it also drops
a BUG_ON that seems not that useful.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200915024007.67163-2-airlied@gmail.com
Instead of letting TTM make an educated guess based on
some mask all drivers should just specify what caching
they want for their CPU mappings.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/390207/
As far as I can tell this was never used either and we just
always fallback to the order cached > wc > uncached anyway.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/390142/
It's not supported to specify more than one of those flags.
So it never made sense to make this a flag in the first place.
Nuke the flags and specify directly which memory type to use.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/389826/?series=81551&rev=1
Since the agp bind/unbind/destroy are now getting called from drivers
rather than via the func table, drop the bdev parameter.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200907204630.1406528-13-airlied@gmail.com
This is used by TTM to communicate the physical address
which should be used with ioremap(), ioremap_wc(). We don't
need to separate the base and offset in any way here.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/389457/
-----BEGIN PGP SIGNATURE-----
iQFSBAABCAA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAl9VerweHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGhc4H/iHD6qLdB36gZB6K
oc2nJyrqyWitv4ti2Mnt5PA7o4wX4l6nnr1QvoaJ4BRs5Ja1czRvb2XDmdzqAoIA
xITGoafqaAeDfxQ91bWrJsVN0pCRKiGwddXlU7TWmqw/riAkfOqi6GYKvav4biJH
+n1mUPQb1M2IbRFsqkAS+ebKHq3CWaRvzKOEneS88nGlL5u31S9NAru8Ru/fkxRn
6CwGcs1XRaBPYaZAhdfIb0NuatUlpkhPC9yhNS9up6SqrWmK3m65vmFVng6H0eCF
fwn1jVztboY/XcNAi5sM9ExpQCql6WLQEEktVikqRDojC8fVtSx6W55tPt7qeaoO
Z6t4/DA=
=bcA4
-----END PGP SIGNATURE-----
Merge tag 'v5.9-rc4' into drm-next
Backmerge 5.9-rc4 as there is a nasty qxl conflict
that needs to be resolved.
Signed-off-by: Dave Airlie <airlied@redhat.com>
That is not used any more.
v2: keep the NULL checks in TTM.
v3: remove unused variable
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Link: https://patchwork.freedesktop.org/patch/388646/
We are trying to remove the io_lru handling and depend
on zero init base, offset and addr here.
v2: init addr as well
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Link: https://patchwork.freedesktop.org/patch/388642/
drm-next reverted the changes to ttm_tt_create() to do the
NULL check inside the function, but drm-misc-next adds new
users of this approach.
Re-apply the NULL check change inside the function to fix this.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/386628/
Backmerging drm-next into drm-misc-next for nouveau and panel updates.
Resolves a conflict between ttm and nouveau, where struct ttm_mem_res got
renamed to struct ttm_resource.
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
The names get/put are associated with reference counting
in the Linux kernel, use alloc/free instead.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/384340/?series=80346&rev=1
* add orientation quirk for ASUS T103HAF
* drm/omap: force runtime PM suspend on system suspend
* drm/tidss: fix modeset init for DPI panels
* re-added docs for drm_gem_flink_ioctl()
* ttm: fix page-offset calculation within TTM
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEchf7rIzpz2NEoWjlaA3BHVMLeiMFAl8pWYUACgkQaA3BHVML
eiNGkAf+Lw+DXDm1C9cpSm2oj9XFiwUBI25G+4aDn6+3o5YAdkWGkVSSqb8wXzaL
XTE8iMmCfsz/LXGIHGa3lOhhwj/zXMOH+Eg9xt+vXvYFuaoOUxeCSz8WiRg7bNwx
gfaQvsX+2lVBkfcyn8s8NEJOBvKSzRl3JQcUrY3QWnvUZcM37E4NueZOkeygAqFl
Jjcq6DdQOskUOzkQb+6lxDtiBJQZ6jTZWIcIW9Ec8Xc6VqsxuKLM3iv17Lo+5qbj
1Enj1YBnOwK8Rs3AvhvB8vRJ/y1MBImnc3bheLxTGZ5NAsxwLarg0jRqzSsxuDOS
t9xz3voeyYUVHN91uaIRENIyqfvtkg==
=gN9V
-----END PGP SIGNATURE-----
Merge tag 'drm-misc-fixes-2020-08-04' of git://anongit.freedesktop.org/drm/drm-misc into drm-next
* backmerge from drm-fixes at v5.8-rc7
* add orientation quirk for ASUS T103HAF
* drm/omap: force runtime PM suspend on system suspend
* drm/tidss: fix modeset init for DPI panels
* re-added docs for drm_gem_flink_ioctl()
* ttm: fix page-offset calculation within TTM
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804125510.GA29670@linux-uq9g
This name better reflects what the object does. I didn't rename
all the pointers it seemed too messy.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-60-airlied@gmail.com
under driver control, this flag isn't needed anymore,
remove the API that used to access it, and consoldiate
with the used api.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-56-airlied@gmail.com
The driver now controls these, the core just controls the system
memory one.
v2: init sysman explicitly and assign it as a driver manager
to simplify the lookup sequence.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-55-airlied@gmail.com
Drivers should all be cleaning up their memory managers
themselves now, so let the core just clean the system one up.
Remove the legacy cleaning interface.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-36-airlied@gmail.com
