Handling of multicast/replicated registers is spread across intel_gt.c
and intel_uncore.c today. As multicast handling and the related
steering logic gets more complicated with the addition of new platforms
and new rules it makes sense to centralize it all in one place.
For now the existing functions have been moved to the new .c/.h as-is.
Function renames and updates to operate in a more consistent manner will
be done in subsequent patches.
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Acked-by: Jani Nikula <jani.nikula@linux.intel.com>
Reviewed-by: Harish Chegondi <harish.chegondi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220615001019.1821989-2-matthew.d.roper@intel.com
Several of our i915 header files, have been including i915_reg.h. This
means that any change to i915_reg.h will trigger a full rebuild of
pretty much every file of the driver, even those that don't have any
kind of register access. Let's delete the i915_reg.h include from all
headers and add an explicit include from the .c files that truly
need the register definitions; those that need a definition of
i915_reg_t for a function definition can get it from i915_reg_defs.h
instead.
We also remove two non-register #define's (VLV_DISPLAY_BASE and
GEN12_SFC_DONE_MAX) into i915_reg_defs.h to allow us to drop the
i915_reg.h include from a couple of headers.
There's probably a lot more header dependency optimization possible, but
the changes here roughly cut the number of files compiled after 'touch
i915_reg.h' in half --- a good first step.
Cc: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220127234334.4016964-7-matthew.d.roper@intel.com
We need to flush TLBs before releasing backing store otherwise userspace
is able to encounter stale entries if a) it is not declaring access to
certain buffers and b) it races with the backing store release from a
such undeclared execution already executing on the GPU in parallel.
The approach taken is to mark any buffer objects which were ever bound
to the GPU and to trigger a serialized TLB flush when their backing
store is released.
Alternatively the flushing could be done on VMA unbind, at which point
we would be able to ascertain whether there is potential a parallel GPU
execution (which could race), but essentially it boils down to paying
the cost of TLB flushes potentially needlessly at VMA unbind time (when
the backing store is not known to be going away so not needed for
safety), versus potentially needlessly at backing store relase time
(since we at that point cannot tell whether there is anything executing
on the GPU which uses that object).
Thereforce simplicity of implementation has been chosen for now with
scope to benchmark and refine later as required.
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reported-by: Sushma Venkatesh Reddy <sushma.venkatesh.reddy@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Dave Airlie <airlied@redhat.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Jon Bloomfield <jon.bloomfield@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
We now support a per-gt uncore, yet we're not able to infer which GT
we're operating upon. Let's store a backpointer for now.
At this point the early initialization of the gt needs to be
broken in two parts where the first is needed to assign to the gt
the i915 private data pointer and the uncore. A temporary
function has been made and the two parts are
__intel_gt_init_early() and intel_gt_init_early(). This split
will be fixed in the future with the multitile patch.
Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211214193346.21231-2-andi.shyti@linux.intel.com
UAPI Changes:
- Add uAPI for using PXP protected objects
Mesa changes: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8064
- Add PCI IDs and LMEM discovery/placement uAPI for DG1
Mesa changes: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11584
- Disable engine bonding on Gen12+ except TGL, RKL and ADL-S
Cross-subsystem Changes:
- Merges 'tip/locking/wwmutex' branch (core kernel tip)
- "mei: pxp: export pavp client to me client bus"
Core Changes:
- Update ttm_move_memcpy for async use (Thomas)
Driver Changes:
- Enable GuC submission by default on DG1 (Matt B)
- Add PXP (Protected Xe Path) support for Gen12 integrated (Daniele,
Sean, Anshuman)
See "drm/i915/pxp: add PXP documentation" for details!
- Remove force_probe protection for ADL-S (Raviteja)
- Add base support for XeHP/XeHP SDV (Matt R, Stuart, Lucas)
- Handle DRI_PRIME=1 on Intel igfx + Intel dgfx hybrid graphics setup (Tvrtko)
- Use Transparent Hugepages when IOMMU is enabled (Tvrtko, Chris)
- Implement LMEM backup and restore for suspend / resume (Thomas)
- Report INSTDONE_GEOM values in error state for DG2 (Matt R)
- Add DG2-specific shadow register table (Matt R)
- Update Gen11/Gen12/XeHP shadow register tables (Matt R)
- Maintain backward-compatible nested batch behavior on TGL+ (Matt R)
- Add new LRI reg offsets for DG2 (Akeem)
- Initialize unused MOCS entries to device specific values (Ayaz)
- Track and use the correct UC MOCS index on Gen12 (Ayaz)
- Add separate MOCS table for Gen12 devices other than TGL/RKL (Ayaz)
- Simplify the locking and eliminate some RCU usage (Daniel)
- Add some flushing for the 64K GTT path (Matt A)
- Mark GPU wedging on driver unregister unrecoverable (Janusz)
- Major rework in the GuC codebase, simplify locking and add docs (Matt B)
- Add DG1 GuC/HuC firmwares (Daniele, Matt B)
- Remember to call i915_sw_fence_fini on guc_state.blocked (Matt A)
- Use "gt" forcewake domain name for error messages instead of "blitter" (Matt R)
- Drop now duplicate LMEM uAPI RFC kerneldoc section (Daniel)
- Fix early tracepoints for requests (Matt A)
- Use locked access to ctx->engines in set_priority (Daniel)
- Convert gen6/gen7/gen8 read operations to fwtable (Matt R)
- Drop gen11/gen12 specific mmio write handlers (Matt R)
- Drop gen11 specific mmio read handlers (Matt R)
- Use designated initializers for init/exit table (Kees)
- Fix syncmap memory leak (Matt B)
- Add pretty printing for buddy allocator state debug (Matt A)
- Fix potential error pointer dereference in pinned_context() (Dan)
- Remove IS_ACTIVE macro (Lucas)
- Static code checker fixes (Nathan)
- Clean up disabled warnings (Nathan)
- Increase timeout in i915_gem_contexts selftests 5x for GuC submission (Matt B)
- Ensure wa_init_finish() is called for ctx workaround list (Matt R)
- Initialize L3CC table in mocs init (Sreedhar, Ayaz, Ram)
- Get PM ref before accessing HW register (Vinay)
- Move __i915_gem_free_object to ttm_bo_destroy (Maarten)
- Deduplicate frequency dump on debugfs (Lucas)
- Make wa list per-gt (Venkata)
- Do not define dummy vma in stack (Venkata)
- Take pinning into account in __i915_gem_object_is_lmem (Matt B, Thomas)
- Do not report currently active engine when describing objects (Tvrtko)
- Fix pdfdocs build error by removing nested grid from GuC docs (Akira)
- Remove false warning from the rps worker (Tejas)
- Flush buffer pools on driver remove (Janusz)
- Fix runtime pm handling in i915_gem_shrink (Maarten)
- Rework TTM object initialization slightly (Thomas)
- Use fixed offset for PTEs location (Michal Wa)
- Verify result from CTB (de)register action and improve error messages (Michal Wa)
- Fix bug in user proto-context creation that leaked contexts (Matt B)
- Re-use Gen11 forcewake read functions on Gen12 (Matt R)
- Make shadow tables range-based (Matt R)
- Ditch the i915_gem_ww_ctx loop member (Thomas, Maarten)
- Use NULL instead of 0 where appropriate (Ville)
- Rename pci/debugfs functions to respect file prefix (Jani, Lucas)
- Drop guc_communication_enabled (Daniele)
- Selftest fixes (Thomas, Daniel, Matt A, Maarten)
- Clean up inconsistent indenting (Colin)
- Use direction definition DMA_BIDIRECTIONAL instead of
PCI_DMA_BIDIRECTIONAL (Cai)
- Add "intel_" as prefix in set_mocs_index() (Ayaz)
From: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/YWAO80MB2eyToYoy@jlahtine-mobl.ger.corp.intel.com
Signed-off-by: Dave Airlie <airlied@redhat.com>
Rather than defining our shadow tables as a list of individual
registers, provide them as a list of register ranges; we'll have some
ranges of multiple registers being added soon (and we already have a
couple adjacent registers that we can squash into a single range now).
This change also defines the table with hex literal values rather than
symbolic register names; since that's how the tables are defined in the
bspec, this change will make it easier to review the tables overall.
v2:
- Force signed comparison on range overlap sanity check
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Caz Yokoyama <caz.yokoyama@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210729152158.2646246-1-matthew.d.roper@intel.com
The power well that we've been referring to as the 'blitter' well is
actually more of a general GT power well which contains a lot of things
other than the blitter engine registers. The FORCEWAKE_BLITTER name in
the code was used for historic reasons, but no longer matches how the
bspec describes this power well and just causes confusion for people not
familiar with this area of the code. Let's rename it to FORCEWAKE_GT to
more accurately describe the role of the power well and match how the
modern bspec refers to it.
v2:
- Add a comment noting that the GT power well includes the blitter
engine. (Jose)
Bspec: 66696, 66534, 67609
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201009194442.3668677-2-matthew.d.roper@intel.com
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Multiple uncore structures will share the debug infrastructure, so
move it to a common place and add extra locking around it.
Also, since we now have a separate object, it is cleaner to have
dedicated functions working on the object to stop and restart the
mmio debug. Apart from the cosmetic changes, this patch introduces
2 functional updates:
- All calls to check_for_unclaimed_mmio will now return false when
the debug is suspended, not just the ones that are active only when
i915_modparams.mmio_debug is set. If we don't trust the result of the
check while a user is doing mmio access then we shouldn't attempt the
check anywhere.
- i915_modparams.mmio_debug is not save/restored anymore around user
access. The value is now never touched by the kernel while debug is
disabled so no need for save/restore.
v2: squash mmio_debug patches, restrict mmio_debug lock usage (Chris)
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20190809063116.7527-1-chris@chris-wilson.co.uk
Currently i915_reset.c mixes calls to intel_uncore, pci and our old
style I915_READ mmio interfaces. Cast aside the old implicit macros,
and harmonise on using uncore throughout.
add/remove: 1/1 grow/shrink: 0/4 up/down: 65/-207 (-142)
Function old new delta
rmw_register - 65 +65
gen8_reset_engines 945 942 -3
g4x_do_reset 407 376 -31
intel_gpu_reset 545 509 -36
clear_register 63 - -63
i915_clear_error_registers 461 387 -74
A little bit of pointer dancing elimination works wonders.
v2: Roll up the helpers into intel_uncore for general use
With the helpers gcc was a little more eager to inline:
add/remove: 0/1 grow/shrink: 1/3 up/down: 99/-133 (-34)
Function old new delta
i915_clear_error_registers 461 560 +99
gen8_reset_engines 945 942 -3
g4x_do_reset 407 376 -31
intel_gpu_reset 545 509 -36
clear_register 63 - -63
Total: Before=1544400, After=1544366, chg -0.00%
Win some, lose some, gcc is gcc.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190405202419.3093-1-chris@chris-wilson.co.uk
When building with ARCH=i386, readq and writeq are not defined,
resulting in:
intel_uncore.h: In function ‘__raw_uncore_read64’:
intel_uncore.h:257:9: error: implicit declaration of function ‘readq’;
did you mean ‘readl’? [-Werror=implicit-function-declaration]
return read##s__(uncore->regs + i915_mmio_reg_offset(reg)); \
^
and:
intel_uncore.h: In function ‘__raw_uncore_write64’:
intel_uncore.h:264:2: error: implicit declaration of function ‘writeq’;
did you mean ‘writel’? [-Werror=implicit-function-declaration]
write##s__(val, uncore->regs + i915_mmio_reg_offset(reg)); \
^
Add the io-64-nonatomic-lo-hi include to have readq and writeq available
for all builds. This header internally includes linux/io.h, so the
native readq and writeq definitions will be used when available.
Fixes: 6cc5ca7688 ("drm/i915: rename raw reg access functions")
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20190326233817.5417-1-daniele.ceraolospurio@intel.com
The only usage we have for it is for the regs pointer. Save a pointer to
the set and ack registers instead of the register offsets to remove this
requirement
v2: Keep passing uncore down to the lowest levels to avoid repeated
pointer chasing in the innermost loops:
add/remove: 0/0 grow/shrink: 3/15 up/down: 33/-155 (-122)
Function old new delta
fw_domain_init 320 351 +31
hdmi_port_clock_valid 319 320 +1
g4x_pre_enable_dp 364 365 +1
ring_request_alloc 1852 1851 -1
intel_engine_lookup_user 50 49 -1
gen11_irq_handler 738 737 -1
__intel_uncore_forcewake_get 115 112 -3
intel_uncore_fw_release_timer 198 194 -4
i915_forcewake_domains 154 150 -4
fw_domain_fini 172 168 -4
__intel_uncore_forcewake_put 163 159 -4
___force_wake_auto 138 134 -4
fw_domains_put 98 92 -6
__err_print_to_sgl 4058 4052 -6
intel_uncore_forcewake_reset 459 444 -15
fw_domains_get 563 548 -15
fw_domain_wait_ack_with_fallback 490 450 -40
fw_domains_get_with_fallback 875 828 -47
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20190313231319.711-2-daniele.ceraolospurio@intel.com
Link: https://patchwork.freedesktop.org/patch/msgid/20190316100045.20240-1-chris@chris-wilson.co.uk
In Gen11, the Video Decode engines (aka VDBOX, aka VCS, aka BSD) and the
Video Enhancement engines (aka VEBOX, aka VECS) could be fused off. Also,
each VDBOX and VEBOX has its own power well, which only exist if the
related engine exists in the HW.
Unfortunately, we have a Catch-22 situation going on: we need the blitter
forcewake to read the register with the fuse info, but we cannot initialize
the forcewake domains without knowin about the engines present in the HW.
We workaround this problem by allowing the initialization of all forcewake
domains and then pruning the fused off ones, as per the fuse information.
Bspec: 20680
v2: We were shifting incorrectly for vebox disable (Vinay)
v3: Assert mmio is ready and warn if we have attempted to initialize
forcewake for fused-off engines (Paulo)
v4:
- Use INTEL_GEN in new code (Tvrtko)
- Shorter local variable (Tvrtko, Michal)
- Keep "if (!...) continue" style (Tvrtko)
- No unnecessary BUG_ON (Tvrtko)
- WARN_ON and cleanup if wrong mask (Tvrtko, Michal)
- Use I915_READ_FW (Michal)
- Use I915_MAX_VCS/VECS macros (Michal)
v5: Rebased by Rodrigo fixing conflicts on top of:
"drm/i915: Simplify intel_engines_init"
v6: Fix v5. Remove info->num_rings. (by Oscar)
v7: Rebase (Rodrigo).
v8:
- s/intel_device_info_fused_off_engines/
intel_device_info_init_mmio (Chris)
- Make vdbox_disable & vebox_disable local variables (Chris)
v9:
- Move function declaration to intel_device_info.h (Michal)
- Missing indent in bit fields definitions (Michal)
- When RC6 is enabled by BIOS, the fuse register cannot be read until
the blitter powerwell is awake. Shuffle where the fuse is read, prune
the forcewake domains after the fact and change the commit message
accordingly (Vinay, Sagar, Chris).
v10:
- Improved commit message (Sagar)
- New line in header file (Sagar)
- Specify the message in fw_domain_reset applies to ICL+ (Sagar)
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Cc: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Sagar Arun Kamble <sagar.a.kamble@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Oscar Mateo <oscar.mateo@intel.com>
Reviewed-by: Sagar Arun Kamble <sagar.a.kamble@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180316121456.11577-1-mika.kuoppala@linux.intel.com
[Mika: soothe checkpatch on commit msg]
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
The main difference with previous GENs is that starting from Gen11
each VCS and VECS engine has its own power well, which only exist
if the related engine exists in the HW.
The fallback forcewake request workaround is only needed on gen9
according to the HSDES WA entry (1604254524), so we can go back to using
the simpler fw_domains_get/put functions.
BSpec: 18331
v2: fix fwtable, use array to test shadow tables, create new
accessors to avoid check on every access (Tvrtko)
v3 (from Paulo): Rebase.
v4:
- Range 09400-097FF should be FORCEWAKE_ALL (Daniele)
- Use the BIT macro for forcewake domains (Daniele)
- Add a comment about the range ordering (Oscar)
- Updated commit message (Oscar)
v5: Rebased
v6: Use I915_MAX_VCS/VECS (Michal)
v7: translate FORCEWAKE_ALL to available domains
v8: rebase, add clarification on fallback ack in commit message.
v9: fix rebase issue, change check in fw_domains_init from IS_GEN11
to GEN >= 11
v10: Generate is_genX_shadowed with a macro (Daniele)
Include gen11_fw_ranges in the selftest (Michel)
v11: Simplify FORCEWAKE_ALL, new line between NEEDS_FORCEWAKEs (Tvrtko)
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Acked-by: Michel Thierry <michel.thierry@intel.com>
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Oscar Mateo <oscar.mateo@intel.com>
Signed-off-by: Michel Thierry <michel.thierry@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180302161501.28594-6-mika.kuoppala@linux.intel.com
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
