diff --git a/Documentation/dev-tools/kunit/architecture.rst b/Documentation/dev-tools/kunit/architecture.rst index aa2cea821e25..ff9c85a0bff2 100644 --- a/Documentation/dev-tools/kunit/architecture.rst +++ b/Documentation/dev-tools/kunit/architecture.rst @@ -26,10 +26,7 @@ The fundamental unit in KUnit is the test case. The KUnit test cases are grouped into KUnit suites. A KUnit test case is a function with type signature ``void (*)(struct kunit *test)``. These test case functions are wrapped in a struct called -``struct kunit_case``. For code, see: - -.. kernel-doc:: include/kunit/test.h - :identifiers: kunit_case +struct kunit_case. .. note: ``generate_params`` is optional for non-parameterized tests. @@ -152,18 +149,12 @@ Parameterized Tests Each KUnit parameterized test is associated with a collection of parameters. The test is invoked multiple times, once for each parameter value and the parameter is stored in the ``param_value`` field. -The test case includes a ``KUNIT_CASE_PARAM()`` macro that accepts a +The test case includes a KUNIT_CASE_PARAM() macro that accepts a generator function. The generator function is passed the previous parameter and returns the next parameter. It also provides a macro to generate common-case generators based on arrays. -For code, see: - -.. kernel-doc:: include/kunit/test.h - :identifiers: KUNIT_ARRAY_PARAM - - kunit_tool (Command Line Test Harness) ====================================== diff --git a/Documentation/locking/locktypes.rst b/Documentation/locking/locktypes.rst index bfa75ea1b66a..9933faad4771 100644 --- a/Documentation/locking/locktypes.rst +++ b/Documentation/locking/locktypes.rst @@ -211,9 +211,6 @@ raw_spinlock_t and spinlock_t raw_spinlock_t -------------- -raw_spinlock_t is a strict spinning lock implementation regardless of the -kernel configuration including PREEMPT_RT enabled kernels. - raw_spinlock_t is a strict spinning lock implementation in all kernels, including PREEMPT_RT kernels. Use raw_spinlock_t only in real critical core code, low-level interrupt handling and places where disabling diff --git a/Documentation/maintainer/index.rst b/Documentation/maintainer/index.rst index f0a60435b124..3e03283c144e 100644 --- a/Documentation/maintainer/index.rst +++ b/Documentation/maintainer/index.rst @@ -12,6 +12,7 @@ additions to this manual. configure-git rebasing-and-merging pull-requests + messy-diffstat maintainer-entry-profile modifying-patches diff --git a/Documentation/maintainer/messy-diffstat.rst b/Documentation/maintainer/messy-diffstat.rst new file mode 100644 index 000000000000..c015f66d7621 --- /dev/null +++ b/Documentation/maintainer/messy-diffstat.rst @@ -0,0 +1,96 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===================================== +Handling messy pull-request diffstats +===================================== + +Subsystem maintainers routinely use ``git request-pull`` as part of the +process of sending work upstream. Normally, the result includes a nice +diffstat that shows which files will be touched and how much of each will +be changed. Occasionally, though, a repository with a relatively +complicated development history will yield a massive diffstat containing a +great deal of unrelated work. The result looks ugly and obscures what the +pull request is actually doing. This document describes what is happening +and how to fix things up; it is derived from The Wisdom of Linus Torvalds, +found in Linus1_ and Linus2_. + +.. _Linus1: https://lore.kernel.org/lkml/CAHk-=wg3wXH2JNxkQi+eLZkpuxqV+wPiHhw_Jf7ViH33Sw7PHA@mail.gmail.com/ +.. _Linus2: https://lore.kernel.org/lkml/CAHk-=wgXbSa8yq8Dht8at+gxb_idnJ7X5qWZQWRBN4_CUPr=eQ@mail.gmail.com/ + +A Git development history proceeds as a series of commits. In a simplified +manner, mainline kernel development looks like this:: + + ... vM --- vN-rc1 --- vN-rc2 --- vN-rc3 --- ... --- vN-rc7 --- vN + +If one wants to see what has changed between two points, a command like +this will do the job:: + + $ git diff --stat --summary vN-rc2..vN-rc3 + +Here, there are two clear points in the history; Git will essentially +"subtract" the beginning point from the end point and display the resulting +differences. The requested operation is unambiguous and easy enough to +understand. + +When a subsystem maintainer creates a branch and commits changes to it, the +result in the simplest case is a history that looks like:: + + ... vM --- vN-rc1 --- vN-rc2 --- vN-rc3 --- ... --- vN-rc7 --- vN + | + +-- c1 --- c2 --- ... --- cN + +If that maintainer now uses ``git diff`` to see what has changed between +the mainline branch (let's call it "linus") and cN, there are still two +clear endpoints, and the result is as expected. So a pull request +generated with ``git request-pull`` will also be as expected. But now +consider a slightly more complex development history:: + + ... vM --- vN-rc1 --- vN-rc2 --- vN-rc3 --- ... --- vN-rc7 --- vN + | | + | +-- c1 --- c2 --- ... --- cN + | / + +-- x1 --- x2 --- x3 + +Our maintainer has created one branch at vN-rc1 and another at vN-rc2; the +two were then subsequently merged into c2. Now a pull request generated +for cN may end up being messy indeed, and developers often end up wondering +why. + +What is happening here is that there are no longer two clear end points for +the ``git diff`` operation to use. The development culminating in cN +started in two different places; to generate the diffstat, ``git diff`` +ends up having pick one of them and hoping for the best. If the diffstat +starts at vN-rc1, it may end up including all of the changes between there +and the second origin end point (vN-rc2), which is certainly not what our +maintainer had in mind. With all of that extra junk in the diffstat, it +may be impossible to tell what actually happened in the changes leading up +to cN. + +Maintainers often try to resolve this problem by, for example, rebasing the +branch or performing another merge with the linus branch, then recreating +the pull request. This approach tends not to lead to joy at the receiving +end of that pull request; rebasing and/or merging just before pushing +upstream is a well-known way to get a grumpy response. + +So what is to be done? The best response when confronted with this +situation is to indeed to do a merge with the branch you intend your work +to be pulled into, but to do it privately, as if it were the source of +shame. Create a new, throwaway branch and do the merge there:: + + ... vM --- vN-rc1 --- vN-rc2 --- vN-rc3 --- ... --- vN-rc7 --- vN + | | | + | +-- c1 --- c2 --- ... --- cN | + | / | | + +-- x1 --- x2 --- x3 +------------+-- TEMP + +The merge operation resolves all of the complications resulting from the +multiple beginning points, yielding a coherent result that contains only +the differences from the mainline branch. Now it will be possible to +generate a diffstat with the desired information:: + + $ git diff -C --stat --summary linus..TEMP + +Save the output from this command, then simply delete the TEMP branch; +definitely do not expose it to the outside world. Take the saved diffstat +output and edit it into the messy pull request, yielding a result that +shows what is really going on. That request can then be sent upstream. diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index 4392b3cb4020..b5feb5b1d905 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -128,6 +128,7 @@ class KernelCmd(Directive): return out def nestedParse(self, lines, fname): + env = self.state.document.settings.env content = ViewList() node = nodes.section() @@ -137,7 +138,7 @@ class KernelCmd(Directive): code_block += "\n " + l lines = code_block + "\n\n" - line_regex = re.compile("^#define LINENO (\S+)\#([0-9]+)$") + line_regex = re.compile("^\.\. LINENO (\S+)\#([0-9]+)$") ln = 0 n = 0 f = fname @@ -154,6 +155,9 @@ class KernelCmd(Directive): self.do_parse(content, node) content = ViewList() + # Add the file to Sphinx build dependencies + env.note_dependency(os.path.abspath(f)) + f = new_f # sphinx counts lines from 0 diff --git a/Documentation/sphinx/kernel_feat.py b/Documentation/sphinx/kernel_feat.py index 8138d69a6987..27b701ed3681 100644 --- a/Documentation/sphinx/kernel_feat.py +++ b/Documentation/sphinx/kernel_feat.py @@ -33,6 +33,7 @@ u""" import codecs import os +import re import subprocess import sys @@ -82,7 +83,7 @@ class KernelFeat(Directive): env = doc.settings.env cwd = path.dirname(doc.current_source) - cmd = "get_feat.pl rest --dir " + cmd = "get_feat.pl rest --enable-fname --dir " cmd += self.arguments[0] if len(self.arguments) > 1: @@ -102,7 +103,22 @@ class KernelFeat(Directive): shell_env["srctree"] = srctree lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env) - nodeList = self.nestedParse(lines, fname) + + line_regex = re.compile("^\.\. FILE (\S+)$") + + out_lines = "" + + for line in lines.split("\n"): + match = line_regex.search(line) + if match: + fname = match.group(1) + + # Add the file to Sphinx build dependencies + env.note_dependency(os.path.abspath(fname)) + else: + out_lines += line + "\n" + + nodeList = self.nestedParse(out_lines, fname) return nodeList def runCmd(self, cmd, **kwargs): diff --git a/Documentation/sphinx/kernel_include.py b/Documentation/sphinx/kernel_include.py index f523aa68a36b..abe768088377 100755 --- a/Documentation/sphinx/kernel_include.py +++ b/Documentation/sphinx/kernel_include.py @@ -59,6 +59,7 @@ class KernelInclude(Include): u"""KernelInclude (``kernel-include``) directive""" def run(self): + env = self.state.document.settings.env path = os.path.realpath( os.path.expandvars(self.arguments[0])) @@ -70,6 +71,8 @@ class KernelInclude(Include): self.arguments[0] = path + env.note_dependency(os.path.abspath(path)) + #return super(KernelInclude, self).run() # won't work, see HINTs in _run() return self._run() diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py index 8189c33b9dda..9395892c7ba3 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -130,7 +130,7 @@ class KernelDocDirective(Directive): result = ViewList() lineoffset = 0; - line_regex = re.compile("^#define LINENO ([0-9]+)$") + line_regex = re.compile("^\.\. LINENO ([0-9]+)$") for line in lines: match = line_regex.search(line) if match: diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py index 24d2b2addcce..cefdbb7e7523 100644 --- a/Documentation/sphinx/kfigure.py +++ b/Documentation/sphinx/kfigure.py @@ -212,7 +212,7 @@ def setupTools(app): if convert_cmd: kernellog.verbose(app, "use convert(1) from: " + convert_cmd) else: - kernellog.warn(app, + kernellog.verbose(app, "Neither inkscape(1) nor convert(1) found.\n" "For SVG to PDF conversion, " "install either Inkscape (https://inkscape.org/) (preferred) or\n" @@ -296,8 +296,10 @@ def convert_image(img_node, translator, src_fname=None): if translator.builder.format == 'latex': if not inkscape_cmd and convert_cmd is None: - kernellog.verbose(app, - "no SVG to PDF conversion available / include SVG raw.") + kernellog.warn(app, + "no SVG to PDF conversion available / include SVG raw." + "\nIncluding large raw SVGs can cause xelatex error." + "\nInstall Inkscape (preferred) or ImageMagick.") img_node.replace_self(file2literal(src_fname)) else: dst_fname = path.join(translator.builder.outdir, fname + '.pdf') diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt index 9a35f50798a6..2c573541ab71 100644 --- a/Documentation/sphinx/requirements.txt +++ b/Documentation/sphinx/requirements.txt @@ -1,2 +1,4 @@ +# jinja2>=3.1 is not compatible with Sphinx<4.0 +jinja2<3.1 sphinx_rtd_theme Sphinx==2.4.4 diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl index 7437e19ba3ac..1389db76cff3 100755 --- a/scripts/get_abi.pl +++ b/scripts/get_abi.pl @@ -327,7 +327,7 @@ sub output_rest { my @filepath = split / /, $data{$what}->{filepath}; if ($enable_lineno) { - printf "#define LINENO %s%s#%s\n\n", + printf ".. LINENO %s%s#%s\n\n", $prefix, $file[0], $data{$what}->{line_no}; } @@ -1023,7 +1023,7 @@ logic (B<--no-rst-source>). =item B<--enable-lineno> -Enable output of #define LINENO lines. +Enable output of .. LINENO lines. =item B<--debug> I diff --git a/scripts/get_feat.pl b/scripts/get_feat.pl index 457712355676..76cfb96b59b6 100755 --- a/scripts/get_feat.pl +++ b/scripts/get_feat.pl @@ -13,6 +13,7 @@ my $man; my $debug; my $arch; my $feat; +my $enable_fname; my $basename = abs_path($0); $basename =~ s,/[^/]+$,/,; @@ -31,6 +32,7 @@ GetOptions( 'arch=s' => \$arch, 'feat=s' => \$feat, 'feature=s' => \$feat, + "enable-fname" => \$enable_fname, man => \$man ) or pod2usage(2); @@ -95,6 +97,10 @@ sub parse_feat { return if ($file =~ m,($prefix)/arch-support.txt,); return if (!($file =~ m,arch-support.txt$,)); + if ($enable_fname) { + printf ".. FILE %s\n", abs_path($file); + } + my $subsys = ""; $subsys = $2 if ( m,.*($prefix)/([^/]+).*,); @@ -580,6 +586,11 @@ Output features for a single specific feature. Changes the location of the Feature files. By default, it uses the Documentation/features directory. +=item B<--enable-fname> + +Prints the file name of the feature files. This can be used in order to +track dependencies during documentation build. + =item B<--debug> Put the script in verbose mode, useful for debugging. Can be called multiple diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 9c084a2ba3b0..7516949bb049 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -424,7 +424,7 @@ sub get_kernel_version() { sub print_lineno { my $lineno = shift; if ($enable_lineno && defined($lineno)) { - print "#define LINENO " . $lineno . "\n"; + print ".. LINENO " . $lineno . "\n"; } } ## @@ -2478,7 +2478,7 @@ May be specified multiple times. =item -enable-lineno -Enable output of #define LINENO lines. +Enable output of .. LINENO lines. =back