Commit Graph

158 Commits

Author SHA1 Message Date
Masahiro Yamada
12fec3d601 kbuild: replace $(dot-target).tmp in filechk with $(tmp-target)
$(tmp-target) is a better fit for local use like this.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-22 23:43:33 +09:00
Masahiro Yamada
ee2162bd4f kbuild: add more comments for KBUILD_NOCMDDEP=1
The cmd-check for KBUILD_NOCMDDEP=1 may not be clear until you see
commit c4d5ee1398 ("kbuild: make KBUILD_NOCMDDEP=1 handle empty
built-in.o").

When a phony target (i.e. FORCE) is the only prerequisite, Kbuild
uses a tricky way to detect that the target does not exist.

Add more comments.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-22 23:43:33 +09:00
Masahiro Yamada
92215e7a80 kbuild: rename cmd_$@ to savedcmd_$@ in *.cmd files
The cmd-check macro compares $(cmd_$@) and $(cmd_$1), but a pitfall is
that you cannot use cmd_<target> as the variable name for the command.

For example, the following code will not work in the top Makefile
or ./Kbuild.

    quiet_cmd_foo = GEN     $@
          cmd_foo = touch $@

    targets += foo
    foo: FORCE
            $(call if_changed,foo)

In this case, both $@ and $1 are expanded to 'foo', so $(cmd_check)
is always empty.

We do not need to use the same prefix for cmd_$@ and cmd_$1.
Rename the former to savedcmd_$@.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-22 23:43:33 +09:00
Masahiro Yamada
6ae4b9868a kbuild: allow to combine multiple V= levels
Commit a6de553da0 ("kbuild: Allow to combine multiple W= levels")
supported W=123 to enable all the extra warning groups.

I think a similar idea is applicable to the V= option.

  V=1 echos the whole command
  V=2 prints the reason for rebuilding

These are orthogonal, and can be enabled at the same time.

This commit supports V=12 to enable both of them.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-22 23:43:32 +09:00
Masahiro Yamada
8962b6b475 kbuild: print short log in addition to the whole command with V=1
"make V=1" prints the whole command instead of the short log, but I
think it is nicer to print both so that you can easily spot the build
rule of your interest.

This commit changes V=1 to print the short log (the line starts with
'#'), followed by the full log.

In parallel builds, the short/full logs from the same build rule may
be interspersed. If you want to avoid it, please add -Otarget option.
Kbuild will never set it by default because Make would buffer the logs
and lose the escape sequences. (Modern compilers print warnings and
errors in color, but only when they write to a terminal.)

This is also a preparation for supporting V=12 because V=2 appends the
reason for rebuilding to the short log.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-22 23:43:32 +09:00
Masahiro Yamada
875ef1a57f kbuild: use .NOTINTERMEDIATE for future GNU Make versions
In Kbuild, some files are generated by chains of pattern/implicit rules.
For example, *.dtb.o files in drivers/of/unittest-data/Makefile are
generated by the chain of 3 pattern rules, like this:

  %.dts  ->  %.dtb  ->  %.dtb.S  ->  %.dtb.o

Here, %.dts is the real source, %.dtb.o is the final target.
%.dtb and %.dtb.S are called "intermediate files".

As GNU Make manual [1] says, intermediate files are treated differently
in two ways:

 (a) The first difference is what happens if the intermediate file does
   not exist. If an ordinary file 'b' does not exist, and make considers
   a target that depends on 'b', it invariably creates 'b' and then
   updates the target from 'b'. But if 'b' is an intermediate file, then
   make can leave well enough alone: it won't create 'b' unless one of
   its prerequisites is out of date. This means the target depending
   on 'b' won't be rebuilt either, unless there is some other reason
   to update that target: for example the target doesn't exist or a
   different prerequisite is newer than the target.

 (b) The second difference is that if make does create 'b' in order to
   update something else, it deletes 'b' later on after it is no longer
   needed. Therefore, an intermediate file which did not exist before
   make also does not exist after make. make reports the deletion to
   you by printing a 'rm' command showing which file it is deleting.

The combination of these is problematic for Kbuild because most of the
build rules depend on FORCE and the if_changed* macros really determine
if the target should be updated. So, all missing files, whether they
are intermediate or not, are always rebuilt.

To see the problem, delete ".SECONDARY:" from scripts/Kbuild.include,
and repeat this command:

  $ make allmodconfig drivers/of/unittest-data/

The intermediate files will be deleted, which results in rebuilding
intermediate and final objects in the next run of make.

In the old days, people suppressed (b) in inconsistent ways.
As commit 54a702f705 ("kbuild: mark $(targets) as .SECONDARY and
remove .PRECIOUS markers") noted, you should not use .PRECIOUS because
.PRECIOUS has the following behavior (c), which is not likely what you
want.

 (c) If make is killed or interrupted during the execution of their
   recipes, the target is not deleted. Also, the target is not deleted
   on error even if .DELETE_ON_ERROR is specified.

.SECONDARY is a much better way to disable (b), but a small problem
is that .SECONDARY enables (a), which gives a side-effect to $?;
prerequisites marked as .SECONDARY do not appear in $?. This is a
drawback for Kbuild.

I thought it was a bug and opened a bug report. As Paul, the GNU Make
maintainer, concluded in [2], this is not a bug.

A good news is that, GNU Make 4.4 added the perfect solution,
.NOTINTERMEDIATE, which cancels both (a) and (b).

For clarificaton, my understanding of .INTERMEDIATE, .SECONDARY,
.PRECIOUS and .NOTINTERMEDIATE are as follows:

                        (a)         (b)         (c)
  .INTERMEDIATE        enable      enable      disable
  .SECONDARY           enable      disable     disable
  .PRECIOUS            disable     disable     enable
  .NOTINTERMEDIATE     disable     disable     disable

However, GNU Make 4.4 has a bug for the global .NOTINTERMEDIATE. [3]
It was fixed by commit 6164608900ad ("[SV 63417] Ensure global
.NOTINTERMEDIATE disables all intermediates"), and will be available
in the next release of GNU Make.

The following is the gain for .NOTINTERMEDIATE:

  [Current Make]

      $ make allnoconfig vmlinux
          [ full build ]
      $ rm include/linux/device.h
      $ make vmlinux
        CALL    scripts/checksyscalls.sh

  Make does not notice the removal of <linux/device.h>.

  [Future Make]

      $ make-latest allnoconfig vmlinux
          [ full build ]
      $ rm include/linux/device.h
      $ make-latest vmlinux
        CC      arch/x86/kernel/asm-offsets.s
      In file included from ./include/linux/writeback.h:13,
                       from ./include/linux/memcontrol.h:22,
                       from ./include/linux/swap.h:9,
                       from ./include/linux/suspend.h:5,
                       from arch/x86/kernel/asm-offsets.c:13:
      ./include/linux/blk_types.h:11:10: fatal error: linux/device.h: No such file or directory
         11 | #include <linux/device.h>
            |          ^~~~~~~~~~~~~~~~
      compilation terminated.
      make-latest[1]: *** [scripts/Makefile.build:114: arch/x86/kernel/asm-offsets.s] Error 1
      make-latest: *** [Makefile:1282: prepare0] Error 2

  Make notices the removal of <linux/device.h>, and rebuilds objects
  that depended on <linux/device.h>. There exists a source file that
  includes <linux/device.h>, and it raises an error.

To see detailed background information, refer to commit 2d3b1b8f0d
("kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild").

[1]: https://www.gnu.org/software/make/manual/make.html#Chained-Rules
[2]: https://savannah.gnu.org/bugs/?55532
[3]: https://savannah.gnu.org/bugs/?63417

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-12-13 22:29:10 +09:00
Masahiro Yamada
6768fa4bcb kbuild: add read-file macro
Since GNU Make 4.2, $(file ...) supports the read operater '<', which
is useful to read a file without forking a new process. No warning is
shown even if the input file is missing.

For older Make versions, it falls back to the cat command.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Alexander Lobakin <alexandr.lobakin@intel.com>
Tested-by: Alexander Lobakin <alexandr.lobakin@intel.com>
2022-12-13 22:29:10 +09:00
Masahiro Yamada
fccb3d3eda kbuild: add test-{ge,gt,le,lt} macros
GNU Make 4.4 introduced $(intcmp ...), which is useful to compare two
integers without forking a new process.

Add test-{ge,gt,le,lt} macros, which work more efficiently with GNU
Make >= 4.4. For older Make versions, they fall back to the 'test'
shell command.

The first two parameters to $(intcmp ...) must not be empty. To avoid
the syntax error, I appended '0' to them. Fortunately, '00' is treated
as '0'. This is needed because CONFIG options may expand to an empty
string when the kernel configuration is not included.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Palmer Dabbelt <palmer@rivosinc.com> # RISC-V
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2022-12-13 22:21:14 +09:00
Masahiro Yamada
a2430b25c3 kbuild: add kbuild-file macro
While building, installing, cleaning, Kbuild visits sub-directories
and includes 'Kbuild' or 'Makefile' that exists there.

Add 'kbuild-file' macro, and reuse it from scripts/Makefie.*

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Alexander Lobakin <alobakin@pm.me>
Tested-by: Alexander Lobakin <alobakin@pm.me>
2022-11-22 23:40:02 +09:00
Masahiro Yamada
a7f3257da8 kbuild: remove the target in signal traps when interrupted
When receiving some signal, GNU Make automatically deletes the target if
it has already been changed by the interrupted recipe.

If the target is possibly incomplete due to interruption, it must be
deleted so that it will be remade from scratch on the next run of make.
Otherwise, the target would remain corrupted permanently because its
timestamp had already been updated.

Thanks to this behavior of Make, you can stop the build any time by
pressing Ctrl-C, and just run 'make' to resume it.

Kbuild also relies on this feature, but it is equivalently important
for any build systems that make decisions based on timestamps (if you
want to support Ctrl-C reliably).

However, this does not always work as claimed; Make immediately dies
with Ctrl-C if its stderr goes into a pipe.

  [Test Makefile]

    foo:
            echo hello > $@
            sleep 3
            echo world >> $@

  [Test Result]

    $ make                         # hit Ctrl-C
    echo hello > foo
    sleep 3
    ^Cmake: *** Deleting file 'foo'
    make: *** [Makefile:3: foo] Interrupt

    $ make 2>&1 | cat              # hit Ctrl-C
    echo hello > foo
    sleep 3
    ^C$                            # 'foo' is often left-over

The reason is because SIGINT is sent to the entire process group.
In this example, SIGINT kills 'cat', and 'make' writes the message to
the closed pipe, then dies with SIGPIPE before cleaning the target.

A typical bad scenario (as reported by [1], [2]) is to save build log
by using the 'tee' command:

    $ make 2>&1 | tee log

This can be problematic for any build systems based on Make, so I hope
it will be fixed in GNU Make. The maintainer of GNU Make stated this is
a long-standing issue and difficult to fix [3]. It has not been fixed
yet as of writing.

So, we cannot rely on Make cleaning the target. We can do it by
ourselves, in signal traps.

As far as I understand, Make takes care of SIGHUP, SIGINT, SIGQUIT, and
SITERM for the target removal. I added the traps for them, and also for
SIGPIPE just in case cmd_* rule prints something to stdout or stderr
(but I did not observe an actual case where SIGPIPE was triggered).

[Note 1]

The trap handler might be worth explaining.

    rm -f $@; trap - $(sig); kill -s $(sig) $$

This lets the shell kill itself by the signal it caught, so the parent
process can tell the child has exited on the signal. Generally, this is
a proper manner for handling signals, in case the calling program (like
Bash) may monitor WIFSIGNALED() and WTERMSIG() for WCE although this may
not be a big deal here because GNU Make handles SIGHUP, SIGINT, SIGQUIT
in WUE and SIGTERM in IUE.

  IUE - Immediate Unconditional Exit
  WUE - Wait and Unconditional Exit
  WCE - Wait and Cooperative Exit

For details, see "Proper handling of SIGINT/SIGQUIT" [4].

[Note 2]

Reverting 392885ee82 ("kbuild: let fixdep directly write to .*.cmd
files") would directly address [1], but it only saves if_changed_dep.
As reported in [2], all commands that use redirection can potentially
leave an empty (i.e. broken) target.

[Note 3]

Another (even safer) approach might be to always write to a temporary
file, and rename it to $@ at the end of the recipe.

   <command>  > $(tmp-target)
   mv $(tmp-target) $@

It would require a lot of Makefile changes, and result in ugly code,
so I did not take it.

[Note 4]

A little more thoughts about a pattern rule with multiple targets (or
a grouped target).

    %.x %.y: %.z
            <recipe>

When interrupted, GNU Make deletes both %.x and %.y, while this solution
only deletes $@. Probably, this is not a big deal. The next run of make
will execute the rule again to create $@ along with the other files.

[1]: https://lore.kernel.org/all/YLeot94yAaM4xbMY@gmail.com/
[2]: https://lore.kernel.org/all/20220510221333.2770571-1-robh@kernel.org/
[3]: https://lists.gnu.org/archive/html/help-make/2021-06/msg00001.html
[4]: https://www.cons.org/cracauer/sigint.html

Fixes: 392885ee82 ("kbuild: let fixdep directly write to .*.cmd files")
Reported-by: Ingo Molnar <mingo@kernel.org>
Reported-by: Rob Herring <robh@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2022-09-29 02:00:29 +09:00
Masahiro Yamada
ebd191b38c kbuild: add cmd_and_savecmd macro
Separate out the command execution part of if_changed, as we did
for if_changed_dep.

This allows us to reuse it in if_changed_rule.

  define rule_foo
          $(call cmd_and_savecmd,foo)
          $(call cmd,bar)
  endef

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
2022-06-01 23:07:29 +09:00
Masahiro Yamada
c25e1c5582 kbuild: do not create *.prelink.o for Clang LTO or IBT
When CONFIG_LTO_CLANG=y, additional intermediate *.prelink.o is created
for each module. Also, objtool is postponed until LLVM IR is converted
to ELF.

CONFIG_X86_KERNEL_IBT works in a similar way to postpone objtool until
objects are merged together.

This commit stops generating *.prelink.o, so the build flow will look
similar with/without LTO.

The following figures show how the LTO build currently works, and
how this commit is changing it.

Current build flow
==================

 [1] single-object module

                                      $(LD)
           $(CC)                     +objtool              $(LD)
    foo.c --------------------> foo.o -----> foo.prelink.o -----> foo.ko
                              (LLVM IR)          (ELF)       |    (ELF)
                                                             |
                                                 foo.mod.o --/
                                                 (LLVM IR)

 [2] multi-object module
                                      $(LD)
           $(CC)         $(AR)       +objtool               $(LD)
    foo1.c -----> foo1.o -----> foo.o -----> foo.prelink.o -----> foo.ko
                           |  (archive)          (ELF)       |    (ELF)
    foo2.c -----> foo2.o --/                                 |
                 (LLVM IR)                       foo.mod.o --/
                                                 (LLVM IR)

  One confusion is that foo.o in multi-object module is an archive
  despite of its suffix.

New build flow
==============

 [1] single-object module

  Since there is only one object, there is no need to keep the LLVM IR.
  Use $(CC)+$(LD) to generate an ELF object in one build rule. When LTO
  is disabled, $(LD) is unneeded because $(CC) produces an ELF object.

               $(CC)+$(LD)+objtool              $(LD)
    foo.c ----------------------------> foo.o ---------> foo.ko
                                        (ELF)     |      (ELF)
                                                  |
                                      foo.mod.o --/
                                      (LLVM IR)

 [2] multi-object module

  Previously, $(AR) was used to combine LLVM IR files into an archive,
  but there was no technical reason to do so. Use $(LD) to merge them
  into a single ELF object.

                               $(LD)
             $(CC)            +objtool          $(LD)
    foo1.c ---------> foo1.o ---------> foo.o ---------> foo.ko
                                 |      (ELF)     |      (ELF)
    foo2.c ---------> foo2.o ----/                |
                     (LLVM IR)        foo.mod.o --/
                                      (LLVM IR)

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
2022-05-29 18:39:35 +09:00
Masahiro Yamada
b8c96a6b46 certs: simplify $(srctree)/ handling and remove config_filename macro
The complex macro, config_filename, was introduced to do:

 [1] drop double-quotes from the string value
 [2] add $(srctree)/ prefix in case the file is not found in $(objtree)
 [3] escape spaces and more

[1] will be more generally handled by Kconfig later.

As for [2], Kbuild uses VPATH to search for files in $(objtree),
$(srctree) in this order. GNU Make can natively handle it.

As for [3], converting $(space) to $(space_escape) back and forth looks
questionable to me. It is well-known that GNU Make cannot handle file
paths with spaces in the first place.

Instead of using the complex macro, use $< so it will be expanded to
the file path of the key.

Remove config_filename, finally.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-01-08 17:46:35 +09:00
Masahiro Yamada
e1f86d7b4b kbuild: warn if FORCE is missing for if_changed(_dep,_rule) and filechk
if_changed, if_changed_dep, and if_changed_rule must have FORCE as a
prerequisite so the command line change is detected.

Documentation/kbuild/makefiles.rst clearly explains it:

  Note: It is a typical mistake to forget the FORCE prerequisite.

However, not all people follow the document.

This mistake occurred again and again, so a compelling force is needed.

Show a warning if FORCE is missing in the prerequisite of if_changed
and friends. Same for filechk.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nicolas Schier <n.schier@avm.de>
2021-09-03 08:17:19 +09:00
Masahiro Yamada
6796e80409 kbuild: macrofy the condition of if_changed and friends
Add a new macro that expands into $(newer-prereqs)$(cmd-check).

It makes it easier to add common code for if_changed, if_changed_dep,
and if_changed_rule.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-09-03 08:17:19 +09:00
Masahiro Yamada
174a1dcc96 kbuild: sink stdout from cmd for silent build
When building with 'make -s', no output to stdout should be printed.

As Arnd Bergmann reported [1], mkimage shows the detailed information
of the generated images.

I think this should be suppressed by the 'cmd' macro instead of by
individual scripts.

Insert 'exec >/dev/null;' in order to redirect stdout to /dev/null for
silent builds.

[Note about this implementation]

'exec >/dev/null;' may look somewhat tricky, but this has a reason.

Appending '>/dev/null' at the end of command line is a common way for
redirection, so I first tried this:

  cmd = @set -e; $(echo-cmd) $(cmd_$(1)) >/dev/null

... but it would not work if $(cmd_$(1)) itself contains a redirection.

For example, cmd_wrap in scripts/Makefile.asm-generic redirects the
output from the 'echo' command into the target file.

It would be expanded into:

  echo "#include <asm-generic/$*.h>" > $@ >/dev/null

Then, the target file gets empty because the string will go to /dev/null
instead of $@.

Next, I tried this:

  cmd = @set -e; $(echo-cmd) { $(cmd_$(1)); } >/dev/null

The form above would be expanded into:

  { echo "#include <asm-generic/$*.h>" > $@; } >/dev/null

This works as expected. However, it would be a syntax error if
$(cmd_$(1)) is empty.

When CONFIG_TRIM_UNUSED_KSYMS is disabled, $(call cmd,gen_ksymdeps) in
scripts/Makefile.build would be expanded into:

  set -e;  { ; } >/dev/null

..., which causes an syntax error.

I also tried this:

  cmd = @set -e; $(echo-cmd) ( $(cmd_$(1)) ) >/dev/null

... but this causes a syntax error for the same reason.

So, finally I adopted:

  cmd = @set -e; $(echo-cmd) exec >/dev/null; $(cmd_$(1))

[1]: https://lore.kernel.org/lkml/20210514135752.2910387-1-arnd@kernel.org/

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-05-27 04:01:50 +09:00
Masahiro Yamada
57fd251c78 kbuild: split cc-option and friends to scripts/Makefile.compiler
scripts/Kbuild.include is included everywhere, but macros such as
cc-option are needed by build targets only.

For example, when 'make clean' traverses the tree, it does not need
to evaluate $(call cc-option,).

Split cc-option, ld-option, etc. to scripts/Makefile.compiler, which
is only included from the top Makefile and scripts/Makefile.build.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-04-25 05:09:32 +09:00
Masahiro Yamada
05f6bbf2d7 kbuild: remove ld-version macro
There is no direct user of ld-version; you can use CONFIG_LD_VERSION
if needed.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
2021-02-22 08:22:04 +09:00
Rasmus Villemoes
8402ee182c kbuild: remove leftover comment for filechk utility
After commit 43fee2b238 ("kbuild: do not redirect the first
prerequisite for filechk"), the rule is no longer automatically passed
$< as stdin, so remove the stale comment.

Fixes: 43fee2b238 ("kbuild: do not redirect the first prerequisite for filechk")
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-10-21 00:28:53 +09:00
Masahiro Yamada
132305b3b4 kbuild: stop filtering out $(GCC_PLUGINS_CFLAGS) from cc-option base
Commit d26e941492 ("kbuild: no gcc-plugins during cc-option tests")
was neeeded because scripts/Makefile.gcc-plugins was too early.

This is unneeded by including scripts/Makefile.gcc-plugins last,
and being careful to not add cc-option tests after it.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-08-10 01:32:59 +09:00
Masahiro Yamada
f2f02ebd8f kbuild: improve cc-option to clean up all temporary files
When cc-option and friends evaluate compiler flags, the temporary file
$$TMP is created as an output object, and automatically cleaned up.
The actual file path of $$TMP is .<pid>.tmp, here <pid> is the process
ID of $(shell ...) invoked from cc-option. (Please note $$$$ is the
escape sequence of $$).

Such garbage files are cleaned up in most cases, but some compiler flags
create additional output files.

For example, -gsplit-dwarf creates a .dwo file.

When CONFIG_DEBUG_INFO_SPLIT=y, you will see a bunch of .<pid>.dwo files
left in the top of build directories. You may not notice them unless you
do 'ls -a', but the garbage files will increase every time you run 'make'.

This commit changes the temporary object path to .tmp_<pid>/tmp, and
removes .tmp_<pid> directory when exiting. Separate build artifacts such
as *.dwo will be cleaned up all together because their file paths are
usually determined based on the base name of the object.

Another example is -ftest-coverage, which outputs the coverage data into
<base-name-of-object>.gcno

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-17 10:20:21 +09:00
Masahiro Yamada
30a7729771 kbuild: use -MMD instead of -MD to exclude system headers from dependency
This omits system headers from the generated header dependency.

System headers are not updated unless you upgrade the compiler. Nor do
they contain CONFIG options, so fixdep does not need to parse them.

Having said that, the effect of this optimization will be quite small
because the kernel code generally does not include system headers
except <stdarg.h>. Host programs include a lot of system headers,
but there are not so many in the kernel tree.

At first, keeping system headers in .*.cmd files might be useful to
detect the compiler update, but there is no guarantee that <stdarg.h>
is included from every file. So, I implemented a more reliable way in
the previous commit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-05-12 13:28:33 +09:00
Masahiro Yamada
88fe89a471 kbuild: remove *.tmp file when filechk fails
Bartosz Golaszewski reports that when "make {menu,n,g,x}config" fails
due to missing packages, a temporary file is left over, which is not
ignored by git.

For example, if GTK+ is not installed:

  $ make gconfig
  *
  * Unable to find the GTK+ installation. Please make sure that
  * the GTK+ 2.0 development package is correctly installed.
  * You need gtk+-2.0 gmodule-2.0 libglade-2.0
  *
  scripts/kconfig/Makefile:208: recipe for target 'scripts/kconfig/gconf-cfg' failed
  make[1]: *** [scripts/kconfig/gconf-cfg] Error 1
  Makefile:567: recipe for target 'gconfig' failed
  make: *** [gconfig] Error 2
  $ git status
  HEAD detached at v5.4
  Untracked files:
    (use "git add <file>..." to include in what will be committed)

          scripts/kconfig/gconf-cfg.tmp

  nothing added to commit but untracked files present (use "git add" to track)

This is because the check scripts are run with filechk, which misses
to clean up the temporary file on failure.

When the line

  { $(filechk_$(1)); } > $@.tmp;

... fails, it exits immediately due to the 'set -e'. Use trap to make
sure to delete the temporary file on exit.

For extra safety, I replaced $@.tmp with $(dot-target).tmp to make it
a hidden file.

Reported-by: Bartosz Golaszewski <bgolaszewski@baylibre.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-01-16 00:26:22 +09:00
Masahiro Yamada
8b41fc4454 kbuild: create modules.builtin without Makefile.modbuiltin or tristate.conf
Commit bc081dd6e9 ("kbuild: generate modules.builtin") added
infrastructure to generate modules.builtin, the list of all
builtin modules.

Basically, it works like this:

  - Kconfig generates include/config/tristate.conf, the list of
    tristate CONFIG options with a value in a capital letter.

  - scripts/Makefile.modbuiltin makes Kbuild descend into
    directories to collect the information of builtin modules.

I am not a big fan of it because Kbuild ends up with traversing
the source tree twice.

I am not sure how perfectly it should work, but this approach cannot
avoid false positives; even if the relevant CONFIG option is tristate,
some Makefiles forces obj-m to obj-y.

Some examples are:

  arch/powerpc/platforms/powermac/Makefile:
    obj-$(CONFIG_NVRAM:m=y)         += nvram.o

  net/ipv6/Makefile:
    obj-$(subst m,y,$(CONFIG_IPV6)) += inet6_hashtables.o

  net/netlabel/Makefile:
    obj-$(subst m,y,$(CONFIG_IPV6)) += netlabel_calipso.o

Nobody has complained about (or noticed) it, so it is probably fine to
have false positives in modules.builtin.

This commit simplifies the implementation. Let's exploit the fact
that every module has MODULE_LICENSE(). (modpost shows a warning if
MODULE_LICENSE is missing. If so, 0-day bot would already have blocked
such a module.)

I added MODULE_FILE to <linux/module.h>. When the code is being compiled
as builtin, it will be filled with the file path of the module, and
collected into modules.builtin.info. Then, scripts/link-vmlinux.sh
extracts the list of builtin modules out of it.

This new approach fixes the false-positives above, but adds another
type of false-positives; non-modular code may have MODULE_LICENSE()
by mistake. This is not a big deal, it is just the code is always
orphan. We can clean it up if we like. You can see cleanup examples by:

  $ git log --grep='make.* explicitly non-modular'

To sum up, this commits deletes lots of code, but still produces almost
equivalent results. Please note it does not increase the vmlinux size at
all. As you can see in include/asm-generic/vmlinux.lds.h, the .modinfo
section is discarded in the link stage.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-01-07 02:18:39 +09:00
Masahiro Yamada
7e826c44f5 kbuild: add stringify helper to quote a string passed to C files
Make $(squote)$(quote)...$(quote)$(squote) a helper macro.
I will reuse it in the next commit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-01-07 02:18:38 +09:00
Masahiro Yamada
eba19032f9 kbuild: rename any-prereq to newer-prereqs
GNU Make manual says:

  $?
      The names of all the prerequisites that are newer than the target,
      with spaces between them.

To reflect this, rename any-prereq to newer-prereqs, which is clearer
and more intuitive.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-11 20:10:01 +09:00
Masahiro Yamada
2d3b1b8f0d kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild
The incremental build of Linux kernel is pretty slow when lots of
objects are compiled. The rebuild of allmodconfig may take a few
minutes even when none of the objects needs to be rebuilt.

The time-consuming part in the incremental build is the evaluation of
if_changed* macros since they are used in the recipes to compile C and
assembly source files into objects.

I notice the following code in if_changed* is expensive:

  $(filter-out $(PHONY) $(wildcard $^),$^)

In the incremental build, every object has its .*.cmd file, which
contains the auto-generated list of included headers. So, $^ are
expanded into the long list of the source file + included headers,
and $(wildcard $^) checks whether they exist.

It may not be clear why this check exists there.

Here is the record of my research.

[1] The first code addition into Kbuild

This code dates back to 2002. It is the pre-git era. So, I copy-pasted
it from the historical git tree.

| commit 4a6db0791528c220655b063cf13fefc8470dbfee (HEAD)
| Author: Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
| Date:   Mon Jun 17 00:22:37 2002 -0500
|
|     kbuild: Handle removed headers
|
|     New and old way to handle dependencies would choke when a file
|     #include'd by other files was removed, since the dependency on it was
|     still recorded, but since it was gone, make has no idea what to do about
|     it (and would complain with "No rule to make <file> ...")
|
|     We now add targets for all the previously included files, so make will
|     just ignore them if they disappear.
|
| diff --git a/Rules.make b/Rules.make
| index 6ef827d3df39..7db5301ea7db 100644
| --- a/Rules.make
| +++ b/Rules.make
| @@ -446,7 +446,7 @@ if_changed = $(if $(strip $? \
|  # execute the command and also postprocess generated .d dependencies
|  # file
|
| -if_changed_dep = $(if $(strip $? \
| +if_changed_dep = $(if $(strip $? $(filter-out FORCE $(wildcard $^),$^)\
|                           $(filter-out $(cmd_$(1)),$(cmd_$@))\
|                           $(filter-out $(cmd_$@),$(cmd_$(1)))),\
|         @set -e; \
| diff --git a/scripts/fixdep.c b/scripts/fixdep.c
| index b5d7bee8efc7..db45bd1888c0 100644
| --- a/scripts/fixdep.c
| +++ b/scripts/fixdep.c
| @@ -292,7 +292,7 @@ void parse_dep_file(void *map, size_t len)
|                 exit(1);
|         }
|         memcpy(s, m, p-m); s[p-m] = 0;
| -       printf("%s: \\\n", target);
| +       printf("deps_%s := \\\n", target);
|         m = p+1;
|
|         clear_config();
| @@ -314,7 +314,8 @@ void parse_dep_file(void *map, size_t len)
|                 }
|                 m = p + 1;
|         }
| -       printf("\n");
| +       printf("\n%s: $(deps_%s)\n\n", target, target);
| +       printf("$(deps_%s):\n", target);
|  }
|
|  void print_deps(void)

The "No rule to make <file> ..." error can be solved by passing -MP to
the compiler, but I think the detection of header removal is a good
feature. When a header is removed, all source files that previously
included it should be re-compiled. This makes sure we has correctly
got rid of #include directives of it.

This is also related with the behavior of $?. The GNU Make manual says:

  $?
      The names of all the prerequisites that are newer than the target,
      with spaces between them.

This does not explain whether a non-existent prerequisite is considered
to be newer than the target.

At this point of time, GNU Make 3.7x was used, where the $? did not
include non-existent prerequisites. Therefore,

  $(filter-out FORCE $(wildcard $^),$^)

was useful to detect the header removal, and to rebuild the related
objects if it is the case.

[2] Change of $? behavior

Later, the behavior of $? was changed (fixed) to include prerequisites
that did not exist.

First, GNU Make commit 64e16d6c00a5 ("Various changes getting ready for
the release of 3.81.") changed it, but in the release test of 3.81, it
turned out to break the kernel build.

See these:

 - http://lists.gnu.org/archive/html/bug-make/2006-03/msg00003.html
 - https://savannah.gnu.org/bugs/?16002
 - https://savannah.gnu.org/bugs/?16051

Then, GNU Make commit 6d8d9b74d9c5 ("Numerous updates to tests for
issues found on Cygwin and Windows.") reverted it for the 3.81 release
to give Linux kernel time to adjust to the new behavior.

After the 3.81 release, GNU Make commit 7595f38f62af ("Fixed a number
of documentation bugs, plus some build/install issues:") re-added it.

[3] Adjustment to the new $? behavior on Kbuild side

Meanwhile, the kernel build was changed by commit 4f1933620f ("kbuild:
change kbuild to not rely on incorrect GNU make behavior") to adjust to
the new $? behavior.

[4] GNU Make 3.82 released in 2010

GNU Make 3.82 was the first release that integrated the correct $?
behavior. At this point, Kbuild dealt with GNU Make versions with
different $? behaviors.

 3.81 or older:
    $? does not contain any non-existent prerequisite.
    $(filter-out $(PHONY) $(wildcard $^),$^) was useful to detect
    removed include headers.

 3.82 or newer:
    $? contains non-existent prerequisites. When a header is removed,
    it appears in $?. $(filter-out $(PHONY) $(wildcard $^),$^) became
    a redundant check.

With the correct $? behavior, we could have dropped the expensive
check for 3.82 or later, but we did not. (Maybe nobody noticed this
optimization.)

[5] The .SECONDARY special target trips up $?

Some time later, I noticed $? did not work as expected under some
circumstances. As above, $? should contain non-existent prerequisites,
but the ones specified as SECONDARY do not appear in $?.

I asked this in GNU Make ML, and it seems a bug:

  https://lists.gnu.org/archive/html/bug-make/2019-01/msg00001.html

Since commit 8e9b61b293 ("kbuild: move .SECONDARY special target to
Kbuild.include"), all files, including headers listed in .*.cmd files,
are treated as secondary.

So, we are back into the incorrect $? behavior.

If we Kbuild want to react to the header removal, we need to keep
$(filter-out $(PHONY) $(wildcard $^),$^) but this makes the rebuild
so slow.

[Summary]

 - I believe noticing the header removal and recompiling related objects
   is a nice feature for the build system.

 - If $? worked correctly, $(filter-out $(PHONY),$?) would be enough
   to detect the header removal.

 - Currently, $? does not work correctly when used with .SECONDARY,
   and Kbuild is hit by this bug.

 - I filed a bug report for this, but not fixed yet as of writing.

 - Currently, the header removal is detected by the following expensive
   code:

    $(filter-out $(PHONY) $(wildcard $^),$^)

 - I do not want to revert commit 8e9b61b293 ("kbuild: move
   .SECONDARY special target to Kbuild.include"). Specifying
   .SECONDARY globally is clean, and it matches to the Kbuild policy.

This commit proactively removes the expensive check since it makes the
incremental build faster. A downside is Kbuild will no longer be able
to notice the header removal.

You can confirm it by the full-build followed by a header removal, and
then re-build.

  $ make defconfig all
    [ full build ]
  $ rm include/linux/device.h
  $ make
    CALL    scripts/checksyscalls.sh
    CALL    scripts/atomic/check-atomics.sh
    DESCEND  objtool
    CHK     include/generated/compile.h
  Kernel: arch/x86/boot/bzImage is ready  (#11)
    Building modules, stage 2.
    MODPOST 12 modules

Previously, Kbuild noticed a missing header and emits a build error.
Now, Kbuild is fine with it. This is an unusual corner-case, not a big
deal. Once the $? bug is fixed in GNU Make, everything will work fine.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-11 20:10:01 +09:00
Masahiro Yamada
13dc8c029c kbuild: remove ar-option and KBUILD_ARFLAGS
Commit 40df759e2b ("kbuild: Fix build with binutils <= 2.19")
introduced ar-option and KBUILD_ARFLAGS to deal with old binutils.

According to Documentation/process/changes.rst, the current minimal
supported version of binutils is 2.21 so you can assume the 'D' option
is always supported. Not only GNU ar but also llvm-ar supports it.

With the 'D' option hard-coded, there is no more user of ar-option
or KBUILD_ARFLAGS.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2019-10-01 09:20:33 +09:00
Masahiro Yamada
b2eff09218 kbuild: remove unused objectify macro
Commit 415008af32 ("docs-rst: convert lsm from DocBook to ReST")
removed the last users of this macro.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-27 12:18:19 +09:00
Masahiro Yamada
c04d1e46fc kbuild: remove unused hostcc-option
We can re-add this whenever it is needed. At this moment, it is unused.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-17 22:37:51 +09:00
Linus Torvalds
39ceda5ce1 Kbuild updates for v5.3
- remove headers_{install,check}_all targets
 
 - remove unreasonable 'depends on !UML' from CONFIG_SAMPLES
 
 - re-implement 'make headers_install' more cleanly
 
 - add new header-test-y syntax to compile-test headers
 
 - compile-test exported headers to ensure they are compilable in
   user-space
 
 - compile-test headers under include/ to ensure they are self-contained
 
 - remove -Waggregate-return, -Wno-uninitialized, -Wno-unused-value flags
 
 - add -Werror=unknown-warning-option for Clang
 
 - add 128-bit built-in types support to genksyms
 
 - fix missed rebuild of modules.builtin
 
 - propagate 'No space left on device' error in fixdep to Make
 
 - allow Clang to use its integrated assembler
 
 - improve some coccinelle scripts
 
 - add a new flag KBUILD_ABS_SRCTREE to request Kbuild to use absolute
   path for $(srctree).
 
 - do not ignore errors when compression utility is missing
 
 - misc cleanups
 -----BEGIN PGP SIGNATURE-----
 
 iQJSBAABCgA8FiEEbmPs18K1szRHjPqEPYsBB53g2wYFAl0oxNkeHHlhbWFkYS5t
 YXNhaGlyb0Bzb2Npb25leHQuY29tAAoJED2LAQed4NsGnhcP/AuM8s+3SYFiLitJ
 ISbznLFP2Xatq0SPXp5+moez/AMTK6Mm1biPcdo20d+TjVEh4+9F2nq12Ii9U8/D
 tds9A6G8+Bb28r9GMIVQPdFohijW6ijtDziS31iQnIWyPsP/yx6PKfLAD9F4ca1x
 7/4btmu+BOMjtN0NrMWSNz5MM47xUzoWIALL40SV4PzGVXLCQZ2PBNPeSRIk22Jt
 ynDNPuNsmDWcFfwAE+sLSDrhCHZlwM8rg8rf6jmYdc4LcN4cj0oho5+K1TRyC9mn
 fO3PT25juFejthxQulxEfyGggnyLM6BNTgPDGcCHSP4nD7mlXA9GcpZICtJOgGGu
 SlDadMZ0GRMK5zcZ0MF0GQboeyViwsbXgrRcYuXt6cUFWX4P/1SeAQ5Mf4u1EKqf
 hEbwFXV/g81ht0lFS8gyWkvdpoNPtxGHNPusLjp65C4rc0/48/s+7EE/u8JTPl1g
 dQTeIOds6XUOkJgqhEfuq+8gfngbjKc9bYhs+ACbkCzBltQdnb6m5aLgk0ODxe8I
 WbGn0+cQcS9VVwre7E5DnFSVWVOHAG5taiUwj0KDcHB0Jxw9Gvorq9WU1ppHHYH2
 XQIFBx7XHdn28d+plS8R23vAPgDgrGdvE5RYK5tNQLhTJ6BbjlZ1n/Tmxzu62scK
 deG3aCOB13Om7OTzTUh9+C3TC9ZQ
 =E2Rz
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

 - remove headers_{install,check}_all targets

 - remove unreasonable 'depends on !UML' from CONFIG_SAMPLES

 - re-implement 'make headers_install' more cleanly

 - add new header-test-y syntax to compile-test headers

 - compile-test exported headers to ensure they are compilable in
   user-space

 - compile-test headers under include/ to ensure they are self-contained

 - remove -Waggregate-return, -Wno-uninitialized, -Wno-unused-value
   flags

 - add -Werror=unknown-warning-option for Clang

 - add 128-bit built-in types support to genksyms

 - fix missed rebuild of modules.builtin

 - propagate 'No space left on device' error in fixdep to Make

 - allow Clang to use its integrated assembler

 - improve some coccinelle scripts

 - add a new flag KBUILD_ABS_SRCTREE to request Kbuild to use absolute
   path for $(srctree).

 - do not ignore errors when compression utility is missing

 - misc cleanups

* tag 'kbuild-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (49 commits)
  kbuild: use -- separater intead of $(filter-out ...) for cc-cross-prefix
  kbuild: Inform user to pass ARCH= for make mrproper
  kbuild: fix compression errors getting ignored
  kbuild: add a flag to force absolute path for srctree
  kbuild: replace KBUILD_SRCTREE with boolean building_out_of_srctree
  kbuild: remove src and obj from the top Makefile
  scripts/tags.sh: remove unused environment variables from comments
  scripts/tags.sh: drop SUBARCH support for ARM
  kbuild: compile-test kernel headers to ensure they are self-contained
  kheaders: include only headers into kheaders_data.tar.xz
  kheaders: remove meaningless -R option of 'ls'
  kbuild: support header-test-pattern-y
  kbuild: do not create wrappers for header-test-y
  kbuild: compile-test exported headers to ensure they are self-contained
  init/Kconfig: add CONFIG_CC_CAN_LINK
  kallsyms: exclude kasan local symbols on s390
  kbuild: add more hints about SUBDIRS replacement
  coccinelle: api/stream_open: treat all wait_.*() calls as blocking
  coccinelle: put_device: Add a cast to an expression for an assignment
  coccinelle: put_device: Adjust a message construction
  ...
2019-07-12 16:03:16 -07:00
Masahiro Yamada
d4a74bbfee kbuild: use -- separater intead of $(filter-out ...) for cc-cross-prefix
arch/mips/Makefile passes prefixes that start with '-' to cc-cross-prefix
when $(tool-archpref) evaluates to the empty string.

They are filtered-out before the $(shell ...) invocation. Otherwise,
'command -v' would be confused.

  $ command -v -linux-gcc
  bash: command: -l: invalid option
  command: usage: command [-pVv] command [arg ...]

Since commit 913ab9780f ("kbuild: use more portable 'command -v' for
cc-cross-prefix"), cc-cross-prefix throws away the stderr output, so
the console is not polluted in any way.

This is not a big deal in practice, but I see a slightly better taste
in adding '--' to teach it that '-linux-gcc' is an argument instead of
a command option.

This will cause extra forking of subshell, but it will not be noticeable
performance regression.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-11 23:34:52 +09:00
Masahiro Yamada
c2341e2a4f kbuild: save $(strip ...) for calling if_changed and friends
The string returned by $(filter-out ...) does not contain any leading
or trailing spaces.

With the previous commit, 'any-prereq' no longer contains any
excessive spaces.

Nor does 'cmd-check' since it expands to a $(filter-out ...) call.

So, only the space that matters is the one between 'any-prereq'
and 'cmd-check'.

By removing it from the code, we can save $(strip ...) evaluation.
This refactoring is possible because $(any-prereq)$(cmd-check) is only
passed to the first argument of $(if ...), so we are only interested
in whether or not it is empty.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-01 10:03:40 +09:00
Masahiro Yamada
93f31bbda4 kbuild: save $(strip ...) for calling any-prepreq
The string returned by $(filter-out ...) does not contain any leading
or trailing spaces.

So, only the space that matters is the one between

  $(filter-out $(PHONY),$?)

and

  $(filter-out $(PHONY) $(wildcard $^),$^)

By removing it from the code, we can save $(strip ...) evaluation.
This refactoring is possible because $(any-prereq) is only passed to
the first argument of $(if ...), so we are only interested in whether
or not it is empty.

This is also the prerequisite for the next commit.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-01 10:03:15 +09:00
Masahiro Yamada
50bcca6ac4 kbuild: rename arg-check to cmd-check
I prefer 'cmd-check' for consistency.

We have 'echo-cmd', 'cmd', 'cmd_and_fixdep', etc. in this file.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-01 10:03:07 +09:00
Masahiro Yamada
a5bae54c10 kbuild: move hdr-inst shorthand to top Makefile
Now that hdr-inst is used only in the top Makefile, move it there
from scripts/Kbuild.include.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-15 19:57:02 +09:00
Mauro Carvalho Chehab
cd238effef docs: kbuild: convert docs to ReST and rename to *.rst
The kbuild documentation clearly shows that the documents
there are written at different times: some use markdown,
some use their own peculiar logic to split sections.

Convert everything to ReST without affecting too much
the author's style and avoiding adding uneeded markups.

The conversion is actually:
  - add blank lines and identation in order to identify paragraphs;
  - fix tables markups;
  - add some lists markups;
  - mark literal blocks;
  - adjust title markups.

At its new index.rst, let's add a :orphan: while this is not linked to
the main index.rst file, in order to avoid build warnings.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-06-14 14:21:21 -06:00
Masahiro Yamada
913ab9780f kbuild: use more portable 'command -v' for cc-cross-prefix
To print the pathname that will be used by shell in the current
environment, 'command -v' is a standardized way. [1]

'which' is also often used in scripts, but it is less portable.

When I worked on commit bd55f96fa9 ("kbuild: refactor cc-cross-prefix
implementation"), I was eager to use 'command -v' but it did not work.
(The reason is explained below.)

I kept 'which' as before but got rid of '> /dev/null 2>&1' as I
thought it was no longer needed. Sorry, I was wrong.

It works well on my Ubuntu machine, but Alexey Brodkin reports noisy
warnings on CentOS7 when 'which' fails to find the given command in
the PATH environment.

  $ which foo
  which: no foo in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)

Given that behavior of 'which' depends on system (and it may not be
installed by default), I want to try 'command -v' once again.

The specification [1] clearly describes the behavior of 'command -v'
when the given command is not found:

  Otherwise, no output shall be written and the exit status shall reflect
  that the name was not found.

However, we need a little magic to use 'command -v' from Make.

$(shell ...) passes the argument to a subshell for execution, and
returns the standard output of the command.

Here is a trick. GNU Make may optimize this by executing the command
directly instead of forking a subshell, if no shell special characters
are found in the command and omitting the subshell will not change the
behavior.

In this case, no shell special character is used. So, Make will try
to run it directly. However, 'command' is a shell-builtin command,
then Make would fail to find it in the PATH environment:

  $ make ARCH=m68k defconfig
  make: command: Command not found
  make: command: Command not found
  make: command: Command not found

In fact, Make has a table of shell-builtin commands because it must
ask the shell to execute them.

Until recently, 'command' was missing in the table.

This issue was fixed by the following commit:

| commit 1af314465e5dfe3e8baa839a32a72e83c04f26ef
| Author: Paul Smith <psmith@gnu.org>
| Date:   Sun Nov 12 18:10:28 2017 -0500
|
|     * job.c: Add "command" as a known shell built-in.
|
|     This is not a POSIX shell built-in but it's common in UNIX shells.
|     Reported by Nick Bowler <nbowler@draconx.ca>.

Because the latest release is GNU Make 4.2.1 in 2016, this commit is
not included in any released versions. (But some distributions may
have back-ported it.)

We need to trick Make to spawn a subshell. There are various ways to
do so:

 1) Use a shell special character '~' as dummy

    $(shell : ~; command -v $(c)gcc)

 2) Use a variable reference that always expands to the empty string
    (suggested by David Laight)

    $(shell command$${x:+} -v $(c)gcc)

 3) Use redirect

    $(shell command -v $(c)gcc 2>/dev/null)

I chose 3) to not confuse people. The stderr would not be polluted
anyway, but it will provide extra safety, and is easy to understand.

Tested on Make 3.81, 3.82, 4.0, 4.1, 4.2, 4.2.1

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html

Fixes: bd55f96fa9 ("kbuild: refactor cc-cross-prefix implementation")
Cc: linux-stable <stable@vger.kernel.org> # 5.1
Reported-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Alexey Brodkin <abrodkin@synopsys.com>
2019-06-08 00:38:47 +09:00
Greg Kroah-Hartman
96ac6d4351 treewide: Add SPDX license identifier - Kbuild
Add SPDX license identifiers to all Make/Kconfig files which:

 - Have no license information of any form

These files fall under the project license, GPL v2 only. The resulting SPDX
license identifier is:

      GPL-2.0

Reported-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-30 11:32:33 -07:00
Nick Desaulniers
055efab312 kbuild: drop support for cc-ldoption
If you want to see if your linker supports a certain flag, then ask the
linker directly with ld-option (not the compiler with cc-ldoption).
Checking for linker flag support is an antipattern that complicates the
usage of various linkers other than bfd via -fuse-ld={bfd|gold|lld}.

Cc: clang-built-linux@googlegroups.com
Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-05-21 00:02:59 +09:00
Masahiro Yamada
cdd750bfb1 kbuild: remove 'addtree' and 'flags' magic for header search paths
The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated
and ugly.

As I mentioned in [1], Kbuild should stop automatic prefixing of header
search path options.

I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and
'flags' have been removed.

Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj)
to the header search path for O= building, but never touches extra
compiler options from ccflags-y etc.

[1]: https://patchwork.kernel.org/patch/9632347/

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-05-18 11:49:57 +09:00
Masahiro Yamada
d3a918c659 kbuild: remove cc-version macro
There is no more direct user of this macro; it is only used by
cc-ifversion.

Calling this macro is not efficient since it invokes the compiler to
get the compiler version. CONFIG_GCC_VERSION is already calculated in
the Kconfig stage, so Makefile can reuse it.

Here is a note about the slight difference between cc-version and
CONFIG_GCC_VERSION:

When using Clang, cc-version is evaluated to '0402' because Clang
defines __GNUC__ and __GNUC__MINOR__, and looks like GCC 4.2 in the
version point of view. On the other hand, CONFIG_GCC_VERSION=0
when $(CC) is clang.

There are currently two users of cc-ifversion:
  arch/mips/loongson64/Platform
  arch/powerpc/Makefile

They are not affected by this change.

The format of cc-version is <major><minor>, while CONFIG_GCC_VERSION
<major><minor><patch>. I adjusted cc-ifversion for the difference of
the number of digits.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-03-04 22:34:59 +09:00
Masahiro Yamada
bd55f96fa9 kbuild: refactor cc-cross-prefix implementation
- $(word 1, <text>) is equivalent to $(firstword <text>)

 - hardcode "gcc" instead of $(CC)

 - minimize the shell script part

A little more notes in case $(filter-out -%, ...) is not clear.

arch/mips/Makefile passes prefixes depending on the configuration.

CROSS_COMPILE := $(call cc-cross-prefix, $(tool-archpref)-linux- \
    $(tool-archpref)-linux-gnu- $(tool-archpref)-unknown-linux-gnu-)

In the Kconfig stage (e.g. when you run 'make defconfig'), neither
CONFIG_32BIT nor CONFIG_64BIT is defined. So, $(tool-archpref) is
empty. As a result, "-linux -linux-gnu- -unknown-linux-gnu" is passed
into cc-cross-prefix. The command 'which' assumes arguments starting
with a hyphen as command options, then emits the following messages:

  Illegal option -l
  Illegal option -l
  Illegal option -u

I think it is strange to define CROSS_COMPILE depending on the CONFIG
options since you need to feed $(CC) to Kconfig, but it is how MIPS
Makefile currently works. Anyway, it would not hurt to filter-out
invalid strings beforehand.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-27 21:41:27 +09:00
Masahiro Yamada
afa974b771 kbuild: add real-prereqs shorthand for $(filter-out FORCE,$^)
In Kbuild, if_changed and friends must have FORCE as a prerequisite.

Hence, $(filter-out FORCE,$^) or $(filter-out $(PHONY),$^) is a common
idiom to get the names of all the prerequisites except phony targets.

Add real-prereqs as a shorthand.

Note:
We cannot replace $(filter %.o,$^) in cmd_link_multi-m because $^ may
include auto-generated dependencies from the .*.cmd file when a single
object module is changed into a multi object module. Refer to commit
69ea912fda ("kbuild: remove unneeded link_multi_deps"). I added some
comment to avoid accidental breakage.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Rob Herring <robh@kernel.org>
2019-01-28 09:11:17 +09:00
Masahiro Yamada
bd352a739f kbuild: remove unused baseprereq
Commit eea199b445 ("kbuild: remove unnecessary LEX_PREFIX and
YACC_PREFIX") removed the last users of this macro.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-01-14 12:19:40 +09:00
Masahiro Yamada
ba97df4558 kbuild: use assignment instead of define ... endef for filechk_* rules
You do not have to use define ... endef for filechk_* rules.

For simple cases, the use of assignment looks cleaner, IMHO.

I updated the usage for scripts/Kbuild.include in case somebody
misunderstands the 'define ... endif' is the requirement.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
2019-01-06 10:22:35 +09:00
Masahiro Yamada
ad77408635 kbuild: change filechk to surround the given command with { }
filechk_* rules often consist of multiple 'echo' lines. They must be
surrounded with { } or ( ) to work correctly. Otherwise, only the
string from the last 'echo' would be written into the target.

Let's take care of that in the 'filechk' in scripts/Kbuild.include
to clean up filechk_* rules.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-01-06 09:46:51 +09:00
Linus Torvalds
668c35f69c Kbuild updates for v4.21
Kbuild core:
  - remove unneeded $(call cc-option,...) switches
  - consolidate Clang compiler flags into CLANG_FLAGS
  - announce the deprecation of SUBDIRS
  - fix single target build for external module
  - simplify the dependencies of 'prepare' stage targets
  - allow fixdep to directly write to .*.cmd files
  - simplify dependency generation for CONFIG_TRIM_UNUSED_KSYMS
  - change if_changed_rule to accept multi-line recipe
  - move .SECONDARY special target to scripts/Kbuild.include
  - remove redundant 'set -e'
  - improve parallel execution for CONFIG_HEADERS_CHECK
  - misc cleanups
 
 Treewide fixes and cleanups
  - set Clang flags correctly for PowerPC boot images
  - fix UML build error with CONFIG_GCC_PLUGINS
  - remove unneeded patterns from .gitignore files
  - refactor firmware/Makefile
  - remove unneeded rules for *offsets.s
  - avoid unneeded regeneration of intermediate .s files
  - clean up ./Kbuild
 
 Modpost:
  - remove unused -M, -K options
  - fix false positive warnings about section mismatch
  - use simple devtable lookup instead of linker magic
  - misc cleanups
 
 Coccinelle:
  - relax boolinit.cocci checks for overall consistency
  - fix warning messages of boolinit.cocci
 
 Other tools:
  - improve -dirty check of scripts/setlocalversion
  - add a tool to generate compile_commands.json from .*.cmd files
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJcJiVeAAoJED2LAQed4NsGXEMP/1fdkBfZGxYlxp4w1tEBeRKZ
 GInxxMfHiHZlu7nFPQ3k1mncczZjeLbnKcLaybnPmqBSfQe1F7tN0ux1sHtWkVfI
 DvqoKaHObZ3lEMEVxCHXQ2bKrN/j9nB2xSxzr4dvc9DscW8dwKElZuKVE7nHKdhl
 z70xsalxHGEGY6hptJrucbv8KTBOSleZ8Gaat79sEDkDSLCTjxXB3WcVMWqDT0M/
 IqN5QCwiPjZC3UCwuqq6+vnG1gyvDUORcbrVgHrBIKxLYAABYzugB5IlLpi8B31C
 ZUDmijSTandHd4SG5gw5uZoyYuK4YhZeI7g4yNyXSqnPurmcJxrGdpiuwRcE7xet
 5yB2uaNbAFO6Fbz4gUdDEvryA9IZEPPn1Z0Btfpp7UOxiWEqE61oHReCNdkiad94
 Oonl+ROZw5UOT3AZD3xCZTf9bTnQoCFccHTmnbaKqqSiDWxxLUXum7sNfnJ6GXEb
 fNQDuwuxtsPStaWoU93QNrGyl5e8jYTKFdphUCnZZ2/hE8ygoQ06PYmeBAk+UKfN
 UI2IqR8PHgmA97XJg+fHhbw6X4nQcPsg/usLqxGItAZNO2O4uXgaN24dOcHG74Bu
 2Dk/gQQB2sVB/Dxfmlaxvvj98MVg7IMtpusAT2bQ7miiSS3EqPFT8KQMZYLICWuZ
 u7QQe20yCho3ZULmsRwH
 =0PCt
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v4.21' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:
 "Kbuild core:
   - remove unneeded $(call cc-option,...) switches
   - consolidate Clang compiler flags into CLANG_FLAGS
   - announce the deprecation of SUBDIRS
   - fix single target build for external module
   - simplify the dependencies of 'prepare' stage targets
   - allow fixdep to directly write to .*.cmd files
   - simplify dependency generation for CONFIG_TRIM_UNUSED_KSYMS
   - change if_changed_rule to accept multi-line recipe
   - move .SECONDARY special target to scripts/Kbuild.include
   - remove redundant 'set -e'
   - improve parallel execution for CONFIG_HEADERS_CHECK
   - misc cleanups

  Treewide fixes and cleanups
   - set Clang flags correctly for PowerPC boot images
   - fix UML build error with CONFIG_GCC_PLUGINS
   - remove unneeded patterns from .gitignore files
   - refactor firmware/Makefile
   - remove unneeded rules for *offsets.s
   - avoid unneeded regeneration of intermediate .s files
   - clean up ./Kbuild

  Modpost:
   - remove unused -M, -K options
   - fix false positive warnings about section mismatch
   - use simple devtable lookup instead of linker magic
   - misc cleanups

  Coccinelle:
   - relax boolinit.cocci checks for overall consistency
   - fix warning messages of boolinit.cocci

  Other tools:
   - improve -dirty check of scripts/setlocalversion
   - add a tool to generate compile_commands.json from .*.cmd files"

* tag 'kbuild-v4.21' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (51 commits)
  kbuild: remove unused cmd_gentimeconst
  kbuild: remove $(obj)/ prefixes in ./Kbuild
  treewide: add intermediate .s files to targets
  treewide: remove explicit rules for *offsets.s
  firmware: refactor firmware/Makefile
  firmware: remove unnecessary patterns from .gitignore
  scripts: remove unnecessary ihex2fw and check-lc_ctypes from .gitignore
  um: remove unused filechk_gen_header in Makefile
  scripts: add a tool to produce a compile_commands.json file
  kbuild: add -Werror=implicit-int flag unconditionally
  kbuild: add -Werror=strict-prototypes flag unconditionally
  kbuild: add -fno-PIE flag unconditionally
  scripts: coccinelle: Correct warning message
  scripts: coccinelle: only suggest true/false in files that already use them
  kbuild: handle part-of-module correctly for *.ll and *.symtypes
  kbuild: refactor part-of-module
  kbuild: refactor quiet_modtag
  kbuild: remove redundant quiet_modtag for $(obj-m)
  kbuild: refactor Makefile.asm-generic
  user/Makefile: Fix typo and capitalization in comment section
  ...
2018-12-29 12:03:17 -08:00
Ingo Molnar
6ac389346e Revert "kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs"
This reverts commit 77b0bf55bc.

See this commit for details about the revert:

  e769742d35 ("Revert "x86/jump-labels: Macrofy inline assembly code to work around GCC inlining bugs"")

 Conflicts:
	arch/x86/Makefile

Reported-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Borislav Petkov <bp@alien8.de>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Richard Biener <rguenther@suse.de>
Cc: Kees Cook <keescook@chromium.org>
Cc: Segher Boessenkool <segher@kernel.crashing.org>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Nadav Amit <namit@vmware.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-12-19 12:00:28 +01:00
Masahiro Yamada
8e9b61b293 kbuild: move .SECONDARY special target to Kbuild.include
In commit 54a702f705 ("kbuild: mark $(targets) as .SECONDARY and
remove .PRECIOUS markers"), I missed one important feature of the
.SECONDARY target:

    .SECONDARY with no prerequisites causes all targets to be
    treated as secondary.

... which agrees with the policy of Kbuild.

Let's move it to scripts/Kbuild.include, with no prerequisites.

Note:
If an intermediate file is generated by $(call if_changed,...), you
still need to add it to "targets" so its .*.cmd file is included.

The arm/arm64 crypto files are generated by $(call cmd,shipped),
so they do not need to be added to "targets", but need to be added
to "clean-files" so "make clean" can properly clean them away.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-12-02 14:11:49 +09:00