linux/tools/bpf/bpftool/Makefile

275 lines
8.0 KiB
Makefile
Raw Normal View History

# SPDX-License-Identifier: GPL-2.0-only
include ../../scripts/Makefile.include
include ../../scripts/utilities.mak
ifeq ($(srctree),)
srctree := $(patsubst %/,%,$(dir $(CURDIR)))
srctree := $(patsubst %/,%,$(dir $(srctree)))
srctree := $(patsubst %/,%,$(dir $(srctree)))
endif
ifeq ($(V),1)
Q =
else
Q = @
endif
BPF_DIR = $(srctree)/tools/lib/bpf
ifneq ($(OUTPUT),)
bpftool: Install libbpf headers instead of including the dir Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-4-quentin@isovalent.com
2021-10-07 19:44:29 +00:00
_OUTPUT := $(OUTPUT)
else
bpftool: Install libbpf headers instead of including the dir Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-4-quentin@isovalent.com
2021-10-07 19:44:29 +00:00
_OUTPUT := $(CURDIR)
endif
bpftool: Install libbpf headers instead of including the dir Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-4-quentin@isovalent.com
2021-10-07 19:44:29 +00:00
BOOTSTRAP_OUTPUT := $(_OUTPUT)/bootstrap/
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
bpftool: Install libbpf headers instead of including the dir Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-4-quentin@isovalent.com
2021-10-07 19:44:29 +00:00
LIBBPF_OUTPUT := $(_OUTPUT)/libbpf/
LIBBPF_DESTDIR := $(LIBBPF_OUTPUT)
LIBBPF_INCLUDE := $(LIBBPF_DESTDIR)/include
LIBBPF_HDRS_DIR := $(LIBBPF_INCLUDE)/bpf
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
LIBBPF := $(LIBBPF_OUTPUT)libbpf.a
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
LIBBPF_BOOTSTRAP_OUTPUT := $(BOOTSTRAP_OUTPUT)libbpf/
LIBBPF_BOOTSTRAP_DESTDIR := $(LIBBPF_BOOTSTRAP_OUTPUT)
LIBBPF_BOOTSTRAP_INCLUDE := $(LIBBPF_BOOTSTRAP_DESTDIR)/include
LIBBPF_BOOTSTRAP_HDRS_DIR := $(LIBBPF_BOOTSTRAP_INCLUDE)/bpf
LIBBPF_BOOTSTRAP := $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
# We need to copy hashmap.h and nlattr.h which is not otherwise exported by
# libbpf, but still required by bpftool.
LIBBPF_INTERNAL_HDRS := $(addprefix $(LIBBPF_HDRS_DIR)/,hashmap.h nlattr.h)
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
LIBBPF_BOOTSTRAP_INTERNAL_HDRS := $(addprefix $(LIBBPF_BOOTSTRAP_HDRS_DIR)/,hashmap.h)
bpftool: Install libbpf headers instead of including the dir Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-4-quentin@isovalent.com
2021-10-07 19:44:29 +00:00
ifeq ($(BPFTOOL_VERSION),)
BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
endif
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
$(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT) $(LIBBPF_HDRS_DIR) $(LIBBPF_BOOTSTRAP_HDRS_DIR):
$(QUIET_MKDIR)mkdir -p $@
bpftool: Do not FORCE-build libbpf In bpftool's Makefile, libbpf has a FORCE dependency, to make sure we rebuild it in case its source files changed. Let's instead make the rebuild depend on the source files directly, through a call to the "$(wildcard ...)" function. This avoids descending into libbpf's directory if there is nothing to update. Do the same for the bootstrap libbpf version. This results in a slightly faster operation and less verbose output when running make a second time in bpftool's directory. Before: Auto-detecting system features: ... libbfd: [ on ] ... disassembler-four-args: [ on ] ... zlib: [ on ] ... libcap: [ on ] ... clang-bpf-co-re: [ on ] make[1]: Entering directory '/root/dev/linux/tools/lib/bpf' make[1]: Entering directory '/root/dev/linux/tools/lib/bpf' make[1]: Nothing to be done for 'install_headers'. make[1]: Leaving directory '/root/dev/linux/tools/lib/bpf' make[1]: Leaving directory '/root/dev/linux/tools/lib/bpf' After: Auto-detecting system features: ... libbfd: [ on ] ... disassembler-four-args: [ on ] ... zlib: [ on ] ... libcap: [ on ] ... clang-bpf-co-re: [ on ] Other ways to clean up the output could be to pass the "-s" option, or to redirect the output to >/dev/null, when calling make recursively to descend into libbpf's directory. However, this would suppress some useful output if something goes wrong during the build. A better alternative would be to pass "--no-print-directory" to the recursive make, but that would still leave us with some noise for "install_headers". Skipping the descent into libbpf's directory if no source file has changed works best, and seems the most logical option overall. Reported-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: John Fastabend <john.fastabend@gmail.com> Link: https://lore.kernel.org/bpf/20211009210341.6291-3-quentin@isovalent.com
2021-10-09 21:03:40 +00:00
$(LIBBPF): $(wildcard $(BPF_DIR)/*.[ch] $(BPF_DIR)/Makefile) | $(LIBBPF_OUTPUT)
bpftool: Install libbpf headers instead of including the dir Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-4-quentin@isovalent.com
2021-10-07 19:44:29 +00:00
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) \
DESTDIR=$(LIBBPF_DESTDIR) prefix= $(LIBBPF) install_headers
$(LIBBPF_INTERNAL_HDRS): $(LIBBPF_HDRS_DIR)/%.h: $(BPF_DIR)/%.h | $(LIBBPF_HDRS_DIR)
$(call QUIET_INSTALL, $@)
$(Q)install -m 644 -t $(LIBBPF_HDRS_DIR) $<
bpftool: Do not FORCE-build libbpf In bpftool's Makefile, libbpf has a FORCE dependency, to make sure we rebuild it in case its source files changed. Let's instead make the rebuild depend on the source files directly, through a call to the "$(wildcard ...)" function. This avoids descending into libbpf's directory if there is nothing to update. Do the same for the bootstrap libbpf version. This results in a slightly faster operation and less verbose output when running make a second time in bpftool's directory. Before: Auto-detecting system features: ... libbfd: [ on ] ... disassembler-four-args: [ on ] ... zlib: [ on ] ... libcap: [ on ] ... clang-bpf-co-re: [ on ] make[1]: Entering directory '/root/dev/linux/tools/lib/bpf' make[1]: Entering directory '/root/dev/linux/tools/lib/bpf' make[1]: Nothing to be done for 'install_headers'. make[1]: Leaving directory '/root/dev/linux/tools/lib/bpf' make[1]: Leaving directory '/root/dev/linux/tools/lib/bpf' After: Auto-detecting system features: ... libbfd: [ on ] ... disassembler-four-args: [ on ] ... zlib: [ on ] ... libcap: [ on ] ... clang-bpf-co-re: [ on ] Other ways to clean up the output could be to pass the "-s" option, or to redirect the output to >/dev/null, when calling make recursively to descend into libbpf's directory. However, this would suppress some useful output if something goes wrong during the build. A better alternative would be to pass "--no-print-directory" to the recursive make, but that would still leave us with some noise for "install_headers". Skipping the descent into libbpf's directory if no source file has changed works best, and seems the most logical option overall. Reported-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: John Fastabend <john.fastabend@gmail.com> Link: https://lore.kernel.org/bpf/20211009210341.6291-3-quentin@isovalent.com
2021-10-09 21:03:40 +00:00
$(LIBBPF_BOOTSTRAP): $(wildcard $(BPF_DIR)/*.[ch] $(BPF_DIR)/Makefile) | $(LIBBPF_BOOTSTRAP_OUTPUT)
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_BOOTSTRAP_OUTPUT) \
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
DESTDIR=$(LIBBPF_BOOTSTRAP_DESTDIR) prefix= \
ARCH= CC=$(HOSTCC) LD=$(HOSTLD) $@ install_headers
$(LIBBPF_BOOTSTRAP_INTERNAL_HDRS): $(LIBBPF_BOOTSTRAP_HDRS_DIR)/%.h: $(BPF_DIR)/%.h | $(LIBBPF_BOOTSTRAP_HDRS_DIR)
$(call QUIET_INSTALL, $@)
$(Q)install -m 644 -t $(LIBBPF_BOOTSTRAP_HDRS_DIR) $<
$(LIBBPF)-clean: FORCE | $(LIBBPF_OUTPUT)
$(call QUIET_CLEAN, libbpf)
tools: bpftool: improve and check builds for different make invocations There are a number of alternative "make" invocations that can be used to compile bpftool. The following invocations are expected to work: - through the kbuild system, from the top of the repository (make tools/bpf) - by telling make to change to the bpftool directory (make -C tools/bpf/bpftool) - by building the BPF tools from tools/ (cd tools && make bpf) - by running make from bpftool directory (cd tools/bpf/bpftool && make) Additionally, setting the O or OUTPUT variables should tell the build system to use a custom output path, for each of these alternatives. The following patch fixes the following invocations: $ make tools/bpf $ make tools/bpf O=<dir> $ make -C tools/bpf/bpftool OUTPUT=<dir> $ make -C tools/bpf/bpftool O=<dir> $ cd tools/ && make bpf O=<dir> $ cd tools/bpf/bpftool && make OUTPUT=<dir> $ cd tools/bpf/bpftool && make O=<dir> After this commit, the build still fails for two variants when passing the OUTPUT variable: $ make tools/bpf OUTPUT=<dir> $ cd tools/ && make bpf OUTPUT=<dir> In order to remember and check what make invocations are supposed to work, and to document the ones which do not, a new script is added to the BPF selftests. Note that some invocations require the kernel to be configured, so the script skips them if no .config file is found. v2: - In make_and_clean(), set $ERROR to 1 when "make" returns non-zero, even if the binary was produced. - Run "make clean" from the correct directory (bpf/ instead of bpftool/, when relevant). Reported-by: Lorenz Bauer <lmb@cloudflare.com> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com> Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-08-30 11:00:38 +00:00
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
$(LIBBPF_BOOTSTRAP)-clean: FORCE | $(LIBBPF_BOOTSTRAP_OUTPUT)
$(call QUIET_CLEAN, libbpf-bootstrap)
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_BOOTSTRAP_OUTPUT) clean >/dev/null
prefix ?= /usr/local
bash_compdir ?= /usr/share/bash-completion/completions
CFLAGS += -O2
CFLAGS += -W -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers
tools/bpftool: Turn off -Wnested-externs warning Turn off -Wnested-externs to avoid annoying warnings in BUILD_BUG_ON macro when compiling bpftool: In file included from /data/users/andriin/linux/tools/include/linux/build_bug.h:5, from /data/users/andriin/linux/tools/include/linux/kernel.h:8, from /data/users/andriin/linux/kernel/bpf/disasm.h:10, from /data/users/andriin/linux/kernel/bpf/disasm.c:8: /data/users/andriin/linux/kernel/bpf/disasm.c: In function ‘__func_get_name’: /data/users/andriin/linux/tools/include/linux/compiler.h:37:38: warning: nested extern declaration of ‘__compiletime_assert_0’ [-Wnested-externs] _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) ^~~~~~~~~~~~~~~~~~~~~ /data/users/andriin/linux/tools/include/linux/compiler.h:16:15: note: in definition of macro ‘__compiletime_assert’ extern void prefix ## suffix(void) __compiletime_error(msg); \ ^~~~~~ /data/users/andriin/linux/tools/include/linux/compiler.h:37:2: note: in expansion of macro ‘_compiletime_assert’ _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) ^~~~~~~~~~~~~~~~~~~ /data/users/andriin/linux/tools/include/linux/build_bug.h:39:37: note: in expansion of macro ‘compiletime_assert’ #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg) ^~~~~~~~~~~~~~~~~~ /data/users/andriin/linux/tools/include/linux/build_bug.h:50:2: note: in expansion of macro ‘BUILD_BUG_ON_MSG’ BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition) ^~~~~~~~~~~~~~~~ /data/users/andriin/linux/kernel/bpf/disasm.c:20:2: note: in expansion of macro ‘BUILD_BUG_ON’ BUILD_BUG_ON(ARRAY_SIZE(func_id_str) != __BPF_FUNC_MAX_ID); ^~~~~~~~~~~~ Signed-off-by: Andrii Nakryiko <andriin@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20200701212816.2072340-1-andriin@fb.com
2020-07-01 21:28:16 +00:00
CFLAGS += $(filter-out -Wswitch-enum -Wnested-externs,$(EXTRA_WARNINGS))
CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ \
-I$(if $(OUTPUT),$(OUTPUT),.) \
bpftool: Install libbpf headers instead of including the dir Bpftool relies on libbpf, therefore it relies on a number of headers from the library and must be linked against the library. The Makefile for bpftool exposes these objects by adding tools/lib as an include directory ("-I$(srctree)/tools/lib"). This is a working solution, but this is not the cleanest one. The risk is to involuntarily include objects that are not intended to be exposed by the libbpf. The headers needed to compile bpftool should in fact be "installed" from libbpf, with its "install_headers" Makefile target. In addition, there is one header which is internal to the library and not supposed to be used by external applications, but that bpftool uses anyway. Adjust the Makefile in order to install the header files properly before compiling bpftool. Also copy the additional internal header file (nlattr.h), but call it out explicitly. Build (and install headers) in a subdirectory under bpftool/ instead of tools/lib/bpf/. When descending from a parent Makefile, this is configurable by setting the OUTPUT, LIBBPF_OUTPUT and LIBBPF_DESTDIR variables. Also adjust the Makefile for BPF selftests, so as to reuse the (host) libbpf compiled earlier and to avoid compiling a separate version of the library just for bpftool. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211007194438.34443-4-quentin@isovalent.com
2021-10-07 19:44:29 +00:00
-I$(LIBBPF_INCLUDE) \
-I$(srctree)/kernel/bpf/ \
-I$(srctree)/tools/include \
-I$(srctree)/tools/include/uapi
CFLAGS += -DBPFTOOL_VERSION='"$(BPFTOOL_VERSION)"'
ifneq ($(EXTRA_CFLAGS),)
CFLAGS += $(EXTRA_CFLAGS)
endif
ifneq ($(EXTRA_LDFLAGS),)
LDFLAGS += $(EXTRA_LDFLAGS)
endif
INSTALL ?= install
RM ?= rm -f
FEATURE_USER = .bpftool
FEATURE_TESTS = libbfd disassembler-four-args reallocarray zlib libcap \
clang-bpf-co-re
FEATURE_DISPLAY = libbfd disassembler-four-args zlib libcap \
clang-bpf-co-re
check_feat := 1
NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install doc-uninstall
ifdef MAKECMDGOALS
ifeq ($(filter-out $(NON_CHECK_FEAT_TARGETS),$(MAKECMDGOALS)),)
check_feat := 0
endif
endif
ifeq ($(check_feat),1)
ifeq ($(FEATURES_DUMP),)
include $(srctree)/tools/build/Makefile.feature
else
include $(FEATURES_DUMP)
endif
endif
ifeq ($(feature-disassembler-four-args), 1)
CFLAGS += -DDISASM_FOUR_ARGS_SIGNATURE
endif
ifeq ($(feature-reallocarray), 0)
CFLAGS += -DCOMPAT_NEED_REALLOCARRAY
endif
LIBS = $(LIBBPF) -lelf -lz
LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz
ifeq ($(feature-libcap), 1)
CFLAGS += -DUSE_LIBCAP
LIBS += -lcap
endif
include $(wildcard $(OUTPUT)*.d)
all: $(OUTPUT)bpftool
BFD_SRCS = jit_disasm.c
SRCS = $(filter-out $(BFD_SRCS),$(wildcard *.c))
ifeq ($(feature-libbfd),1)
LIBS += -lbfd -ldl -lopcodes
else ifeq ($(feature-libbfd-liberty),1)
LIBS += -lbfd -ldl -lopcodes -liberty
else ifeq ($(feature-libbfd-liberty-z),1)
LIBS += -lbfd -ldl -lopcodes -liberty -lz
endif
ifneq ($(filter -lbfd,$(LIBS)),)
CFLAGS += -DHAVE_LIBBFD_SUPPORT
SRCS += $(BFD_SRCS)
endif
BPFTOOL_BOOTSTRAP := $(BOOTSTRAP_OUTPUT)bpftool
tools/bpftool: Fix cross-build When the bootstrap and final bpftool have different architectures, we need to build two distinct disasm.o objects. Add a recipe for the bootstrap disasm.o. After commit d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.") cross-building bpftool didn't work anymore, because the bootstrap bpftool was linked using objects from different architectures: $ make O=/tmp/bpftool ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/bpftool/ V=1 [...] aarch64-linux-gnu-gcc ... -c -MMD -o /tmp/bpftool/disasm.o /home/z/src/linux/kernel/bpf/disasm.c gcc ... -c -MMD -o /tmp/bpftool//bootstrap/main.o main.c gcc ... -o /tmp/bpftool//bootstrap/bpftool /tmp/bpftool//bootstrap/main.o ... /tmp/bpftool/disasm.o /usr/bin/ld: /tmp/bpftool/disasm.o: Relocations in generic ELF (EM: 183) /usr/bin/ld: /tmp/bpftool/disasm.o: Relocations in generic ELF (EM: 183) /usr/bin/ld: /tmp/bpftool/disasm.o: Relocations in generic ELF (EM: 183) /usr/bin/ld: /tmp/bpftool/disasm.o: error adding symbols: file in wrong format collect2: error: ld returned 1 exit status [...] The final bpftool was built for e.g. arm64, while the bootstrap bpftool, executed on the host, was built for x86. The problem here was that disasm.o linked into the bootstrap bpftool was arm64 rather than x86. With the fix we build two disasm.o, one for the target bpftool in arm64, and one for the bootstrap bpftool in x86. Fixes: d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.") Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20210603170515.1854642-1-jean-philippe@linaro.org
2021-06-03 17:05:16 +00:00
BOOTSTRAP_OBJS = $(addprefix $(BOOTSTRAP_OUTPUT),main.o common.o json_writer.o gen.o btf.o xlated_dumper.o btf_dumper.o disasm.o)
libbpf: Add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare the deprecation of two API functions. This macro marks functions as deprecated when libbpf's version reaches the values passed as an argument. As part of this change libbpf_version.h header is added with recorded major (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros. They are now part of libbpf public API and can be relied upon by user code. libbpf_version.h is installed system-wide along other libbpf public headers. Due to this new build-time auto-generated header, in-kernel applications relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to include libbpf's output directory as part of a list of include search paths. Better fix would be to use libbpf's make_install target to install public API headers, but that clean up is left out as a future improvement. The build changes were tested by building kernel (with KBUILD_OUTPUT and O= specified explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No problems were detected. Note that because of the constraints of the C preprocessor we have to write a few lines of macro magic for each version used to prepare deprecation (0.6 for now). Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of btf__get_from_id() and btf__load(), which are replaced by btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively, starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]). [0] Closes: https://github.com/libbpf/libbpf/issues/278 Co-developed-by: Quentin Monnet <quentin@isovalent.com> Co-developed-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20210908213226.1871016-1-andrii@kernel.org
2021-09-08 21:32:26 +00:00
$(BOOTSTRAP_OBJS): $(LIBBPF_BOOTSTRAP)
OBJS = $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
$(OBJS): $(LIBBPF) $(LIBBPF_INTERNAL_HDRS)
bpftool: Introduce "prog profile" command With fentry/fexit programs, it is possible to profile BPF program with hardware counters. Introduce bpftool "prog profile", which measures key metrics of a BPF program. bpftool prog profile command creates per-cpu perf events. Then it attaches fentry/fexit programs to the target BPF program. The fentry program saves perf event value to a map. The fexit program reads the perf event again, and calculates the difference, which is the instructions/cycles used by the target program. Example input and output: ./bpftool prog profile id 337 duration 3 cycles instructions llc_misses 4228 run_cnt 3403698 cycles (84.08%) 3525294 instructions # 1.04 insn per cycle (84.05%) 13 llc_misses # 3.69 LLC misses per million isns (83.50%) This command measures cycles and instructions for BPF program with id 337 for 3 seconds. The program has triggered 4228 times. The rest of the output is similar to perf-stat. In this example, the counters were only counting ~84% of the time because of time multiplexing of perf counters. Note that, this approach measures cycles and instructions in very small increments. So the fentry/fexit programs introduce noticeable errors to the measurement results. The fentry/fexit programs are generated with BPF skeletons. Therefore, we build bpftool twice. The first time _bpftool is built without skeletons. Then, _bpftool is used to generate the skeletons. The second time, bpftool is built with skeletons. Signed-off-by: Song Liu <songliubraving@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Quentin Monnet <quentin@isovalent.com> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20200309173218.2739965-2-songliubraving@fb.com
2020-03-09 17:32:15 +00:00
VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \
$(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \
../../../vmlinux \
/sys/kernel/btf/vmlinux \
/boot/vmlinux-$(shell uname -r)
VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS))))
bootstrap: $(BPFTOOL_BOOTSTRAP)
ifneq ($(VMLINUX_BTF)$(VMLINUX_H),)
ifeq ($(feature-clang-bpf-co-re),1)
BUILD_BPF_SKELS := 1
$(OUTPUT)vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL_BOOTSTRAP)
ifeq ($(VMLINUX_H),)
$(QUIET_GEN)$(BPFTOOL_BOOTSTRAP) btf dump file $< format c > $@
else
$(Q)cp "$(VMLINUX_H)" $@
endif
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
$(OUTPUT)%.bpf.o: skeleton/%.bpf.c $(OUTPUT)vmlinux.h $(LIBBPF_BOOTSTRAP)
$(QUIET_CLANG)$(CLANG) \
-I$(if $(OUTPUT),$(OUTPUT),.) \
-I$(srctree)/tools/include/uapi/ \
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
-I$(LIBBPF_BOOTSTRAP_INCLUDE) \
-g -O2 -Wall -target bpf -c $< -o $@ && $(LLVM_STRIP) -g $@
bpftool: Introduce "prog profile" command With fentry/fexit programs, it is possible to profile BPF program with hardware counters. Introduce bpftool "prog profile", which measures key metrics of a BPF program. bpftool prog profile command creates per-cpu perf events. Then it attaches fentry/fexit programs to the target BPF program. The fentry program saves perf event value to a map. The fexit program reads the perf event again, and calculates the difference, which is the instructions/cycles used by the target program. Example input and output: ./bpftool prog profile id 337 duration 3 cycles instructions llc_misses 4228 run_cnt 3403698 cycles (84.08%) 3525294 instructions # 1.04 insn per cycle (84.05%) 13 llc_misses # 3.69 LLC misses per million isns (83.50%) This command measures cycles and instructions for BPF program with id 337 for 3 seconds. The program has triggered 4228 times. The rest of the output is similar to perf-stat. In this example, the counters were only counting ~84% of the time because of time multiplexing of perf counters. Note that, this approach measures cycles and instructions in very small increments. So the fentry/fexit programs introduce noticeable errors to the measurement results. The fentry/fexit programs are generated with BPF skeletons. Therefore, we build bpftool twice. The first time _bpftool is built without skeletons. Then, _bpftool is used to generate the skeletons. The second time, bpftool is built with skeletons. Signed-off-by: Song Liu <songliubraving@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Quentin Monnet <quentin@isovalent.com> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20200309173218.2739965-2-songliubraving@fb.com
2020-03-09 17:32:15 +00:00
$(OUTPUT)%.skel.h: $(OUTPUT)%.bpf.o $(BPFTOOL_BOOTSTRAP)
$(QUIET_GEN)$(BPFTOOL_BOOTSTRAP) gen skeleton $< > $@
$(OUTPUT)prog.o: $(OUTPUT)profiler.skel.h
tools/bpftool: Show info for processes holding BPF map/prog/link/btf FDs Add bpf_iter-based way to find all the processes that hold open FDs against BPF object (map, prog, link, btf). bpftool always attempts to discover this, but will silently give up if kernel doesn't yet support bpf_iter BPF programs. Process name and PID are emitted for each process (task group). Sample output for each of 4 BPF objects: $ sudo ./bpftool prog show 2694: cgroup_device tag 8c42dee26e8cd4c2 gpl loaded_at 2020-06-16T15:34:32-0700 uid 0 xlated 648B jited 409B memlock 4096B pids systemd(1) 2907: cgroup_skb name egress tag 9ad187367cf2b9e8 gpl loaded_at 2020-06-16T18:06:54-0700 uid 0 xlated 48B jited 59B memlock 4096B map_ids 2436 btf_id 1202 pids test_progs(2238417), test_progs(2238445) $ sudo ./bpftool map show 2436: array name test_cgr.bss flags 0x400 key 4B value 8B max_entries 1 memlock 8192B btf_id 1202 pids test_progs(2238417), test_progs(2238445) 2445: array name pid_iter.rodata flags 0x480 key 4B value 4B max_entries 1 memlock 8192B btf_id 1214 frozen pids bpftool(2239612) $ sudo ./bpftool link show 61: cgroup prog 2908 cgroup_id 375301 attach_type egress pids test_progs(2238417), test_progs(2238445) 62: cgroup prog 2908 cgroup_id 375344 attach_type egress pids test_progs(2238417), test_progs(2238445) $ sudo ./bpftool btf show 1202: size 1527B prog_ids 2908,2907 map_ids 2436 pids test_progs(2238417), test_progs(2238445) 1242: size 34684B pids bpftool(2258892) Signed-off-by: Andrii Nakryiko <andriin@fb.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Reviewed-by: Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/bpf/20200619231703.738941-9-andriin@fb.com
2020-06-19 23:17:02 +00:00
$(OUTPUT)pids.o: $(OUTPUT)pid_iter.skel.h
endif
endif
bpftool: Introduce "prog profile" command With fentry/fexit programs, it is possible to profile BPF program with hardware counters. Introduce bpftool "prog profile", which measures key metrics of a BPF program. bpftool prog profile command creates per-cpu perf events. Then it attaches fentry/fexit programs to the target BPF program. The fentry program saves perf event value to a map. The fexit program reads the perf event again, and calculates the difference, which is the instructions/cycles used by the target program. Example input and output: ./bpftool prog profile id 337 duration 3 cycles instructions llc_misses 4228 run_cnt 3403698 cycles (84.08%) 3525294 instructions # 1.04 insn per cycle (84.05%) 13 llc_misses # 3.69 LLC misses per million isns (83.50%) This command measures cycles and instructions for BPF program with id 337 for 3 seconds. The program has triggered 4228 times. The rest of the output is similar to perf-stat. In this example, the counters were only counting ~84% of the time because of time multiplexing of perf counters. Note that, this approach measures cycles and instructions in very small increments. So the fentry/fexit programs introduce noticeable errors to the measurement results. The fentry/fexit programs are generated with BPF skeletons. Therefore, we build bpftool twice. The first time _bpftool is built without skeletons. Then, _bpftool is used to generate the skeletons. The second time, bpftool is built with skeletons. Signed-off-by: Song Liu <songliubraving@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Quentin Monnet <quentin@isovalent.com> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20200309173218.2739965-2-songliubraving@fb.com
2020-03-09 17:32:15 +00:00
CFLAGS += $(if $(BUILD_BPF_SKELS),,-DBPFTOOL_WITHOUT_SKELETONS)
tools/bpftool: Fix cross-build When the bootstrap and final bpftool have different architectures, we need to build two distinct disasm.o objects. Add a recipe for the bootstrap disasm.o. After commit d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.") cross-building bpftool didn't work anymore, because the bootstrap bpftool was linked using objects from different architectures: $ make O=/tmp/bpftool ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/bpf/bpftool/ V=1 [...] aarch64-linux-gnu-gcc ... -c -MMD -o /tmp/bpftool/disasm.o /home/z/src/linux/kernel/bpf/disasm.c gcc ... -c -MMD -o /tmp/bpftool//bootstrap/main.o main.c gcc ... -o /tmp/bpftool//bootstrap/bpftool /tmp/bpftool//bootstrap/main.o ... /tmp/bpftool/disasm.o /usr/bin/ld: /tmp/bpftool/disasm.o: Relocations in generic ELF (EM: 183) /usr/bin/ld: /tmp/bpftool/disasm.o: Relocations in generic ELF (EM: 183) /usr/bin/ld: /tmp/bpftool/disasm.o: Relocations in generic ELF (EM: 183) /usr/bin/ld: /tmp/bpftool/disasm.o: error adding symbols: file in wrong format collect2: error: ld returned 1 exit status [...] The final bpftool was built for e.g. arm64, while the bootstrap bpftool, executed on the host, was built for x86. The problem here was that disasm.o linked into the bootstrap bpftool was arm64 rather than x86. With the fix we build two disasm.o, one for the target bpftool in arm64, and one for the bootstrap bpftool in x86. Fixes: d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.") Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20210603170515.1854642-1-jean-philippe@linaro.org
2021-06-03 17:05:16 +00:00
$(BOOTSTRAP_OUTPUT)disasm.o: $(srctree)/kernel/bpf/disasm.c
$(QUIET_CC)$(HOSTCC) $(CFLAGS) -c -MMD -o $@ $<
$(OUTPUT)disasm.o: $(srctree)/kernel/bpf/disasm.c
$(QUIET_CC)$(CC) $(CFLAGS) -c -MMD -o $@ $<
bpftool: Turn check on zlib from a phony target into a conditional error One of bpftool's object files depends on zlib. To make sure we do not attempt to build that object when the library is not available, commit d66fa3c70e59 ("tools: bpftool: add feature check for zlib") introduced a feature check to detect whether zlib is present. This check comes as a rule for which the target ("zdep") is a nonexistent file (phony target), which means that the Makefile always attempts to rebuild it. It is mostly harmless. However, one side effect is that, on running again once bpftool is already built, make considers that "something" (the recipe for zdep) was executed, and does not print the usual message "make: Nothing to be done for 'all'", which is a user-friendly indicator that the build went fine. Before, with some level of debugging information: $ make --debug=m [...] Reading makefiles... Auto-detecting system features: ... libbfd: [ on ] ... disassembler-four-args: [ on ] ... zlib: [ on ] ... libcap: [ on ] ... clang-bpf-co-re: [ on ] Updating makefiles.... Updating goal targets.... File 'all' does not exist. File 'zdep' does not exist. Must remake target 'zdep'. File 'all' does not exist. Must remake target 'all'. Successfully remade target file 'all'. After the patch: $ make --debug=m [...] Auto-detecting system features: ... libbfd: [ on ] ... disassembler-four-args: [ on ] ... zlib: [ on ] ... libcap: [ on ] ... clang-bpf-co-re: [ on ] Updating makefiles.... Updating goal targets.... File 'all' does not exist. Must remake target 'all'. Successfully remade target file 'all'. make: Nothing to be done for 'all'. (Note the last line, which is not part of make's debug information.) Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: John Fastabend <john.fastabend@gmail.com> Link: https://lore.kernel.org/bpf/20211009210341.6291-4-quentin@isovalent.com
2021-10-09 21:03:41 +00:00
$(OUTPUT)feature.o:
ifneq ($(feature-zlib), 1)
$(error "No zlib found")
endif
$(BPFTOOL_BOOTSTRAP): $(BOOTSTRAP_OBJS) $(LIBBPF_BOOTSTRAP)
$(QUIET_LINK)$(HOSTCC) $(CFLAGS) $(LDFLAGS) -o $@ $(BOOTSTRAP_OBJS) \
$(LIBS_BOOTSTRAP)
$(OUTPUT)bpftool: $(OBJS) $(LIBBPF)
$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(OBJS) $(LIBS)
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
$(BOOTSTRAP_OUTPUT)%.o: %.c $(LIBBPF_BOOTSTRAP_INTERNAL_HDRS) | $(BOOTSTRAP_OUTPUT)
$(QUIET_CC)$(HOSTCC) \
$(subst -I$(LIBBPF_INCLUDE),-I$(LIBBPF_BOOTSTRAP_INCLUDE),$(CFLAGS)) \
-c -MMD -o $@ $<
$(OUTPUT)%.o: %.c
$(QUIET_CC)$(CC) $(CFLAGS) -c -MMD -o $@ $<
feature-detect-clean:
$(call QUIET_CLEAN, feature-detect)
$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
clean: $(LIBBPF)-clean $(LIBBPF_BOOTSTRAP)-clean feature-detect-clean
$(call QUIET_CLEAN, bpftool)
$(Q)$(RM) -- $(OUTPUT)bpftool $(OUTPUT)*.o $(OUTPUT)*.d
$(Q)$(RM) -- $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
$(Q)$(RM) -r -- $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT)
$(call QUIET_CLEAN, core-gen)
$(Q)$(RM) -- $(OUTPUT)FEATURE-DUMP.bpftool
$(Q)$(RM) -r -- $(OUTPUT)feature/
install-bin: $(OUTPUT)bpftool
$(call QUIET_INSTALL, bpftool)
$(Q)$(INSTALL) -m 0755 -d $(DESTDIR)$(prefix)/sbin
$(Q)$(INSTALL) $(OUTPUT)bpftool $(DESTDIR)$(prefix)/sbin/bpftool
install: install-bin
$(Q)$(INSTALL) -m 0755 -d $(DESTDIR)$(bash_compdir)
$(Q)$(INSTALL) -m 0644 bash-completion/bpftool $(DESTDIR)$(bash_compdir)
uninstall:
$(call QUIET_UNINST, bpftool)
$(Q)$(RM) -- $(DESTDIR)$(prefix)/sbin/bpftool
$(Q)$(RM) -- $(DESTDIR)$(bash_compdir)/bpftool
doc:
$(call descend,Documentation)
doc-clean:
$(call descend,Documentation,clean)
doc-install:
$(call descend,Documentation,install)
doc-uninstall:
$(call descend,Documentation,uninstall)
FORCE:
.SECONDARY:
bpftool: Install libbpf headers for the bootstrap version, too We recently changed bpftool's Makefile to make it install libbpf's headers locally instead of pulling them from the source directory of the library. Although bpftool needs two versions of libbpf, a "regular" one and a "bootstrap" version, we would only install headers for the regular libbpf build. Given that this build always occurs before the bootstrap build when building bpftool, this is enough to ensure that the bootstrap bpftool will have access to the headers exported through the regular libbpf build. However, this did not account for the case when we only want the bootstrap version of bpftool, through the "bootstrap" target. For example, perf needs the bootstrap version only, to generate BPF skeletons. In that case, when are the headers installed? For some time, the issue has been masked, because we had a step (the installation of headers internal to libbpf) which would depend on the regular build of libbpf and hence trigger the export of the headers, just for the sake of creating a directory. But this changed with commit 8b6c46241c77 ("bpftool: Remove Makefile dep. on $(LIBBPF) for $(LIBBPF_INTERNAL_HDRS)"), where we cleaned up that stage and removed the dependency on the regular libbpf build. As a result, when we only want the bootstrap bpftool version, the regular libbpf is no longer built. The bootstrap libbpf version is built, but headers are not exported, and the bootstrap bpftool build fails because of the missing headers. To fix this, we also install the library headers for the bootstrap version of libbpf, to use them for the bootstrap bpftool and for generating the skeletons. Fixes: f012ade10b34 ("bpftool: Install libbpf headers instead of including the dir") Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Link: https://lore.kernel.org/bpf/20211105015813.6171-1-quentin@isovalent.com
2021-11-05 01:58:13 +00:00
.PHONY: all FORCE bootstrap clean install-bin install uninstall
.PHONY: doc doc-clean doc-install doc-uninstall
.DEFAULT_GOAL := all