With this commit, the error report is shown earlier, even before
running kconfig.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

If you try out-of-tree build with an unclean source tree, Kbuild
suggests to run make mrproper. The path to the source tree may be
shown with a relative path, for example, "make O=foo" emits the
following:

  .. is not clean, please run 'make mrproper'
  in the '..' directory.

This is somewhat confusing if you ran "make O=foo" in the source tree.
Using the absolute path will be clearer.

This commit changes the error message like follows:

***
*** The source tree is not clean, please run 'make mrproper'
*** in /absolute/path/to/linux
***
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Since commit 3a475b21 ("kbuild: Inform user to pass ARCH= for make
mrproper"), if you try out-of-tree build with an unclean source tree,
it suggests to run 'make ARCH=<ARCH> mrproper'.

This looks odd when you are not cross-compiling the kernel. Show the
'ARCH=<ARCH>' part only when ARCH= was given from the command line.
If ARCH is the default (native build) or came from the environment,
it should simply suggest 'make mrproper' as before.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

You already know the location of the source tree without this message.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Both relative path and absolute path have pros and cons. For example,
we can move the source and objtree around together by using the
relative path to the source tree.

Do not force the absolute path to the source tree. If you prefer the
absolute path, you can specify KBUILD_ABS_SRCTREE=1.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit 415008af ("docs-rst: convert lsm from DocBook to ReST")
stopped using if_changed_rule. There is no more users of if_changed*
for the doc targets. Hence, fixdep is unneeded. Remove the dependency
on scripts_basic.

All the doc targets are phony. The dependency on FORCE is not needed
either.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Kbuild descends into scripts/basic/ even before the Kconfig.
I do not expect any other host programs added to this Makefile.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

CONFIG_CC_OPTIMIZE_FOR_SIZE was originally an independent boolean
option, but commit 877417e6 ("Kbuild: change CC_OPTIMIZE_FOR_SIZE
definition") turned it into a choice between _PERFORMANCE and _SIZE.

The phrase "If unsure, say N." sounds like an independent option.
Reword the help text to make it appropriate for the choice menu.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Commit 055efab3 ("kbuild: drop support for cc-ldoption") correctly
removed the cc-ldoption from Documentation/kbuild/makefiles.txt, but
commit cd238eff ("docs: kbuild: convert docs to ReST and rename
to *.rst") revived it. I guess it was a rebase mistake.

Remove it again.

Fixes: cd238eff ("docs: kbuild: convert docs to ReST and rename to *.rst")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

I see the following warnings when I open this document with a ReST
viewer, retext:

/home/masahiro/ref/linux/Documentation/kbuild/makefiles.rst:1142: (WARNING/2) Inline emphasis start-string without end-string.
/home/masahiro/ref/linux/Documentation/kbuild/makefiles.rst:1152: (WARNING/2) Inline emphasis start-string without end-string.
/home/masahiro/ref/linux/Documentation/kbuild/makefiles.rst:1154: (WARNING/2) Inline emphasis start-string without end-string.

These hunks were added by commit e846f0dc ("kbuild: add support
for ensuring headers are self-contained") and commit 1e21cbfa
("kbuild: support header-test-pattern-y"), respectively. They were
written not for ReST but for the plain text, and merged via the
kbuild tree.

In the same development cycle, this document was converted to ReST
by commit cd238eff ("docs: kbuild: convert docs to ReST and rename
to *.rst"), and merged via the doc sub-system.

Merging them together into Linus' tree resulted in the current situation.

To fix the syntax, surround the asterisks with back-quotes, and
use :: for the code sample.

Fixes: 39ceda5c ("Merge tag 'kbuild-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This line contains $(MAKE), so Make knows that it will invoke sub-make
without help of the '+' marker.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

scripts/package/Makefile does not use $(obj) or $(src) at all.
It actually generates files and directories in the top of $(objtree).
I do not see much sense in descending into scripts/package/.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

These are not real targets. Adding them to PHONY is preferred.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

I am not a big fan of the $(objtree)/ hack for clean-files/clean-dirs.

These are created in the top of $(objtree), so let's clean them up
from the top Makefile.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

When we execute make after merging the configurations we ignore any
errors it produces causing whatever is running merge_config.sh to be
unaware of any failures.  This issue was noticed by Guillaume Tucker
while looking at problems with testing of clang only builds in KernelCI
which caused Kbuild to be unable to find a working host compiler.

This implementation was suggested by Yamada-san.
Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reported-by: Guillaume Tucker <guillaume.tucker@collabora.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Makefile.lib is included by Makefile.modfinal as well as Makefile.build.

Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c
in Makefile.modfinal. Move modkern_cflags as well for consistency.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Add CONFIG_ASM_MODVERSIONS. This allows to remove one if-conditional
nesting in scripts/Makefile.build.

scripts/Makefile.build is run every time Kbuild descends into a
sub-directory. So, I want to avoid $(wildcard ...) evaluation
where possible although computing $(wildcard ...) is so cheap that
it may not make measurable performance difference.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>

The pattern '*.order' was added by commit c6025f4c ("kbuild: ignore
*.order files") to ignore modules.order files.

I do not see any other user of the '.order' extension.

Ignore 'modules.order' explicitly instead of '*.order'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

I think splitting the modpost and linking modules into separate
Makefiles will be useful especially when more complex build steps
come in. The main motivation of this commit is to integrate the
proposed klp-convert feature cleanly.

I moved the logging 'Building modules, stage 2.' to Makefile.modpost
to avoid the code duplication although I do not know whether or not
this message is needed in the first place.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, the timestamp of module linker scripts are not checked.
Add them to the dependency of modules so they are correctly rebuilt.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

These three variables are not intended to be tweaked by users.
Move them from kbuild.rst to makefiles.rst.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Now that the single target build descends into sub-directories in the
same way as the normal build, these dummy Makefiles are not needed
any more.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, the single target build directly descends into the directory
of the target. For example,

  $ make foo/bar/baz.o

... directly descends into foo/bar/.

On the other hand, the normal build usually descends one directory at
a time, i.e. descends into foo/, and then foo/bar/.

This difference causes some problems.

[1] miss subdir-asflags-y, subdir-ccflags-y in upper Makefiles

    The options in subdir-{as,cc}flags-y take effect in the current
    and its sub-directories. In other words, they are inherited
    downward. In the example above, the single target will miss
    subdir-{as,cc}flags-y if they are defined in foo/Makefile.

[2] could be built in a different directory

    As Documentation/kbuild/modules.rst section 4.3 says, Kbuild can
    handle files that are spread over several sub-directories.

    The build rule of foo/bar/baz.o may not necessarily be specified in
    foo/bar/Makefile. It might be specifies in foo/Makefile as follows:

    [foo/Makefile]
    obj-y := bar/baz.o

    This often happens when a module is so big that its source files
    are divided into sub-directories.

    In this case, there is no Makefile in the foo/bar/ directory, yet
    the single target descends into foo/bar/, then fails due to the
    missing Makefile. You can still do 'make foo/bar/' for partial
    building, but cannot do 'make foo/bar/baz.s'. I believe the single
    target '%.s' is a useful feature for inspecting the compiler output.

    Some modules work around this issue by putting an empty Makefile
    in every sub-directory.

This commit fixes those problems by making the single target build
descend in the same way as the normal build does.

Another change is the single target build will observe the CONFIG
options. Previously, it allowed users to build the foo.o even when
the corresponding CONFIG_FOO is disabled:

   obj-$(CONFIG_FOO) += foo.o

In the new behavior, the single target build will just fail and show
"No rule to make target ..." (or "Nothing to be done for ..." if the
stale object already exists, but cannot be updated).

The disadvantage of this commit is the build speed. Now that the
single target build visits every directory and parses lots of
Makefiles, it is slower than before. (But, I hope it will not be
too slow.)
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

When kallsyms generation happens, temporary vmlinux outputs are linked
but the quiet make output didn't report it, giving the impression that
the prior command is taking longer than expected.

Instead, report the linking step explicitly. While at it, this
consolidates the repeated "kallsyms generation step" into a single
function and removes the existing copy/pasting.
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

scripts/headers_check.pl can detect references to CONFIG options in
exported headers, but it has been disabled for more than a decade.

Reverting commit 7e3fa561 ("kbuild: drop check for CONFIG_ in
headers_check") would emit the following warnings for headers_check
on x86:

usr/include/mtd/ubi-user.h:283: leaks CONFIG_MTD_UBI_BEB_LIMIT to userspace where it is not valid
usr/include/linux/cm4000_cs.h:26: leaks CONFIG_COMPAT to userspace where it is not valid
usr/include/linux/pkt_cls.h:301: leaks CONFIG_NET_CLS_ACT to userspace where it is not valid
usr/include/linux/videodev2.h:2465: leaks CONFIG_VIDEO_ADV_DEBUG to userspace where it is not valid
usr/include/linux/bpf.h:249: leaks CONFIG_EFFICIENT_UNALIGNED_ACCESS to userspace where it is not valid
usr/include/linux/bpf.h:819: leaks CONFIG_CGROUP_NET_CLASSID to userspace where it is not valid
usr/include/linux/bpf.h:1011: leaks CONFIG_IP_ROUTE_CLASSID to userspace where it is not valid
usr/include/linux/bpf.h:1742: leaks CONFIG_BPF_KPROBE_OVERRIDE to userspace where it is not valid
usr/include/linux/bpf.h:1747: leaks CONFIG_FUNCTION_ERROR_INJECTION to userspace where it is not valid
usr/include/linux/bpf.h:1936: leaks CONFIG_XFRM to userspace where it is not valid
usr/include/linux/bpf.h:2184: leaks CONFIG_BPF_LIRC_MODE2 to userspace where it is not valid
usr/include/linux/bpf.h:2210: leaks CONFIG_BPF_LIRC_MODE2 to userspace where it is not valid
usr/include/linux/bpf.h:2227: leaks CONFIG_SOCK_CGROUP_DATA to userspace where it is not valid
usr/include/linux/bpf.h:2311: leaks CONFIG_NET to userspace where it is not valid
usr/include/linux/bpf.h:2348: leaks CONFIG_NET to userspace where it is not valid
usr/include/linux/bpf.h:2422: leaks CONFIG_BPF_LIRC_MODE2 to userspace where it is not valid
usr/include/linux/bpf.h:2528: leaks CONFIG_NET to userspace where it is not valid
usr/include/linux/pktcdvd.h:37: leaks CONFIG_CDROM_PKTCDVD_WCACHE to userspace where it is not valid
usr/include/linux/hw_breakpoint.h:27: leaks CONFIG_HAVE_MIXED_BREAKPOINTS_REGS to userspace where it is not valid
usr/include/linux/raw.h:17: leaks CONFIG_MAX_RAW_DEVS to userspace where it is not valid
usr/include/linux/elfcore.h:62: leaks CONFIG_BINFMT_ELF_FDPIC to userspace where it is not valid
usr/include/linux/eventpoll.h:82: leaks CONFIG_PM_SLEEP to userspace where it is not valid
usr/include/linux/atmdev.h:104: leaks CONFIG_COMPAT to userspace where it is not valid
usr/include/asm-generic/unistd.h:651: leaks CONFIG_MMU to userspace where it is not valid
usr/include/asm-generic/bitsperlong.h:9: leaks CONFIG_64BIT to userspace where it is not valid
usr/include/asm-generic/fcntl.h:119: leaks CONFIG_64BIT to userspace where it is not valid
usr/include/asm/auxvec.h:14: leaks CONFIG_IA32_EMULATION to userspace where it is not valid
usr/include/asm/e820.h:14: leaks CONFIG_NODES_SHIFT to userspace where it is not valid
usr/include/asm/e820.h:39: leaks CONFIG_X86_PMEM_LEGACY to userspace where it is not valid
usr/include/asm/e820.h:49: leaks CONFIG_INTEL_TXT to userspace where it is not valid
usr/include/asm/mman.h:7: leaks CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS to userspace where it is not valid

Most of these are false positives because scripts/headers_check.pl
parses comment lines.

It is also false negative. arch/x86/include/uapi/asm/auxvec.h contains
CONFIG_IA32_EMULATION and CONFIG_X86_64, but the only former is reported.

It would be possible to fix scripts/headers_check.pl, of course.
However, we already have some duplicated checks between headers_check
and CONFIG_UAPI_HEADER_TEST. At this moment of time, there are still
dozens of headers excluded from the header test (usr/include/Makefile),
but we might be able to remove headers_check eventually.

I re-implemented it in scripts/headers_install.sh by using sed because
the most of code in scripts/headers_install.sh is written in sed.

This patch works like this:

[1] Run scripts/unifdef first because we need to drop the code
    surrounded by #ifdef __KERNEL__ ... #endif

[2] Remove all C style comments. The sed code is somewhat complicated
    since we need to deal with both single and multi line comments.

    Precisely speaking, a comment block is replaced with a space just
    in case.

      CONFIG_FOO/* this is a comment */CONFIG_BAR

    should be converted into:

      CONFIG_FOO CONFIG_BAR

    instead of:

      CONFIG_FOOCONFIG_BAR

[3] Match CONFIG_... pattern. It correctly matches to all CONFIG
    options that appear in a single line.

After this commit, this would detect the following warnings, all of
which are real ones.

warning: include/uapi/linux/pktcdvd.h: leak CONFIG_CDROM_PKTCDVD_WCACHE to user-space
warning: include/uapi/linux/hw_breakpoint.h: leak CONFIG_HAVE_MIXED_BREAKPOINTS_REGS to user-space
warning: include/uapi/linux/raw.h: leak CONFIG_MAX_RAW_DEVS to user-space
warning: include/uapi/linux/elfcore.h: leak CONFIG_BINFMT_ELF_FDPIC to user-space
warning: include/uapi/linux/eventpoll.h: leak CONFIG_PM_SLEEP to user-space
warning: include/uapi/linux/atmdev.h: leak CONFIG_COMPAT to user-space
warning: include/uapi/asm-generic/fcntl.h: leak CONFIG_64BIT to user-space
warning: arch/x86/include/uapi/asm/auxvec.h: leak CONFIG_IA32_EMULATION to user-space
warning: arch/x86/include/uapi/asm/auxvec.h: leak CONFIG_X86_64 to user-space
warning: arch/x86/include/uapi/asm/mman.h: leak CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS to user-space

However, it is not nice to show them right now. I created a list of
existing leakages. They are not warned, but a new leakage will be
blocked by the 0-day bot.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>

Factor out the duplicated code for in-kernel and external module
cleaning.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

The in-kernel build and external module build have similar code
for descending into sub-directories.

Factor out the code into the common place.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

There is no need to set 0 to variables such as config-targets,
mixed-targets, etc.

Unset instead of setting 0 in order to use 'ifdef' to test them.

I also renamed:

  config-targets  ->  config-build
  mixed-targets   ->  mixed-build
  dot-config      ->  need-config

to clarify what we are doing.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

'make clean' descends into ./Kbuild, but does not clean anything
since everything is added to no-clean-files.

There is no need to descend to ./Kbuild in the first place.
We can drop the no-clean-files assignment.

With this, there is no more user of no-clean-files. I will keep it
for a while to see whether a new user will appear.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

timeconst.h is generated by $(call filechk,...), missing-syscalls and
old-atomics are invoked by $(call cmd,...)

None of them needs to be added to 'targets'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

'make /' is just an alias for 'make ./'; this builds all objects of an
external module, but skips the modpost stage.

I am not a big fan of 'make /' since it looks as if it were touching
the root directory of the system. I like 'make ./' better.

I do not know how many people are using it, but let's show a hint if
it is used. Also, move it close to the external module rules since
this only makes sense for external modules.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

For the single target building %.symtypes from %.S, $(a_flags) is
expanded into the _KERNEL flags even if the object is a part of a
module.

$(real-obj-m:.o=.symtypes): modkern_aflags := $(KBUILD_AFLAGS_MODULE) $(AFLAGS_MODULE)

... would fix the issue, but it is not nice to duplicate similar code
for every suffix.

Implement modkern_aflags in the same way as modkern_cflags.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Make it even shorter.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

$(objtree)/Module.symvers is not required for descending into
sub-directories. It is needed for the modpost stage.

Move the Module.symvers check to the right place.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Currently, Kbuild treats an object as multi-used when any of
$(foo-objs), $(foo-y), $(foo-m) is set. It makes more sense to
check $(foo-) as well.

In the context of foo-$(CONFIG_FOO_FEATURE1), CONFIG_FOO_FEATURE1
could be unset.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

This builds module objects, so [M] makes sense.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Update the build scripts and the version magic to reflect when
CONFIG_PREEMPT_RT is enabled in the same way as CONFIG_PREEMPT is treated.

The resulting version strings:

  Linux m 5.3.0-rc1+ #100 SMP Fri Jul 26 ...
  Linux m 5.3.0-rc1+ #101 SMP PREEMPT Fri Jul 26 ...
  Linux m 5.3.0-rc1+ #102 SMP PREEMPT_RT Fri Jul 26 ...

The module vermagic:

  5.3.0-rc1+ SMP mod_unload modversions
  5.3.0-rc1+ SMP preempt mod_unload modversions
  5.3.0-rc1+ SMP preempt_rt mod_unload modversions
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Flex and bison are used for kconfig, dtc, genksyms, all of which are
host programs. I never imagine the kernel embeds a parser or a lexer.

Move the flex and bison rules to scripts/Makefile.host. This file is
included only when hostprogs-y etc. is present in the Makefile in the
directory. So, parsing these rules are skipped in most of directories.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

We generally expect bison to create not only a C file, but also a
header, which will be included from the lexer.

Currently, Kbuild generates them in separate rules. So, for instance,
when building Kconfig, you will notice bison is invoked twice:

  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.h
  HOSTCC  scripts/kconfig/lexer.lex.o
  YACC    scripts/kconfig/parser.tab.c
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTLD  scripts/kconfig/conf

Make handles such cases nicely in pattern rules [1]. Merge the two
rules so that one invokcation of bison can generate both of them.

  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTLD  scripts/kconfig/conf

[1] Pattern rule

GNU Make manual says:
"Pattern rules may have more than one target. Unlike normal rules,
this does not act as many different rules with the same prerequisites
and recipe. If a pattern rule has multiple targets, make knows that
the rule's recipe is responsible for making all of the targets. The
recipe is executed only once to make all the targets. When searching
for a pattern rule to match a target, the target patterns of a rule
other than the one that matches the target in need of a rule are
incidental: make worries only about giving a recipe and prerequisites
to the file presently in question. However, when this file's recipe is
run, the other targets are marked as having been updated themselves."

https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.htmlSigned-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

$(basename ...) trims the last suffix. Using it is more intuitive in
my opinion.

This pattern rule makes %.asn1.c and %.asn1.h at the same time.
Previously, the short log showed only either of them, depending on
the target file in question.

To clarify that two files are being generated by the single recipe,
I changed the log as follows:

Before:

  ASN.1   crypto/asymmetric_keys/x509.asn1.c

After:

  ASN.1   crypto/asymmetric_keys/x509.asn1.[ch]
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>