1. 25 Nov, 2019 14 commits
  2. 23 Nov, 2019 7 commits
  3. 14 Nov, 2019 2 commits
    • Masahiro Yamada's avatar
      kbuild: move headers_check rule to usr/include/Makefile · 7ecaf069
      Masahiro Yamada authored
      Currently, some sanity checks for uapi headers are done by
      scripts/headers_check.pl, which is wired up to the 'headers_check'
      target in the top Makefile.
      
      It is true compiling headers has better test coverage, but there
      are still several headers excluded from the compile test. I like
      to keep headers_check.pl for a while, but we can delete a lot of
      code by moving the build rule to usr/include/Makefile.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      7ecaf069
    • Masahiro Yamada's avatar
      kbuild: remove header compile test · fcbb8461
      Masahiro Yamada authored
      There are both positive and negative options about this feature.
      At first, I thought it was a good idea, but actually Linus stated a
      negative opinion (https://lkml.org/lkml/2019/9/29/227). I admit it
      is ugly and annoying.
      
      The baseline I'd like to keep is the compile-test of uapi headers.
      (Otherwise, kernel developers have no way to ensure the correctness
      of the exported headers.)
      
      I will maintain a small build rule in usr/include/Makefile.
      Remove the other header test functionality.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      fcbb8461
  4. 11 Nov, 2019 17 commits
    • Masahiro Yamada's avatar
      kbuild: rename any-prereq to newer-prereqs · eba19032
      Masahiro Yamada authored
      GNU Make manual says:
      
        $?
            The names of all the prerequisites that are newer than the target,
            with spaces between them.
      
      To reflect this, rename any-prereq to newer-prereqs, which is clearer
      and more intuitive.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      eba19032
    • Masahiro Yamada's avatar
      kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild · 2d3b1b8f
      Masahiro Yamada authored
      The incremental build of Linux kernel is pretty slow when lots of
      objects are compiled. The rebuild of allmodconfig may take a few
      minutes even when none of the objects needs to be rebuilt.
      
      The time-consuming part in the incremental build is the evaluation of
      if_changed* macros since they are used in the recipes to compile C and
      assembly source files into objects.
      
      I notice the following code in if_changed* is expensive:
      
        $(filter-out $(PHONY) $(wildcard $^),$^)
      
      In the incremental build, every object has its .*.cmd file, which
      contains the auto-generated list of included headers. So, $^ are
      expanded into the long list of the source file + included headers,
      and $(wildcard $^) checks whether they exist.
      
      It may not be clear why this check exists there.
      
      Here is the record of my research.
      
      [1] The first code addition into Kbuild
      
      This code dates back to 2002. It is the pre-git era. So, I copy-pasted
      it from the historical git tree.
      
      | commit 4a6db079 (HEAD)
      | Author: Kai Germaschewski <kai@tp1.ruhr-uni-bochum.de>
      | Date:   Mon Jun 17 00:22:37 2002 -0500
      |
      |     kbuild: Handle removed headers
      |
      |     New and old way to handle dependencies would choke when a file
      |     #include'd by other files was removed, since the dependency on it was
      |     still recorded, but since it was gone, make has no idea what to do about
      |     it (and would complain with "No rule to make <file> ...")
      |
      |     We now add targets for all the previously included files, so make will
      |     just ignore them if they disappear.
      |
      | diff --git a/Rules.make b/Rules.make
      | index 6ef827d3df39..7db5301ea7db 100644
      | --- a/Rules.make
      | +++ b/Rules.make
      | @@ -446,7 +446,7 @@ if_changed = $(if $(strip $? \
      |  # execute the command and also postprocess generated .d dependencies
      |  # file
      |
      | -if_changed_dep = $(if $(strip $? \
      | +if_changed_dep = $(if $(strip $? $(filter-out FORCE $(wildcard $^),$^)\
      |                           $(filter-out $(cmd_$(1)),$(cmd_$@))\
      |                           $(filter-out $(cmd_$@),$(cmd_$(1)))),\
      |         @set -e; \
      | diff --git a/scripts/fixdep.c b/scripts/fixdep.c
      | index b5d7bee8efc7..db45bd1888c0 100644
      | --- a/scripts/fixdep.c
      | +++ b/scripts/fixdep.c
      | @@ -292,7 +292,7 @@ void parse_dep_file(void *map, size_t len)
      |                 exit(1);
      |         }
      |         memcpy(s, m, p-m); s[p-m] = 0;
      | -       printf("%s: \\\n", target);
      | +       printf("deps_%s := \\\n", target);
      |         m = p+1;
      |
      |         clear_config();
      | @@ -314,7 +314,8 @@ void parse_dep_file(void *map, size_t len)
      |                 }
      |                 m = p + 1;
      |         }
      | -       printf("\n");
      | +       printf("\n%s: $(deps_%s)\n\n", target, target);
      | +       printf("$(deps_%s):\n", target);
      |  }
      |
      |  void print_deps(void)
      
      The "No rule to make <file> ..." error can be solved by passing -MP to
      the compiler, but I think the detection of header removal is a good
      feature. When a header is removed, all source files that previously
      included it should be re-compiled. This makes sure we has correctly
      got rid of #include directives of it.
      
      This is also related with the behavior of $?. The GNU Make manual says:
      
        $?
            The names of all the prerequisites that are newer than the target,
            with spaces between them.
      
      This does not explain whether a non-existent prerequisite is considered
      to be newer than the target.
      
      At this point of time, GNU Make 3.7x was used, where the $? did not
      include non-existent prerequisites. Therefore,
      
        $(filter-out FORCE $(wildcard $^),$^)
      
      was useful to detect the header removal, and to rebuild the related
      objects if it is the case.
      
      [2] Change of $? behavior
      
      Later, the behavior of $? was changed (fixed) to include prerequisites
      that did not exist.
      
      First, GNU Make commit 64e16d6c00a5 ("Various changes getting ready for
      the release of 3.81.") changed it, but in the release test of 3.81, it
      turned out to break the kernel build.
      
      See these:
      
       - http://lists.gnu.org/archive/html/bug-make/2006-03/msg00003.html
       - https://savannah.gnu.org/bugs/?16002
       - https://savannah.gnu.org/bugs/?16051
      
      Then, GNU Make commit 6d8d9b74d9c5 ("Numerous updates to tests for
      issues found on Cygwin and Windows.") reverted it for the 3.81 release
      to give Linux kernel time to adjust to the new behavior.
      
      After the 3.81 release, GNU Make commit 7595f38f62af ("Fixed a number
      of documentation bugs, plus some build/install issues:") re-added it.
      
      [3] Adjustment to the new $? behavior on Kbuild side
      
      Meanwhile, the kernel build was changed by commit 4f193362 ("kbuild:
      change kbuild to not rely on incorrect GNU make behavior") to adjust to
      the new $? behavior.
      
      [4] GNU Make 3.82 released in 2010
      
      GNU Make 3.82 was the first release that integrated the correct $?
      behavior. At this point, Kbuild dealt with GNU Make versions with
      different $? behaviors.
      
       3.81 or older:
          $? does not contain any non-existent prerequisite.
          $(filter-out $(PHONY) $(wildcard $^),$^) was useful to detect
          removed include headers.
      
       3.82 or newer:
          $? contains non-existent prerequisites. When a header is removed,
          it appears in $?. $(filter-out $(PHONY) $(wildcard $^),$^) became
          a redundant check.
      
      With the correct $? behavior, we could have dropped the expensive
      check for 3.82 or later, but we did not. (Maybe nobody noticed this
      optimization.)
      
      [5] The .SECONDARY special target trips up $?
      
      Some time later, I noticed $? did not work as expected under some
      circumstances. As above, $? should contain non-existent prerequisites,
      but the ones specified as SECONDARY do not appear in $?.
      
      I asked this in GNU Make ML, and it seems a bug:
      
        https://lists.gnu.org/archive/html/bug-make/2019-01/msg00001.html
      
      Since commit 8e9b61b2 ("kbuild: move .SECONDARY special target to
      Kbuild.include"), all files, including headers listed in .*.cmd files,
      are treated as secondary.
      
      So, we are back into the incorrect $? behavior.
      
      If we Kbuild want to react to the header removal, we need to keep
      $(filter-out $(PHONY) $(wildcard $^),$^) but this makes the rebuild
      so slow.
      
      [Summary]
      
       - I believe noticing the header removal and recompiling related objects
         is a nice feature for the build system.
      
       - If $? worked correctly, $(filter-out $(PHONY),$?) would be enough
         to detect the header removal.
      
       - Currently, $? does not work correctly when used with .SECONDARY,
         and Kbuild is hit by this bug.
      
       - I filed a bug report for this, but not fixed yet as of writing.
      
       - Currently, the header removal is detected by the following expensive
         code:
      
          $(filter-out $(PHONY) $(wildcard $^),$^)
      
       - I do not want to revert commit 8e9b61b2 ("kbuild: move
         .SECONDARY special target to Kbuild.include"). Specifying
         .SECONDARY globally is clean, and it matches to the Kbuild policy.
      
      This commit proactively removes the expensive check since it makes the
      incremental build faster. A downside is Kbuild will no longer be able
      to notice the header removal.
      
      You can confirm it by the full-build followed by a header removal, and
      then re-build.
      
        $ make defconfig all
          [ full build ]
        $ rm include/linux/device.h
        $ make
          CALL    scripts/checksyscalls.sh
          CALL    scripts/atomic/check-atomics.sh
          DESCEND  objtool
          CHK     include/generated/compile.h
        Kernel: arch/x86/boot/bzImage is ready  (#11)
          Building modules, stage 2.
          MODPOST 12 modules
      
      Previously, Kbuild noticed a missing header and emits a build error.
      Now, Kbuild is fine with it. This is an unusual corner-case, not a big
      deal. Once the $? bug is fixed in GNU Make, everything will work fine.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      2d3b1b8f
    • Masahiro Yamada's avatar
      kbuild: update compile-test header list for v5.5-rc1 · d2a99dbd
      Masahiro Yamada authored
      Since commit 707816c8 ("netfilter: remove deprecation warnings from
      uapi headers."), you can compile linux/netfilter_ipv4/ipt_LOG.h and
      linux/netfilter_ipv6/ip6t_LOG.h without warnings.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      d2a99dbd
    • Masahiro Yamada's avatar
      modpost: remove unneeded local variable in contains_namespace() · 76b54cf0
      Masahiro Yamada authored
      The local variable, ns_entry, is unneeded.
      
      While I was here, I also cleaned up the comparison with NULL or 0.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Reviewed-by: default avatarMatthias Maennich <maennich@google.com>
      76b54cf0
    • Masahiro Yamada's avatar
      scripts/nsdeps: support nsdeps for external module builds · bc35d4bd
      Masahiro Yamada authored
      scripts/nsdeps is written to take care of only in-tree modules.
      Perhaps, this is not a bug, but just a design. At least,
      Documentation/core-api/symbol-namespaces.rst focuses on in-tree modules.
      
      Having said that, some people already tried nsdeps for external modules.
      So, it would be nice to support it.
      Reported-by: default avatarSteve French <smfrench@gmail.com>
      Reported-by: default avatarJessica Yu <jeyu@kernel.org>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Tested-by: default avatarJessica Yu <jeyu@kernel.org>
      Acked-by: default avatarJessica Yu <jeyu@kernel.org>
      Reviewed-by: default avatarMatthias Maennich <maennich@google.com>
      Tested-by: default avatarMatthias Maennich <maennich@google.com>
      bc35d4bd
    • Masahiro Yamada's avatar
      modpost: dump missing namespaces into a single modules.nsdeps file · bbc55bde
      Masahiro Yamada authored
      The modpost, with the -d option given, generates per-module .ns_deps
      files.
      
      Kbuild generates per-module .mod files to carry module information.
      This is convenient because Make handles multiple jobs in parallel
      when the -j option is given.
      
      On the other hand, the modpost always runs as a single thread.
      I do not see a strong reason to produce separate .ns_deps files.
      
      This commit changes the modpost to generate just one file,
      modules.nsdeps, each line of which has the following format:
      
        <module_name>: <list of missing namespaces>
      
      Please note it contains *missing* namespaces instead of required ones.
      So, modules.nsdeps is empty if the namespace dependency is all good.
      
      This will work more efficiently because spatch will no longer process
      already imported namespaces. I removed the '(if needed)' from the
      nsdeps log since spatch is invoked only when needed.
      
      This also solves the stale .ns_deps problem reported by Jessica Yu:
      
        https://lkml.org/lkml/2019/10/28/467Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Tested-by: default avatarJessica Yu <jeyu@kernel.org>
      Acked-by: default avatarJessica Yu <jeyu@kernel.org>
      Reviewed-by: default avatarMatthias Maennich <maennich@google.com>
      Tested-by: default avatarMatthias Maennich <maennich@google.com>
      bbc55bde
    • Masahiro Yamada's avatar
      modpost: free ns_deps_buf.p after writing ns_deps files · 0241ea8c
      Masahiro Yamada authored
      buf_write() allocates memory. Free it.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      0241ea8c
    • Masahiro Yamada's avatar
      modpost: do not invoke extra modpost for nsdeps · bff9c62b
      Masahiro Yamada authored
      'make nsdeps' invokes the modpost three times at most; before linking
      vmlinux, before building modules, and finally for generating .ns_deps
      files. Running the modpost again and again is not efficient.
      
      The last two can be unified. When the -d option is given, the modpost
      still does the usual job, and in addition, generates .ns_deps files.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Tested-by: default avatarMatthias Maennich <maennich@google.com>
      Reviewed-by: default avatarMatthias Maennich <maennich@google.com>
      bff9c62b
    • Bhaskar Chowdhury's avatar
    • Alyssa Ross's avatar
      kconfig: be more helpful if pkg-config is missing · 46b2afa6
      Alyssa Ross authored
      If ncurses is installed, but at a non-default location, the previous
      error message was not helpful in resolving the situation.  Now it will
      suggest that pkg-config might need to be installed in addition to
      ncurses.
      Signed-off-by: default avatarAlyssa Ross <hi@alyssa.is>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      46b2afa6
    • Laura Abbott's avatar
      kconfig: Add option to get the full help text with listnewconfig · 5d8b42aa
      Laura Abbott authored
      make listnewconfig will list the individual options that need to be set.
      This is useful but there's no easy way to get the help text associated
      with the options at the same time. Introduce a new targe
      'make helpnewconfig' which lists the full help text of all the
      new options as well. This makes it easier to automatically generate
      changes that are easy for humans to review. This command also adds
      markers between each option for easier parsing.
      Signed-off-by: default avatarLaura Abbott <labbott@redhat.com>
      Acked-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      5d8b42aa
    • Matteo Croce's avatar
      kbuild: Add make dir-pkg build option · af7db99a
      Matteo Croce authored
      Add a 'dir-pkg' target which just creates the same directory structures
      as in tar-pkg, but doesn't package anything.
      Useful when the user wants to copy the kernel tree on a machine using
      ssh, rsync or whatever.
      Signed-off-by: default avatarMatteo Croce <mcroce@redhat.com>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      af7db99a
    • Geert Uytterhoeven's avatar
      kbuild: Extend defconfig field size from 24 to 27 · 4234448b
      Geert Uytterhoeven authored
      There are 6 defconfigs with names longer than 24 characters, breaking
      alignment in "make help".
      
      The "winner" is "ecovec24-romimage_defconfig", counting in at 27
      characters.
      
      Extend the defconfig field size to 27 to restore alignment.
      Don't use a larger value, to not encourage people to create even longer
      defconfig names.
      Signed-off-by: default avatarGeert Uytterhoeven <geert+renesas@glider.be>
      Acked-by: default avatarHans-Christian Egtvedt <egtvedt@samfundet.no>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      4234448b
    • Geert Uytterhoeven's avatar
      kbuild: Wrap long "make help" text lines · a64c0440
      Geert Uytterhoeven authored
      Some "make help" text lines extend beyond 80 characters.
      Wrap them before an opening parenthesis, or before 80 characters.
      Signed-off-by: default avatarGeert Uytterhoeven <geert+renesas@glider.be>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      a64c0440
    • Bhaskar Chowdhury's avatar
      scripts: setlocalversion: replace backquote to dollar parenthesis · 3c96bdd0
      Bhaskar Chowdhury authored
      This patch replaces backquote to dollar parenthesis syntax for better
      readability.
      Signed-off-by: default avatarBhaskar Chowdhury <unixbhaskar@gmail.com>
      Acked-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Acked-by: default avatarNico Schottelius <nico-linuxsetlocalversion@schottelius.org>
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      3c96bdd0
    • Masahiro Yamada's avatar
      asm-generic/export.h: remove unneeded __kcrctab_* symbols · 03034dbd
      Masahiro Yamada authored
      EXPORT_SYMBOL from assembly code produces an unused symbol __kcrctab_*.
      
      kcrctab is used as a section name (prefixed with three underscores),
      but never used as a symbol.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      03034dbd
    • Masahiro Yamada's avatar
      asm-generic/export.h: make __ksymtab_* local symbols · a31ec048
      Masahiro Yamada authored
      For EXPORT_SYMBOL from C files, <linux/export.h> defines __ksymtab_*
      as local symbols.
      
      For EXPORT_SYMBOL from assembly, in contrast, <asm-generic/export.h>
      produces globally-visible __ksymtab_* symbols due to this .globl
      directive.
      
      I do not know why this must be global. It still works without this.
      Signed-off-by: default avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      a31ec048