1. 04 Nov, 2017 2 commits
  2. 03 Nov, 2017 25 commits
  3. 02 Nov, 2017 13 commits
    • Madhavan Srinivasan's avatar
      powerpc/perf: Fix core-imc hotplug callback failure during imc initialization · 7ecb37f6
      Madhavan Srinivasan authored
      Call trace observed during boot:
      
        nest_capp0_imc performance monitor hardware support registered
        nest_capp1_imc performance monitor hardware support registered
        core_imc memory allocation for cpu 56 failed
        Unable to handle kernel paging request for data at address 0xffa400010
        Faulting instruction address: 0xc000000000bf3294
        0:mon> e
        cpu 0x0: Vector: 300 (Data Access) at [c000000ff38ff8d0]
            pc: c000000000bf3294: mutex_lock+0x34/0x90
            lr: c000000000bf3288: mutex_lock+0x28/0x90
            sp: c000000ff38ffb50
           msr: 9000000002009033
           dar: ffa400010
         dsisr: 80000
          current = 0xc000000ff383de00
          paca    = 0xc000000007ae0000	 softe: 0	 irq_happened: 0x01
            pid   = 13, comm = cpuhp/0
        Linux version 4.11.0-39.el7a.ppc64le (mockbuild@ppc-058.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Tue Oct 3 07:42:44 EDT 2017
        0:mon> t
        [c000000ff38ffb80] c0000000002ddfac perf_pmu_migrate_context+0xac/0x470
        [c000000ff38ffc40] c00000000011385c ppc_core_imc_cpu_offline+0x1ac/0x1e0
        [c000000ff38ffc90] c000000000125758 cpuhp_invoke_callback+0x198/0x5d0
        [c000000ff38ffd00] c00000000012782c cpuhp_thread_fun+0x8c/0x3d0
        [c000000ff38ffd60] c0000000001678d0 smpboot_thread_fn+0x290/0x2a0
        [c000000ff38ffdc0] c00000000015ee78 kthread+0x168/0x1b0
        [c000000ff38ffe30] c00000000000b368 ret_from_kernel_thread+0x5c/0x74
      
      While registering the cpuhoplug callbacks for core-imc, if we fails
      in the cpuhotplug online path for any random core (either because opal call to
      initialize the core-imc counters fails or because memory allocation fails for
      that core), ppc_core_imc_cpu_offline() will get invoked for other cpus who
      successfully returned from cpuhotplug online path.
      
      But in the ppc_core_imc_cpu_offline() path we are trying to migrate the event
      context, when core-imc counters are not even initialized. Thus creating the
      above stack dump.
      
      Add a check to see if core-imc counters are enabled or not in the cpuhotplug
      offline path before migrating the context to handle this failing scenario.
      
      Fixes: 885dcd70 ("powerpc/perf: Add nest IMC PMU support")
      Signed-off-by: default avatarMadhavan Srinivasan <maddy@linux.vnet.ibm.com>
      Signed-off-by: default avatarAnju T Sudhakar <anju@linux.vnet.ibm.com>
      Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
      7ecb37f6
    • Linus Torvalds's avatar
      Kbuild: don't pass "-C" to preprocessor when processing linker scripts · 5cb0512c
      Linus Torvalds authored
      For some odd historical reason, we preprocessed the linker scripts with
      "-C", which keeps comments around.  That makes no sense, since the
      comments are not meaningful for the build anyway.
      
      And it actually breaks things, since linker scripts can't have C++ style
      "//" comments in them, so keeping comments after preprocessing now
      limits us in odd and surprising ways in our header files for no good
      reason.
      
      The -C option goes back to pre-git and pre-bitkeeper times, but seems to
      have been historically used (along with "-traditional") for some
      odd-ball architectures (ia64, MIPS and SH).  It probably didn't matter
      back then either, but might possibly have been used to minimize the
      difference between the original file and the pre-processed result.
      
      The reason for this may be lost in time, but let's not perpetuate it
      only because we can't remember why we did this crazy thing.
      
      This was triggered by the recent addition of SPDX lines to the source
      tree, where people apparently were confused about why header files
      couldn't use the C++ comment format.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Greg KH <gregkh@linuxfoundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      5cb0512c
    • Linus Torvalds's avatar
      Revert "x86: do not use cpufreq_quick_get() for /proc/cpuinfo "cpu MHz"" · 890da9cf
      Linus Torvalds authored
      This reverts commit 51204e06.
      
      There wasn't really any good reason for it, and people are complaining
      (rightly) that it broke existing practice.
      
      Cc: Len Brown <len.brown@intel.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      890da9cf
    • Linus Torvalds's avatar
      Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux · 6daa0839
      Linus Torvalds authored
      Pull arm64 fix from Catalin Marinas:
       "Check addr_limit in arm64 __dump_instr()"
      
      * tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
        arm64: ensure __dump_instr() checks addr_limit
      6daa0839
    • Mark Rutland's avatar
      arm64: ensure __dump_instr() checks addr_limit · 7a7003b1
      Mark Rutland authored
      It's possible for a user to deliberately trigger __dump_instr with a
      chosen kernel address.
      
      Let's avoid problems resulting from this by using get_user() rather than
      __get_user(), ensuring that we don't erroneously access kernel memory.
      
      Where we use __dump_instr() on kernel text, we already switch to
      KERNEL_DS, so this shouldn't adversely affect those cases.
      
      Fixes: 60ffc30d ("arm64: Exception handling")
      Cc: stable@vger.kernel.org
      Acked-by: default avatarWill Deacon <will.deacon@arm.com>
      Signed-off-by: default avatarMark Rutland <mark.rutland@arm.com>
      Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
      7a7003b1
    • Linus Torvalds's avatar
      Merge tag 'spdx_identifiers-4.14-rc8' of... · ead75150
      Linus Torvalds authored
      Merge tag 'spdx_identifiers-4.14-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core
      
      Pull initial SPDX identifiers from Greg KH:
       "License cleanup: add SPDX license identifiers to some files
      
        Many source files in the tree are missing licensing information, which
        makes it harder for compliance tools to determine the correct license.
      
        By default all files without license information are under the default
        license of the kernel, which is GPL version 2.
      
        Update the files which contain no license information with the
        'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally
        binding shorthand, which can be used instead of the full boiler plate
        text.
      
        This patch is based on work done by Thomas Gleixner and Kate Stewart
        and Philippe Ombredanne.
      
        How this work was done:
      
        Patches were generated and checked against linux-4.14-rc6 for a subset
        of the use cases:
      
         - file had no licensing information it it.
      
         - file was a */uapi/* one with no licensing information in it,
      
         - file was a */uapi/* one with existing licensing information,
      
        Further patches will be generated in subsequent months to fix up cases
        where non-standard license headers were used, and references to
        license had to be inferred by heuristics based on keywords.
      
        The analysis to determine which SPDX License Identifier to be applied
        to a file was done in a spreadsheet of side by side results from of
        the output of two independent scanners (ScanCode & Windriver)
        producing SPDX tag:value files created by Philippe Ombredanne.
        Philippe prepared the base worksheet, and did an initial spot review
        of a few 1000 files.
      
        The 4.13 kernel was the starting point of the analysis with 60,537
        files assessed. Kate Stewart did a file by file comparison of the
        scanner results in the spreadsheet to determine which SPDX license
        identifier(s) to be applied to the file. She confirmed any
        determination that was not immediately clear with lawyers working with
        the Linux Foundation.
      
        Criteria used to select files for SPDX license identifier tagging was:
      
         - Files considered eligible had to be source code files.
      
         - Make and config files were included as candidates if they contained
           >5 lines of source
      
         - File already had some variant of a license header in it (even if <5
           lines).
      
        All documentation files were explicitly excluded.
      
        The following heuristics were used to determine which SPDX license
        identifiers to apply.
      
         - when both scanners couldn't find any license traces, file was
           considered to have no license information in it, and the top level
           COPYING file license applied.
      
           For non */uapi/* files that summary was:
      
             SPDX license identifier                            # files
             ---------------------------------------------------|-------
             GPL-2.0                                              11139
      
           and resulted in the first patch in this series.
      
           If that file was a */uapi/* path one, it was "GPL-2.0 WITH
           Linux-syscall-note" otherwise it was "GPL-2.0". Results of that
           was:
      
             SPDX license identifier                            # files
             ---------------------------------------------------|-------
             GPL-2.0 WITH Linux-syscall-note                        930
      
           and resulted in the second patch in this series.
      
         - if a file had some form of licensing information in it, and was one
           of the */uapi/* ones, it was denoted with the Linux-syscall-note if
           any GPL family license was found in the file or had no licensing in
           it (per prior point). Results summary:
      
             SPDX license identifier                            # files
             ---------------------------------------------------|------
             GPL-2.0 WITH Linux-syscall-note                       270
             GPL-2.0+ WITH Linux-syscall-note                      169
             ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
             ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
             LGPL-2.1+ WITH Linux-syscall-note                      15
             GPL-1.0+ WITH Linux-syscall-note                       14
             ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
             LGPL-2.0+ WITH Linux-syscall-note                       4
             LGPL-2.1 WITH Linux-syscall-note                        3
             ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
             ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1
      
           and that resulted in the third patch in this series.
      
         - when the two scanners agreed on the detected license(s), that
           became the concluded license(s).
      
         - when there was disagreement between the two scanners (one detected
           a license but the other didn't, or they both detected different
           licenses) a manual inspection of the file occurred.
      
         - In most cases a manual inspection of the information in the file
           resulted in a clear resolution of the license that should apply
           (and which scanner probably needed to revisit its heuristics).
      
         - When it was not immediately clear, the license identifier was
           confirmed with lawyers working with the Linux Foundation.
      
         - If there was any question as to the appropriate license identifier,
           the file was flagged for further research and to be revisited later
           in time.
      
        In total, over 70 hours of logged manual review was done on the
        spreadsheet to determine the SPDX license identifiers to apply to the
        source files by Kate, Philippe, Thomas and, in some cases,
        confirmation by lawyers working with the Linux Foundation.
      
        Kate also obtained a third independent scan of the 4.13 code base from
        FOSSology, and compared selected files where the other two scanners
        disagreed against that SPDX file, to see if there was new insights.
        The Windriver scanner is based on an older version of FOSSology in
        part, so they are related.
      
        Thomas did random spot checks in about 500 files from the spreadsheets
        for the uapi headers and agreed with SPDX license identifier in the
        files he inspected. For the non-uapi files Thomas did random spot
        checks in about 15000 files.
      
        In initial set of patches against 4.14-rc6, 3 files were found to have
        copy/paste license identifier errors, and have been fixed to reflect
        the correct identifier.
      
        Additionally Philippe spent 10 hours this week doing a detailed manual
        inspection and review of the 12,461 patched files from the initial
        patch version early this week with:
      
         - a full scancode scan run, collecting the matched texts, detected
           license ids and scores
      
         - reviewing anything where there was a license detected (about 500+
           files) to ensure that the applied SPDX license was correct
      
         - reviewing anything where there was no detection but the patch
           license was not GPL-2.0 WITH Linux-syscall-note to ensure that the
           applied SPDX license was correct
      
        This produced a worksheet with 20 files needing minor correction. This
        worksheet was then exported into 3 different .csv files for the
        different types of files to be modified.
      
        These .csv files were then reviewed by Greg. Thomas wrote a script to
        parse the csv files and add the proper SPDX tag to the file, in the
        format that the file expected. This script was further refined by Greg
        based on the output to detect more types of files automatically and to
        distinguish between header and source .c files (which need different
        comment types.) Finally Greg ran the script using the .csv files to
        generate the patches.
      Reviewed-by: default avatarKate Stewart <kstewart@linuxfoundation.org>
      Reviewed-by: default avatarPhilippe Ombredanne <pombredanne@nexb.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
        Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>"
      
      * tag 'spdx_identifiers-4.14-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core:
        License cleanup: add SPDX license identifier to uapi header files with a license
        License cleanup: add SPDX license identifier to uapi header files with no license
        License cleanup: add SPDX GPL-2.0 license identifier to files with no license
      ead75150
    • Linus Torvalds's avatar
      Merge tag 'linux-kselftest-4.14-rc7' of... · fdebad11
      Linus Torvalds authored
      Merge tag 'linux-kselftest-4.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
      
      Pull kselftest fix from Shuah Khan:
       "This consists of a single fix to a regression to printing individual
        test results to the console. An earlier commit changed it to printing
        just the summary of results, which will negatively impact users that
        rely on console log to look at the individual test failures.
      
        This fix makes it optional to print summary and by default results get
        printed to the console"
      
      * tag 'linux-kselftest-4.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
        selftests: lib.mk: print individual test results to console by default
      fdebad11
    • Linus Torvalds's avatar
      Merge tag 'sound-4.14-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound · 0f07e10f
      Linus Torvalds authored
      Pull sound fixes from Takashi Iwai:
       "Unfortunately we still have received a significant amount of changes
        at the late stage, but at least all are small and clear fixes.
      
        There are two fixes for ALSA core stuff, yet another timer race fix
        and sequencer lockdep annotation fix. Both are spotted by syzkaller,
        and not too serious but better to paper over quickly.
      
        All other commits are about ASoC drivers, most notably, a revert of
        RT5514 hotword control that was included in 4.14-rc (due to a kind of
        abuse of kctl TLV ABI), together with topology API fixes and other
        device-specific small fixes that should go for stable, too"
      
      * tag 'sound-4.14-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound:
        ALSA: seq: Fix nested rwsem annotation for lockdep splat
        ALSA: timer: Add missing mutex lock for compat ioctls
        ASoC: rt5616: fix 0x91 default value
        ASoC: rt5659: connect LOUT Amp with Charge Pump
        ASoC: rt5659: register power bit of LOUT Amp
        ASoC: rt5663: Change the dev getting function in rt5663_irq
        ASoC: rt5514: Revert Hotword Model control
        ASoC: topology: Fix a potential memory leak in 'soc_tplg_dapm_widget_denum_create()'
        ASoC: topology: Fix a potential NULL pointer dereference in 'soc_tplg_dapm_widget_denum_create()'
        ASoC: rt5514-spi: check irq status to schedule data copy
        ASoC: adau17x1: Workaround for noise bug in ADC
      0f07e10f
    • Linus Torvalds's avatar
      Merge branch 'fixes-v4.14-rc7' of... · 6965f1aa
      Linus Torvalds authored
      Merge branch 'fixes-v4.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
      
      Pull key handling fixes from James Morris:
       "Fixes for the Keys subsystem by Eric Biggers"
      
      * 'fixes-v4.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security:
        KEYS: fix out-of-bounds read during ASN.1 parsing
        KEYS: trusted: fix writing past end of buffer in trusted_read()
        KEYS: return full count in keyring_read() if buffer is too small
      6965f1aa
    • Jiri Slaby's avatar
      futex: futex_wake_op, do not fail on invalid op · e78c38f6
      Jiri Slaby authored
      In commit 30d6e0a4 ("futex: Remove duplicated code and fix undefined
      behaviour"), I let FUTEX_WAKE_OP to fail on invalid op.  Namely when op
      should be considered as shift and the shift is out of range (< 0 or > 31).
      
      But strace's test suite does this madness:
      
        futex(0x7fabd78bcffc, 0x5, 0xfacefeed, 0xb, 0x7fabd78bcffc, 0xa0caffee);
        futex(0x7fabd78bcffc, 0x5, 0xfacefeed, 0xb, 0x7fabd78bcffc, 0xbadfaced);
        futex(0x7fabd78bcffc, 0x5, 0xfacefeed, 0xb, 0x7fabd78bcffc, 0xffffffff);
      
      When I pick the first 0xa0caffee, it decodes as:
      
        0x80000000 & 0xa0caffee: oparg is shift
        0x70000000 & 0xa0caffee: op is FUTEX_OP_OR
        0x0f000000 & 0xa0caffee: cmp is FUTEX_OP_CMP_EQ
        0x00fff000 & 0xa0caffee: oparg is sign-extended 0xcaf = -849
        0x00000fff & 0xa0caffee: cmparg is sign-extended 0xfee = -18
      
      That means the op tries to do this:
      
        (futex |= (1 << (-849))) == -18
      
      which is completely bogus. The new check of op in the code is:
      
              if (encoded_op & (FUTEX_OP_OPARG_SHIFT << 28)) {
                      if (oparg < 0 || oparg > 31)
                              return -EINVAL;
                      oparg = 1 << oparg;
              }
      
      which results obviously in the "Invalid argument" errno:
      
        FAIL: futex
        ===========
      
        futex(0x7fabd78bcffc, 0x5, 0xfacefeed, 0xb, 0x7fabd78bcffc, 0xa0caffee) = -1: Invalid argument
        futex.test: failed test: ../futex failed with code 1
      
      So let us soften the failure to print only a (ratelimited) message, crop
      the value and continue as if it were right.  When userspace keeps up, we
      can switch this to return -EINVAL again.
      
      [v2] Do not return 0 immediatelly, proceed with the cropped value.
      
      Fixes: 30d6e0a4 ("futex: Remove duplicated code and fix undefined behaviour")
      Signed-off-by: default avatarJiri Slaby <jslaby@suse.cz>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Darren Hart <dvhart@infradead.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      e78c38f6
    • Greg Kroah-Hartman's avatar
      License cleanup: add SPDX license identifier to uapi header files with a license · e2be04c7
      Greg Kroah-Hartman authored
      Many user space API headers have licensing information, which is either
      incomplete, badly formatted or just a shorthand for referring to the
      license under which the file is supposed to be.  This makes it hard for
      compliance tools to determine the correct license.
      
      Update these files with an SPDX license identifier.  The identifier was
      chosen based on the license information in the file.
      
      GPL/LGPL licensed headers get the matching GPL/LGPL SPDX license
      identifier with the added 'WITH Linux-syscall-note' exception, which is
      the officially assigned exception identifier for the kernel syscall
      exception:
      
         NOTE! This copyright does *not* cover user programs that use kernel
         services by normal system calls - this is merely considered normal use
         of the kernel, and does *not* fall under the heading of "derived work".
      
      This exception makes it possible to include GPL headers into non GPL
      code, without confusing license compliance tools.
      
      Headers which have either explicit dual licensing or are just licensed
      under a non GPL license are updated with the corresponding SPDX
      identifier and the GPLv2 with syscall exception identifier.  The format
      is:
              ((GPL-2.0 WITH Linux-syscall-note) OR SPDX-ID-OF-OTHER-LICENSE)
      
      SPDX license identifiers are a legally binding shorthand, which can be
      used instead of the full boiler plate text.  The update does not remove
      existing license information as this has to be done on a case by case
      basis and the copyright holders might have to be consulted. This will
      happen in a separate step.
      
      This patch is based on work done by Thomas Gleixner and Kate Stewart and
      Philippe Ombredanne.  See the previous patch in this series for the
      methodology of how this patch was researched.
      Reviewed-by: default avatarKate Stewart <kstewart@linuxfoundation.org>
      Reviewed-by: default avatarPhilippe Ombredanne <pombredanne@nexb.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      e2be04c7
    • Greg Kroah-Hartman's avatar
      License cleanup: add SPDX license identifier to uapi header files with no license · 6f52b16c
      Greg Kroah-Hartman authored
      Many user space API headers are missing licensing information, which
      makes it hard for compliance tools to determine the correct license.
      
      By default are files without license information under the default
      license of the kernel, which is GPLV2.  Marking them GPLV2 would exclude
      them from being included in non GPLV2 code, which is obviously not
      intended. The user space API headers fall under the syscall exception
      which is in the kernels COPYING file:
      
         NOTE! This copyright does *not* cover user programs that use kernel
         services by normal system calls - this is merely considered normal use
         of the kernel, and does *not* fall under the heading of "derived work".
      
      otherwise syscall usage would not be possible.
      
      Update the files which contain no license information with an SPDX
      license identifier.  The chosen identifier is 'GPL-2.0 WITH
      Linux-syscall-note' which is the officially assigned identifier for the
      Linux syscall exception.  SPDX license identifiers are a legally binding
      shorthand, which can be used instead of the full boiler plate text.
      
      This patch is based on work done by Thomas Gleixner and Kate Stewart and
      Philippe Ombredanne.  See the previous patch in this series for the
      methodology of how this patch was researched.
      Reviewed-by: default avatarKate Stewart <kstewart@linuxfoundation.org>
      Reviewed-by: default avatarPhilippe Ombredanne <pombredanne@nexb.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      6f52b16c
    • Greg Kroah-Hartman's avatar
      License cleanup: add SPDX GPL-2.0 license identifier to files with no license · b2441318
      Greg Kroah-Hartman authored
      Many source files in the tree are missing licensing information, which
      makes it harder for compliance tools to determine the correct license.
      
      By default all files without license information are under the default
      license of the kernel, which is GPL version 2.
      
      Update the files which contain no license information with the 'GPL-2.0'
      SPDX license identifier.  The SPDX identifier is a legally binding
      shorthand, which can be used instead of the full boiler plate text.
      
      This patch is based on work done by Thomas Gleixner and Kate Stewart and
      Philippe Ombredanne.
      
      How this work was done:
      
      Patches were generated and checked against linux-4.14-rc6 for a subset of
      the use cases:
       - file had no licensing information it it.
       - file was a */uapi/* one with no licensing information in it,
       - file was a */uapi/* one with existing licensing information,
      
      Further patches will be generated in subsequent months to fix up cases
      where non-standard license headers were used, and references to license
      had to be inferred by heuristics based on keywords.
      
      The analysis to determine which SPDX License Identifier to be applied to
      a file was done in a spreadsheet of side by side results from of the
      output of two independent scanners (ScanCode & Windriver) producing SPDX
      tag:value files created by Philippe Ombredanne.  Philippe prepared the
      base worksheet, and did an initial spot review of a few 1000 files.
      
      The 4.13 kernel was the starting point of the analysis with 60,537 files
      assessed.  Kate Stewart did a file by file comparison of the scanner
      results in the spreadsheet to determine which SPDX license identifier(s)
      to be applied to the file. She confirmed any determination that was not
      immediately clear with lawyers working with the Linux Foundation.
      
      Criteria used to select files for SPDX license identifier tagging was:
       - Files considered eligible had to be source code files.
       - Make and config files were included as candidates if they contained >5
         lines of source
       - File already had some variant of a license header in it (even if <5
         lines).
      
      All documentation files were explicitly excluded.
      
      The following heuristics were used to determine which SPDX license
      identifiers to apply.
      
       - when both scanners couldn't find any license traces, file was
         considered to have no license information in it, and the top level
         COPYING file license applied.
      
         For non */uapi/* files that summary was:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|-------
         GPL-2.0                                              11139
      
         and resulted in the first patch in this series.
      
         If that file was a */uapi/* path one, it was "GPL-2.0 WITH
         Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|-------
         GPL-2.0 WITH Linux-syscall-note                        930
      
         and resulted in the second patch in this series.
      
       - if a file had some form of licensing information in it, and was one
         of the */uapi/* ones, it was denoted with the Linux-syscall-note if
         any GPL family license was found in the file or had no licensing in
         it (per prior point).  Results summary:
      
         SPDX license identifier                            # files
         ---------------------------------------------------|------
         GPL-2.0 WITH Linux-syscall-note                       270
         GPL-2.0+ WITH Linux-syscall-note                      169
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
         LGPL-2.1+ WITH Linux-syscall-note                      15
         GPL-1.0+ WITH Linux-syscall-note                       14
         ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
         LGPL-2.0+ WITH Linux-syscall-note                       4
         LGPL-2.1 WITH Linux-syscall-note                        3
         ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
         ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1
      
         and that resulted in the third patch in this series.
      
       - when the two scanners agreed on the detected license(s), that became
         the concluded license(s).
      
       - when there was disagreement between the two scanners (one detected a
         license but the other didn't, or they both detected different
         licenses) a manual inspection of the file occurred.
      
       - In most cases a manual inspection of the information in the file
         resulted in a clear resolution of the license that should apply (and
         which scanner probably needed to revisit its heuristics).
      
       - When it was not immediately clear, the license identifier was
         confirmed with lawyers working with the Linux Foundation.
      
       - If there was any question as to the appropriate license identifier,
         the file was flagged for further research and to be revisited later
         in time.
      
      In total, over 70 hours of logged manual review was done on the
      spreadsheet to determine the SPDX license identifiers to apply to the
      source files by Kate, Philippe, Thomas and, in some cases, confirmation
      by lawyers working with the Linux Foundation.
      
      Kate also obtained a third independent scan of the 4.13 code base from
      FOSSology, and compared selected files where the other two scanners
      disagreed against that SPDX file, to see if there was new insights.  The
      Windriver scanner is based on an older version of FOSSology in part, so
      they are related.
      
      Thomas did random spot checks in about 500 files from the spreadsheets
      for the uapi headers and agreed with SPDX license identifier in the
      files he inspected. For the non-uapi files Thomas did random spot checks
      in about 15000 files.
      
      In initial set of patches against 4.14-rc6, 3 files were found to have
      copy/paste license identifier errors, and have been fixed to reflect the
      correct identifier.
      
      Additionally Philippe spent 10 hours this week doing a detailed manual
      inspection and review of the 12,461 patched files from the initial patch
      version early this week with:
       - a full scancode scan run, collecting the matched texts, detected
         license ids and scores
       - reviewing anything where there was a license detected (about 500+
         files) to ensure that the applied SPDX license was correct
       - reviewing anything where there was no detection but the patch license
         was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
         SPDX license was correct
      
      This produced a worksheet with 20 files needing minor correction.  This
      worksheet was then exported into 3 different .csv files for the
      different types of files to be modified.
      
      These .csv files were then reviewed by Greg.  Thomas wrote a script to
      parse the csv files and add the proper SPDX tag to the file, in the
      format that the file expected.  This script was further refined by Greg
      based on the output to detect more types of files automatically and to
      distinguish between header and source .c files (which need different
      comment types.)  Finally Greg ran the script using the .csv files to
      generate the patches.
      Reviewed-by: default avatarKate Stewart <kstewart@linuxfoundation.org>
      Reviewed-by: default avatarPhilippe Ombredanne <pombredanne@nexb.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      b2441318