Commit Graph

2626 Commits

Author SHA1 Message Date
Sean Brogan c8543b8d83 BaseTools/Plugin: Update HostBasedUnitTestRunner to support Linux
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2513

Update HostBasedUnitTestRunner plugin to support the Linux environment
and remove any Windows only logic.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-05-08 05:54:46 +00:00
Leif Lindholm f355b98606 BaseTools: add handling for 'S:' flag to GetMaintainer.py
GetMaintainer.py already extracts the value of any S: tags for sections,
but it doesn't do anything with that information.

Print a warning message, with the status, for each matching section with
a status explicitly set to anything other than 'Supported' or
'Maintained'.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Leif Lindholm <leif@nuviainc.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-05-08 04:37:08 +00:00
Rebecca Cran 3a3713e62c BaseTools: add repo name option to SetupGit.py
Allow users who didn't clone one of the TianoCore repos from a
canonical URL to specify the name of the repo (edk2, edk2-platforms
or edk2-non-osi) when running SetupGit.py to allow them to configure
their repo properly.

The new option is:

  -n repo, --name repo  set the repo name to configure for, if not
                        detected automatically

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
2020-05-08 03:59:29 +00:00
Abner Chang ea56fa3d47 BaseTools: Enable RISC-V architecture for RISC-V EDK2 CI.
BZ:2562:
https://bugzilla.tianocore.org/show_bug.cgi?id=2562

EDK CI for RISC-V architecture

Enable RISC-V architecture for RISC-V EDK2 CI testing.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Reviewed-by: Sean Brogan <sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
Cc: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
2020-05-07 03:17:15 +00:00
Abner Chang f60d5ca97f BaseTools: BaseTools changes for RISC-V platform.
Python code changes for building EDK2 RISC-V platform.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Co-authored-by: Gilbert Chen <gilbert.chen@hpe.com>
Co-authored-by: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
2020-04-29 02:52:08 +00:00
Abner Chang 178938b2b9 BaseTools: BaseTools changes for RISC-V platform.
Tools definitions template file changes for building EDK2 RISC-V platform.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Co-authored-by: Gilbert Chen <gilbert.chen@hpe.com>
Co-authored-by: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
2020-04-29 02:52:08 +00:00
Abner Chang ad1db975c0 BaseTools: BaseTools changes for RISC-V platform.
C code changes for building EDK2 RISC-V platform.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Co-authored-by: Gilbert Chen <gilbert.chen@hpe.com>
Co-authored-by: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
2020-04-29 02:52:08 +00:00
Fan, Zhiju 3a3a3af4a2 BaseTools:Add the spare space FV image size checker
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=10000.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-04-23 01:29:34 +00:00
Shenglei Zhang 06033f5aba BaseTools: Make brotli a submodule
Use submodule way to access brotli in BaseTools based on
brotli version 666c3280cc11dc433c303d79a83d4ffbdd12cc8d.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2558

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-04-16 01:58:38 +00:00
Shenglei Zhang 776ec4ea3c BaseTools/WindowsVsToolChain.py: Update toolchain plugin
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2659
Allow WindowsVsToolChain Plugin to add libraries and headers
of user defined ARCH for VS2017 and VS2019.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
2020-04-13 03:13:36 +00:00
Vitaly Cheptsov d4bc5378e0 BaseTools: Use SEH exceptions in CLANGPDB for IA32
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2628

This patch reduces the size of IA32 binaries by ensuring that
no .debug_frame / .eh_frame sections are generated through forcing
SEH exception model, which is already the default in clang for X64.

EDK II does not support exceptions, and in future we should disable
them instead of switching to some other variant. Currently this
is not possible due to the following LLVM bugs:

https://bugs.llvm.org/show_bug.cgi?id=45324
https://bugs.llvm.org/show_bug.cgi?id=45325

Upon applying this patch OvmfPkgIA32.dsc compilation in DEBUG mode
gets the following size decrease with clang 9.0.1.

Before: FV Space Information
SECFV [11%Full] 212992 total, 24512 used, 188480 free
PEIFV [22%Full] 917504 total, 203048 used, 714456 free
DXEFV [36%Full] 11534336 total, 4215672 used, 7318664 free
FVMAIN_COMPACT [37%Full] 3440640 total, 1287776 used, 2152864 free

After: FV Space Information
SECFV [10%Full] 212992 total, 22112 used, 190880 free
PEIFV [19%Full] 917504 total, 176392 used, 741112 free
DXEFV [31%Full] 11534336 total, 3657112 used, 7877224 free
FVMAIN_COMPACT [33%Full] 3440640 total, 1153896 used, 2286744 free

Cc: Liming Gao <liming.gao@intel.com>
Cc: Marvin H?user <mhaeuser@outlook.de>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Zhiguang Liu <zhiguang.liu@intel.com>
Signed-off-by: Vitaly Cheptsov <vit9696@protonmail.com>
2020-04-08 14:24:09 +00:00
Sean Brogan 48f0e94921 BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640

Azure Pipelines agents have 2 threads. This commit has been shown to
reduce the build time in half on those agents.

Cc: Bob C Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Bob C Feng <bob.c.feng@intel.com>
2020-04-07 01:22:04 +00:00
Fan, ZhijuX 8c944c9383 BaseTools:GuidedSectionTools.txt is not generated correctly
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-31 04:09:43 +00:00
Fan, ZhijuX 2f524a745e BaseTools:Fix build tools print traceback info issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2610

We meet a case that the DEC file declaring the PCD isn't
included in the INF.it cause build tools report Traceback error.

Remove raise statements that generate Tracebacks that were only
intended for development/debug. With the raise statements removed
proper error messages are shown.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-24 02:27:27 +00:00
Fan, ZhijuX 0dee1d1358 BaseTools:fix issue for decode the stdout/stderr byte arrays
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2365

This patch is to fix a build tool regression issue which was introduced
by commit 8ddec24dea.

compiler output message includes localized string.
So build failed when code decode the stdout/stderr byte arrays.
The cause of the build failed is that Commit 8ddec24dea
removed "errors='ignore'".

The build tool does not need to deal with localized string,
so we need to add "errors='ignore'".

this function is only invoked for structure PCDs.
Build failed if structurePcd is used in platform dsc file.
The patch is going to fixed this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-24 02:27:27 +00:00
Fan, ZhijuX 484b1534ed BaseTools:copy the common PcdValueCommon.c to output directory
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2568

PcdValueInit shares the same Edk2\BaseTools\Source\C\PcdValueCommon.c.
To avoid the conflict, it should copy this file to its output directory,
If so, PcdValueCommon.c file will be private for PcdValueInit

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-12 02:02:55 +00:00
Bob Feng 2be4828af1 BaseTools: Remove invalid leading space before !INCLUDE in Makefile
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2563

This patch is to fix a incremental build regression bug
which happen when using nmake. That's introduced by 818283de3f.

If there is white space before !INCLUDE instruction, nmake will not
process it. Source code's dependent header files are listed in
${deps_file} file, if it's not included successfully, nmake will
not detect the change of those header file.

This patch has been verified in Windows with VS2015 and Linux with GCC5.
The header file add/modify/delete can trig the incremental build with this fix.
There is no impact on the clean build.

Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
2020-03-02 02:36:25 +00:00
Liu, Zhiguang 58bccfa57c BaseTools: remove -DNO_MSABI_VA_FUNCS option in CLANGPDB tool chain
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2524

remove -DNO_MSABI_VA_FUNCS option in CLANGPDB tool chain After CLANGPDB is
switched to GNU mode, to use MS ABI version of GCC built-in macros for
variable argument lists as same as CLANG38 tool chain.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-13 05:53:20 +00:00
Fan, ZhijuX 422bf2725b BaseTools:build failure in CLANGPDB tool chain
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2520

Incremental build failure in CLANGPDB tool chain on Windows host
The build failure is like below when do incremental build.
The root cause is in generated deps_target file. It has one line ":".

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-13 03:42:36 +00:00
Fan, ZhijuX ec97412b7c BaseTools: Fixed build failure when using python38
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2304

SyntaxWarning: "is" with a literal. Did you mean "=="?
Using "is" instead of "==" is an irregular syntax

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Bob C Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 14:54:03 +00:00
Fan, ZhijuX 82af1cbf0d BaseTools:fix Ecc tool issue for check StructPcd
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2142

gAdvancedFeaturePkgTokenSpaceGuid.PcdSmbiosType0BiosInformation|
{0x0}|SMBIOS_TABLE_TYPE0|0x80010000 {
    <HeaderFiles>
      IndustryStandard/SmBios.h
    <Packages>
      MdePkg/MdePkg.dec
      AdvancedFeaturePkg/AdvancedFeaturePkg.dec
  }

If there's a PcdStructHF or PcdStructPKGs in StructPcd,
EccTool report error,IndexError: list index out of range

This patch is going to fix this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 09:26:22 +00:00
Pierre Gondois c44e0a896c BaseTools: Remove caret in NASM_INC macro
NASM_INC contains the list of directory to include when using
the nasm assembler.

In nmake makefiles, a trailing backslash escapes the newline char
and replaces it with a space ' '. To have a literal trailing
backslash, it must be escaped with a caret '^'. This is not
necessary for GNU makefiles.

On windows platforms, for the NASM_INC macro, a caret escaping
a trailing a backslash was appended to the last included
folder regardless of the makefile type.
For instance, "/Include/" was replaced by "/Include/^\".

This is causing a build failure on windows platforms using
GNU makefiles since the caret '^' doesn't escape any chars in
GNU makefiles and is thus conserved.
"/Include^\" was replaced by "/Include\/" in nmake makefiles,
but remained "/Include/^\" in GNU makefiles.

This patch removes the caret '^' on the build using GNU makefiles.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Pierre Gondois 818283de3f BaseTools: Rationalise makefile generation
The GenMake.py script tests the platform environment
to determine the type of makefile that needs to be
generated. If a Windows build host is detected, the
makefile generated is of Nmake type. Otherwise a
GNUmake type is generated.

Furthermore, the <TARGET>_<TAGNAME>_<ARCH>_MAKE_PATH
option in tools_def.template defines the make tool
to use.
E.g.: for VS2017 this is configured to use Nmake, cf.
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
while for GCC5 it is setup to use GNU make.
*_GCC5_*_MAKE_PATH = DEF(GCC_HOST_PREFIX)make

This prevents using the GCC compiler toolchain on a
Windows build host.

To address this issue this patch introduces 2 factors
to determine the generated makefile output.
  1. Platform -> to determine shell commands used
                 in makefile.
  2. MakeTool -> to determine the type of makefile
                 that needs to be generated.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Vitaly Cheptsov 69c135462d BaseTools: Switch to GNU mode for CLANGPDB
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2397

Signed-off-by: Vitaly Cheptsov <vit9696@protonmail.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-11 12:21:21 +00:00
Bob Feng b34ed98694 BaseTools: Fixed a Incremental build issue
The .map file is not update to FFS_OUTPUT_DIR folder
in the incremental build.

Signed-off-by: Guo Dong <guo.dong@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-09 09:29:02 +00:00
Michael D Kinney 0a5d2b505c BaseTools/Plugin: Add HostBasedUnitTestRunner plugin
https://bugzilla.tianocore.org/show_bug.cgi?id=2505

Add plugin to BaseTools to run host based unit tests.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
Acked-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 19:18:53 +00:00
Sean Brogan c6a6193d12 BaseTools/PcdValueCommon: Fix 64-bit host compiler error
https://bugzilla.tianocore.org/show_bug.cgi?id=2496

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Sean Brogan 3b4ad37ebe BaseTools/WindowsVsToolChain: Setup VS2017/VS2019 env
https://bugzilla.tianocore.org/show_bug.cgi?id=2495

Update the WindowsVsToolChain plugin to setup the VS2017
or VS2019 development environment.  This is required to
build BaseTools and Structured PCD host applications.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Sean Brogan d14feb6cb7 BaseTools/WindowsVsToolChain: Clean up Python source formatting
https://bugzilla.tianocore.org/show_bug.cgi?id=2495

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Kinney, Michael D 45b0be3840 BaseTools/Build: Do not use Common.lib in Structured PCD app
https://bugzilla.tianocore.org/show_bug.cgi?id=2496

Reduce the build and env dependencies for the Structured PCD
application by removing the dependency on Common.lib that
is only built when BaseTools is built which does not
happen if pre-compiled BaseToools are used.  Change the
makefile for the Structure PCD application to build all
files from sources which adds PcdValueCommon.c to the
makefile.  Also remove PcdValueCommon.c from Common.lib.

With the change to the makefile for the Structured PCD
application, multiple C files are compiled.  Only
PcdValueInit.c contains the extra information expected
by the error/warning message parser.  Only parse the
DSC line number into an error message if there is an
error/warning in PcdValueInit.c.  Errors/warnings in
other files should be passed through.  This fixes a build
failure with no useful log information that was observed
when there was a compiler error in PcdValueCommon.c.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Steven 77b738b36f BaseTools: Enhance call stack unwindability for CLANGPDB x64 binary
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2487

The call stack unwindability of the COFF X64 binary requires
the binary to remain the pdata and xdata sections.
Details see the MSVC X64 calling convertion doc in below link:
https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention

Current build options discard or zero the data in pdata and xdata
sections which cause the debugger cannot correctly unwind the
X64 binary call stack in the runtime.
Enhance the build options to force emit the unwind tables and
keep the data of pdata and xdata sections correct in the binary.

Signed-off-by: Steven Shi <steven.shi@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-07 05:38:39 +00:00
Liming Gao b5808fe960 BaseTools tools_def.template: Add back -fno-pie option in GCC49 tool chain
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2502
This option is required to make GCC49 tool chain work with the high
version GCC compiler.

Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 03:07:21 +00:00
Pierre Gondois 1549651da6 BaseTools: Script for converting .aml to .hex
The "-tc" option of the iasl compiler allows to generate a
.hex file containing a C array storing AML bytecode.

An online discussion suggested that this "-tc" option
was specific to the iasl compiler and it shouldn't be relied
on. This conversation is available at:
https://edk2.groups.io/g/devel/topic/39786201#49659

A way to address this issue is to implement a compiler
independent script that takes an AML file as input, and
generates a .hex file.

This patch implements a Python script that converts an AML
file to a .hex file, containing a C array storing AML bytecode.
This scipt has been tested with the AML output from the
following compilers supported by the EDKII implementation:
  * Intel ASL compiler
  * Microsoft ASL compiler

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-06 13:52:11 +00:00
Philippe Mathieu-Daude c37cce7a84 BaseTools/Scripts/PatchCheck.py: Do not use mailmap
We check the author/committer name/email are properly displayed
since commits 8ffa47fb3ab..c0328cf3803. However if PatchCheck.py
uses the mailmap, it will check sanitized names/emails.
Use the --no-use-mailmap option so PatchCheck.py will check
unsanitized input.

Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-06 13:16:09 +00:00
Philippe Mathieu-Daude 38ed2ff3dd BaseTools/Scripts/PatchCheck.py: Detect emails rewritten by Groups.Io
Due to strict DMARC / DKIM / SPF rules, Groups.Io sometimes rewrite
the author email. See for example commit df851da3ce.
Add a check to detect these rewrites with PatchCheck.py.

Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-06 13:16:09 +00:00
Philippe Mathieu-Daude a4960cf1b6 BaseTools/Scripts: Add log.mailmap to SetupGit.py
We added .mailmap to the repository in commit 4a1aeca3bd
to display commit mistakes fixed. Use this option by default in our
git setup.

Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-06 13:16:09 +00:00
Liu, Zhiguang 7990438f14 BaseTools: append -DNO_MSABI_VA_FUNCS option in CLANGPDB tool chain
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2415

Define NO_MSABI_VA_FUNCS to use GCC built-in macros for variable argument
lists for CLANGPDB tool chain.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-06 08:55:58 +00:00
Michael D Kinney b47fe2655d BaseTools/DscBuildData: Fix PCD autogen include file conflict
https://bugzilla.tianocore.org/show_bug.cgi?id=2494

When using structured PCDs, a C application is auto generated
to fill in the structured PCD value.  The C application uses
the standard include files <stdio.h>, <stdlib.h>, and <string.h>.
This C application also supports include paths from package DEC
files when a structured PCD declaration provides a <Packages>
list.  The complete list of include paths are -I options for
include paths from package DEC files and the compiler's standard
include paths.

-I include paths are higher priority than the standard include
paths.  If the -I included paths from package DEC files contain
<stdio.h>, <stdlib.h>, or <string.h> the wrong include files are
used to compile the C application for the structured PCD value.

Update GenerateByteArrayValue() to skip a package DEC include
paths that contain <stdio.h>, <stdlib.h>, or <string.h>.

Build failures were observed when adding a structured PCD to
CryptoPkg.  CryptoPkg contains <stdio.h>, <stdlib.h>, and
<string.h> in the path CryptoPkg/Library/Include to support
building Open SSL.  The Library/Include path is listed as a
private include path in CryptoPkg.dec.  Without this change, the
standard include files designed to support build OpenSLL are
used to build the structured PCD C application, and that build
fails.

Other packages that provide a standard C lib or a gasket for
a subset of the standard C lib will run into this same issue
if they also define and use a Structured PCD.  So this issue
is not limited to the CryptoPkg.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-04 20:46:22 +00:00
Laszlo Ersek eafd990f26 BaseTools/Conf/gitattributes: fix "--function-context" for C source code
The "--function-context" ("-W") option of git-diff displays the entire
body of a modified function, not just small modified hunks within the
function. It is useful for reviewers when the code changes to the function
are small, but they could affect, or depend on, control flow that is far
away in the same function.

Of course, the size of the displayed context can be controlled with the
"-U" option anyway, but such fixed-size contexts are usually either too
small, or too large, in the above scenario.

It turns out that "--function-context" does not work correctly for C
source files in edk2. In particular, labels for the goto instruction
(which the edk2 coding style places in the leftmost column) appear to
terminate "--function-context".

The "git" utility contains built-in hunk header patterns for the C and C++
languages. However, they do not take effect in edk2 because we don't
explicitly assign the "cpp" git-diff driver to our C files. The
gitattributes(5) manual explains that this is required:

>            There are a few built-in patterns to make this easier, and
>            tex is one of them, so you do not have to write the above in
>            your configuration file (you still need to enable this with
>            the attribute mechanism, via .gitattributes). The following
>            built in patterns are available:
>
>            [...]
>
>            *   cpp suitable for source code in the C and C++
>                languages.

The key statement is the one in parentheses.

Grab the suffix lists from the [C-Code-File] and [Acpi-Table-Code-File]
sections of "BaseTools/Conf/build_rule.template", add "*.h" and "*.H", and
mark those as belonging to the "cpp" git-diff driver.

This change has a dramatic effect on the following command, for example:

$ git show -W 2ef0c27cb8

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20200120094245.9010-1-lersek@redhat.com>
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-01-31 16:13:06 +00:00
Michael D Kinney c8b8157e12 BaseTools/Scripts/PatchCheck.py: Remove submodule false positives
https://bugzilla.tianocore.org/show_bug.cgi?id=2484
https://bugzilla.tianocore.org/show_bug.cgi?id=2485

Update PatchCheck to not enforce no tabs and not enforce CR/LF
line endings for .gitmodules files.  These files are updated by
git when a git submodule command is used and the updates by git
use tab characters and LF line endings.

Also update patch check to not enforce CR/LF line endings for
patch lines that create a submodule directory.  These patch
lines use LF line endings.  The git submodule directory is
added as a new file with attributes 160000 that can be detected
by looking for the pattern "new file mode 160000".

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2020-01-24 18:30:13 +00:00
Bob Feng 5cd3d4bc43 BaseTools: Fixed a incremental build bug
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2451

If removing a header file from source code and file
system, the incremental build will fail.

This patch is to fix this issue by setting each header file
as a target without any actions in makefile.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-01-19 07:46:18 +00:00
Li, Aaron f6f66e0c30 BaseTools/Capsule: Add capsule dependency support
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2412

Capsule generate tool support encode capsule dependencies through '-j'
command with a JSON file. To enable dependency feature, "Dependencies"
field for each payload in JSON file is required.
The value of "Dependencies" field is C style infix notation expression.
For example:
  "Dependencies":"72E2945A-00DA-448E-9AA7-075AD840F9D4 > 0x00000001"

The relation of Dependency Expression Opcode in UEFI2.8 chap 23.2 and
infix notation expression value is as follows:
+-----------------------------+--------------------------+
| OPCODE                      | INFIX EXPRESSION VALUE   |
+-----------------------------+--------------------------+
| 0x00 (PUSH_GUID)            | {GUID}                   |
| 0x01 (PUSH_VERSION)         | {UINT32}                 |
| 0x02 (DECLEAR_VERSION_NAME} | DECLEAR "{VERSION_NAME}" |
| 0x03 (AND)                  | &&                       |
| 0x04 (OR)                   | ||                       |
| 0x05 (NOT)                  | ~                        |
| 0x06 (TRUE)                 | TRUE                     |
| 0x07 (FALSE)                | FALSE                    |
| 0x08 (EQ)                   | ==                       |
| 0x09 (GT)                   | >                        |
| 0x0A (GTE)                  | >=                       |
| 0x0B (LT)                   | <                        |
| 0x0C (LTE)                  | <=                       |
+-----------------------------+--------------------------+

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Signed-off-by: Aaron Li <aaron.li@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-01-15 03:16:46 +00:00
Michael D Kinney b112ec225f BaseTools/Scripts/PatchCheck: Address false error conditions
https://bugzilla.tianocore.org/show_bug.cgi?id=2406

* Always print subject line after the git commit id to make
  it easier to know the context of warnings or errors.
* Allow UTF-8 characters in subject line
* Error if subject line length > 75 without CVE-xxx-xxxxx present
* Error if subject line length > 92 with CVE-xxxx-xxxxx present
* If body line length is > 75, then print warning instead of error.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
2020-01-13 18:18:03 +00:00
Fan, ZhijuX 4465cd124f BaseTools:Fix GenFds issue for BuildOption replace GenFdsOption
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2455

BuildOption is used by TargetTxtClassObj.py
GenFdsOption is used by GenFds.py
When the GenFds tool is used alone (e.g. python3 -m GenFds.GenFds -h)
With the OptionParser function, the first detected function
prints the help message

import TargetTxtClassObj to GenFds,
The BuildOption will be executed and replace GenFdsOption

We removed all objects associated with this problem that
were created directly during the import process
(e.g. BuildOption, BuildTarget = MyOptionParser(),
 TargetTxt = TargetTxtDict())

The Patch is going to fix this issue

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-01-13 02:08:46 +00:00
Fan, ZhijuX 072b9c2839 BaseTools:Change the case rules for ECC check pointer names
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2087

In CryptHkdf.c  line 42

  EVP_PKEY_CTX *pHkdfCtx;

Variable pHkdfCtx begins with lower case 'p',
which should be acceptable because it it is a pointer.
(Refer to CCS_2_1_Draft, 4.3.3.3)

So ECC tool should be improved to handle issues like this.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
2020-01-13 02:08:46 +00:00
Philippe Mathieu-Daude c0328cf380 BaseTools/PatchCheck.py: Check the patch author email address
To avoid patches committed with incorrect email address,
use the EmailAddressCheck class on the author email too.

Example:

  $ python BaseTools/Scripts/PatchCheck.py 1a04951309
  Checking git commit: 1a04951309
  The 'Author' email address is not valid:
  * The email address cannot contain a space: /o=Intel/ou=External \
    (FYDIBOHF25SPDLT)/cn=Recipients/cn=fe425ca7e5f4401abed22b904fe5d964

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
2020-01-10 04:06:42 +00:00
Philippe Mathieu-Daude 8120390aab BaseTools/PatchCheck.py: Let EmailAddressCheck describe email checked
We are checking different emails from the signature list. We are
going to check more. To be able to differency, add a description
field, so the error reported is clearer.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
2020-01-10 04:06:42 +00:00
Philippe Mathieu-Daude 8f38b08b50 BaseTools/PatchCheck.py: Check the committer email address
To avoid patches committed with incorrect email address,
use the EmailAddressCheck class on the committer email too.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
2020-01-10 04:06:42 +00:00
Philippe Mathieu-Daude 8ffa47fb3a BaseTools/PatchCheck.py: Extract email check code to EmailAddressCheck
As we are going to reuse this code out of the CommitMessageCheck
class, extract it in a new class: EmailAddressCheck.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
2020-01-10 04:06:42 +00:00
Philippe Mathieu-Daud? 2649a735b2 BaseTools/PatchCheck.py: Ignore CR and LF characters in subject length
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=113

Strip the trailing characters before checking the subject line is
less than 72 characters.

Fixes: e61406708c
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
2020-01-09 06:09:33 +00:00