Commit Graph

2646 Commits

Author SHA1 Message Date
Yuwei Chen d35773d5c0 BaseTools: Add Guid name support in GenFfs.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2762

The Fv Section in the FDF files use hard coding Guid values
which is inconvenient to manage. This patch adds Guid name
support in GenFfs to solve this problem.

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
2020-07-17 05:25:41 +00:00
Michael D Kinney 5b86bbf891 BaseTools/Python: Allow HOST_APPLICATION to use NULL libraries
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2797

Update HOST_APPLICATION module type to use NULL library instances.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Sean Brogan <sean.brogan@microsoft.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-07-15 05:25:21 +00:00
Leif Lindholm 627d1d6693 BaseTools: convert diff.order to LF-only
SetupGit.py sets the git config option diff.orderFile to
{edk2 directory}/BaseTools/Conf/diff.order, to override the default order
in which files are shown in a diff/patch/whatever. This is in imitation
of what is done manually in Laszlo's Unkempt Guide.

However, the version currently in the tree is in CRLF format, which makes
git interpret e.g. *.c as matching on *.c<CR>, finding no matches and
failing to apply the desired reordering. Note: this is true regardless of
whether running on Linux or Windows.

Convert the file to LF-only to make it work as expected.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Leif Lindholm <leif@nuviainc.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-07-03 11:20:01 +00:00
Leif Lindholm f56d52c7f5 BaseTools: explicitly import email.header PatchCheck.py
On Debian 10 (Buster), when running PatchCheck.py with python2, a
backtrace is printed, starting from:

  File "../edk2/BaseTools/Scripts/PatchCheck.py", line 595,
   in find_patch_pieces
    parts = email.header.decode_header(pmail.get('subject'))
  AttributeError: 'module' object has no attribute 'header'

When using python3, this backtrace does not appear.

Explicitly importing email.header resolves this for python2 and does not
appear to cause any issues with python3.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Leif Lindholm <leif@nuviainc.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-03 00:18:50 +00:00
Leif Lindholm c267eb889f BaseTools/PatchCheck.py: add exception for diff orderfile
SetupGit.py adds BaseTools/Conf/diff.order as a diff orderfile, but that
file currently has CRLF line endings, which causes all pattern matches
to fail and the ordering remaining unaffected.

Add an exception to PatchCheck.py (to the existing .gitmodules clause),
so that we can merge the fix to the config file.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Leif Lindholm <leif@nuviainc.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-03 00:18:50 +00:00
Pierre Gondois 0622a7b1b2 BaseTools: Fix string concatenation
Using Python 3.7.2 on win32, when printing a FileBuildRule
instance, the following error occurs:
File "edk2\BaseTools\Source\Python\AutoGen\BuildEngine.py",
line 177, in __str__
  DestString = ", ".join(self.DestFileList)
  TypeError: sequence item 0: expected str instance, PathClass found

This patch converts each PathClass element of the list to a string
instance before concatenating them.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 0a4aa20e8d BaseTools: Compile AML bytecode arrays into .obj file
The AmlToHex script and Posix/WindowsLike wrappers convert
an AML file to a .hex file, containing a C array storing
AML bytecode. This ".hex" file can then be included in a
C file, allowing to access the AML bytecode from this C
file.

The EDK2 build system doesn't allow to a depict dependency
orders between files of different languages. For instance,
in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a
".c" file since this later ".hex" file may or may not
have been created yet.

This patch modifies the AmlToC script to generate a C file
instead of a ".hex" file.
It also adds the generation of an intermediate ".amli" file
when compiling an ASL file, and adds a rule to convert this
".amli" to a C file.

This allows to generate a C file containing the AML bytecode
from an ASL file. This C file will then be handled by the EDK2
build system to generate an object file.
Thus, no file inclusion will be required anymore. The C file
requiring the AML bytecode as a C array, and the ASL file,
will be compiled independently. The C array must be defined
as an external symbol. The linker is resolving the
reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;
 -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 88228db38e BaseTools: Rename AmlToHex script to AmlToC
The AmlToHex script and Posix/WindowsLike wrappers convert
an AML file to a .hex file, containing a C array storing
AML bytecode. This ".hex" file can then be included in a
C file, allowing to access the AML bytecode from this C
file.

The EDK2 build system doesn't allow to a depict dependency
orders between files of different languages. For instance,
in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a
".c" file since this later ".hex" file may or may not
have been created yet.

This patch renames the script as AmlToC. It is posted as
a separate patch to prevent git from seeing the renaming
as a deletion plus addition of a new file.
The ending line of the posix-like bin-wrapper script has
also been corrected.

This is a first step toward generating a C file containing
the AML bytecode from an ASL file. This C file will then
be handled by the EDK2 build system to generate an object
file.
Thus, no file inclusion will be required anymore. The C file
requiring the AML bytecode as a C array, and the ASL file,
will be compiled independently. The C array must be defined
as an external symbol. The linker is resolving the
reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;
 -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 6511277827 BaseTools: Generate multiple rules when multiple output files
This patch modifies the Makefile generation not to stop
adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a
module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files
are a final target. However, there is still [Rule2] to
apply to .Z files.

This patch also adds a dependency between the first
ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y
and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the
execution of the Makefile. Indeed, .Y and .Z files are
generated by the execution of a common set of commands,
and without this rule, there is no explicit dependency
relation between them.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 01356d2963 BaseTools: PatchCheck: Exclude bash scripts from CRLF check
Bash scripts require LF line endings to work.
PatchCheck.py checks that the files added in a patch have CRLF
line endings. It excludes files ending with the ".sh" extension
from this check.

Some bash script don't have a ".sh" extension. Most of them are
located in:
 - BaseTools/BinWrappers/PosixLike/
 - BaseTools/Bin/CYGWIN_NT-5.1-i686/

This patch excludes these folder plus BaseTools/BuildEnv from
this CRLF check.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 09:29:38 +00:00
Abner Chang 20286e168b BaseTools: Add external definitions for RISC-V assembly build
Add opensbi external definitions to RISC-V build for assembly
code. Use GCC5_RISCV_OPENSBI_TYPES to refer to edk2 data type
for assembly files instead of using opensbi data type.

Signed-off-by: Abner Chang <abner.chang@hpe.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Daniel Schaefer <daniel.schaefer@hpe.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
2020-06-24 06:51:45 +00:00
Shenglei Zhang a4cfb842fc BaseTools/PatchCheck.py: Add LicenseCheck
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
For files to be added to the tree, this feature will check
whether it has BSD plus patent license. If not, licenses listed in
Readme are also accepted but warning will be reported.
Otherwise, it should be error.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-06-12 06:26:13 +00:00
Liming Gao 3ee4f6cb36 BaseTools GenFv: Report the correct spare FV image size
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2790

If the top FFS is placed in FV image, current FV will show there is no space.
In fact, the pad ffs in FV image can be regarded as the spare space.
This change reports the max pad ffs size as the spare space for use.

Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-06-11 14:36:54 +00:00
Bob Feng cfd73e0065 BaseTools: Warn user the file not found issue instead of break build.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2770

The Trim.py would break the build process when the file not found
issue occures, however sometimes we do not care about this issue.
This patch changes the error with warning in order to solve this
kind of break.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
2020-06-07 13:36:33 +00:00
Irene Park 28dd887d68 BaseTools/build.py: Exit with 1 when AutoGen error occurred
AutoGen manager/workers halt the progress when an error occurs but
doesn't propagate the error code to main and allows main exit with 0
and gets the build system unable to catch the occurrence of an error.
This change informs main with an error when a progress is halted and
helps main exit with 1.

Signed-off-by: Irene Park <ipark@nvidia.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
2020-06-07 12:51:25 +00:00
Sami Mujawar b1357a40fc BaseTools: Remove deprecated Visual Studio Option
The VS2017 compiler reports 'warning D9035 : option
'Gm' has been deprecated and will be removed in a
future release'

The documentation for the 'Gm' option at
https://docs.microsoft.com/en-us/cpp/build/reference/gm-enable-minimal-rebuild?view=vs-2019
indicates that this option can be safely removed
from the project.

Therefore, remove the deprecated 'Gm' Visual Studio
Compiler option.

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2660

Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-05-21 14:19:17 +00:00
Oleksiy Yakovlev 5c7526f501 BaseTools: Bootable NVDIMM namespaces
Provided a mechanism for UEFI FW to identify and hand off bootable
NVDIMM namespaces to the OS by standardizing the EFI device path.
EFI device path for physical NVDIMM devices changed from an ACPI
_ADR device to an ACPI NVDIMM device for correctness.
(UEFI 2.8 mantis 1858)

Signed-off-by: Oleksiy Yakovlev <oleksiyy@ami.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Zhiguang Liu <zhiguang.liu@intel.com>
2020-05-15 05:05:52 +00:00
Oleksiy Yakovlev ebe377f9eb BaseTools: REST style formset
In question level, a new flag EFI_IFR_FLAG_REST_STYLE is defined.

(UEFI 2.8 mantis 1853)

Signed-off-by: Oleksiy Yakovlev <oleksiyy@ami.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Zhiguang Liu <zhiguang.liu@intel.com>
2020-05-15 05:05:52 +00:00
Feng, YunhuaX ceacd9e992 BaseTools: Fix parse PCD GUID expression issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2686

The build tool will give an incorrect GUID value if the GUID includes character ' or " ASCII value.
This patch is going to fix this issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-05-13 06:37:18 +00:00
Michael Kubacki 242ab73d7f BaseTools/Ecc: Replace deprecated function time.clock()
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2707

Ecc fails with Python 3.8 because it uses the deprecated time.clock()
function - https://docs.python.org/3.7/library/time.html#time.clock

This change updates EccMain.py to use time.perf_counter().

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-05-13 05:52:03 +00:00
Sean Brogan c8543b8d83 BaseTools/Plugin: Update HostBasedUnitTestRunner to support Linux
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2513

Update HostBasedUnitTestRunner plugin to support the Linux environment
and remove any Windows only logic.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-05-08 05:54:46 +00:00
Leif Lindholm f355b98606 BaseTools: add handling for 'S:' flag to GetMaintainer.py
GetMaintainer.py already extracts the value of any S: tags for sections,
but it doesn't do anything with that information.

Print a warning message, with the status, for each matching section with
a status explicitly set to anything other than 'Supported' or
'Maintained'.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Leif Lindholm <leif@nuviainc.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-05-08 04:37:08 +00:00
Rebecca Cran 3a3713e62c BaseTools: add repo name option to SetupGit.py
Allow users who didn't clone one of the TianoCore repos from a
canonical URL to specify the name of the repo (edk2, edk2-platforms
or edk2-non-osi) when running SetupGit.py to allow them to configure
their repo properly.

The new option is:

  -n repo, --name repo  set the repo name to configure for, if not
                        detected automatically

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
2020-05-08 03:59:29 +00:00
Abner Chang ea56fa3d47 BaseTools: Enable RISC-V architecture for RISC-V EDK2 CI.
BZ:2562:
https://bugzilla.tianocore.org/show_bug.cgi?id=2562

EDK CI for RISC-V architecture

Enable RISC-V architecture for RISC-V EDK2 CI testing.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Reviewed-by: Sean Brogan <sean.brogan@microsoft.com>

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
Cc: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
2020-05-07 03:17:15 +00:00
Abner Chang f60d5ca97f BaseTools: BaseTools changes for RISC-V platform.
Python code changes for building EDK2 RISC-V platform.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Co-authored-by: Gilbert Chen <gilbert.chen@hpe.com>
Co-authored-by: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
2020-04-29 02:52:08 +00:00
Abner Chang 178938b2b9 BaseTools: BaseTools changes for RISC-V platform.
Tools definitions template file changes for building EDK2 RISC-V platform.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Co-authored-by: Gilbert Chen <gilbert.chen@hpe.com>
Co-authored-by: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
2020-04-29 02:52:08 +00:00
Abner Chang ad1db975c0 BaseTools: BaseTools changes for RISC-V platform.
C code changes for building EDK2 RISC-V platform.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Co-authored-by: Gilbert Chen <gilbert.chen@hpe.com>
Co-authored-by: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
2020-04-29 02:52:08 +00:00
Fan, Zhiju 3a3a3af4a2 BaseTools:Add the spare space FV image size checker
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=10000.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-04-23 01:29:34 +00:00
Shenglei Zhang 06033f5aba BaseTools: Make brotli a submodule
Use submodule way to access brotli in BaseTools based on
brotli version 666c3280cc11dc433c303d79a83d4ffbdd12cc8d.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2558

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-04-16 01:58:38 +00:00
Shenglei Zhang 776ec4ea3c BaseTools/WindowsVsToolChain.py: Update toolchain plugin
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2659
Allow WindowsVsToolChain Plugin to add libraries and headers
of user defined ARCH for VS2017 and VS2019.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
2020-04-13 03:13:36 +00:00
Vitaly Cheptsov d4bc5378e0 BaseTools: Use SEH exceptions in CLANGPDB for IA32
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2628

This patch reduces the size of IA32 binaries by ensuring that
no .debug_frame / .eh_frame sections are generated through forcing
SEH exception model, which is already the default in clang for X64.

EDK II does not support exceptions, and in future we should disable
them instead of switching to some other variant. Currently this
is not possible due to the following LLVM bugs:

https://bugs.llvm.org/show_bug.cgi?id=45324
https://bugs.llvm.org/show_bug.cgi?id=45325

Upon applying this patch OvmfPkgIA32.dsc compilation in DEBUG mode
gets the following size decrease with clang 9.0.1.

Before: FV Space Information
SECFV [11%Full] 212992 total, 24512 used, 188480 free
PEIFV [22%Full] 917504 total, 203048 used, 714456 free
DXEFV [36%Full] 11534336 total, 4215672 used, 7318664 free
FVMAIN_COMPACT [37%Full] 3440640 total, 1287776 used, 2152864 free

After: FV Space Information
SECFV [10%Full] 212992 total, 22112 used, 190880 free
PEIFV [19%Full] 917504 total, 176392 used, 741112 free
DXEFV [31%Full] 11534336 total, 3657112 used, 7877224 free
FVMAIN_COMPACT [33%Full] 3440640 total, 1153896 used, 2286744 free

Cc: Liming Gao <liming.gao@intel.com>
Cc: Marvin H?user <mhaeuser@outlook.de>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Zhiguang Liu <zhiguang.liu@intel.com>
Signed-off-by: Vitaly Cheptsov <vit9696@protonmail.com>
2020-04-08 14:24:09 +00:00
Sean Brogan 48f0e94921 BaseTools: Update Edk2ToolsBuild.py to use multiple threads on Linux
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2640

Azure Pipelines agents have 2 threads. This commit has been shown to
reduce the build time in half on those agents.

Cc: Bob C Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Bob C Feng <bob.c.feng@intel.com>
2020-04-07 01:22:04 +00:00
Fan, ZhijuX 8c944c9383 BaseTools:GuidedSectionTools.txt is not generated correctly
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-31 04:09:43 +00:00
Fan, ZhijuX 2f524a745e BaseTools:Fix build tools print traceback info issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2610

We meet a case that the DEC file declaring the PCD isn't
included in the INF.it cause build tools report Traceback error.

Remove raise statements that generate Tracebacks that were only
intended for development/debug. With the raise statements removed
proper error messages are shown.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-24 02:27:27 +00:00
Fan, ZhijuX 0dee1d1358 BaseTools:fix issue for decode the stdout/stderr byte arrays
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2365

This patch is to fix a build tool regression issue which was introduced
by commit 8ddec24dea.

compiler output message includes localized string.
So build failed when code decode the stdout/stderr byte arrays.
The cause of the build failed is that Commit 8ddec24dea
removed "errors='ignore'".

The build tool does not need to deal with localized string,
so we need to add "errors='ignore'".

this function is only invoked for structure PCDs.
Build failed if structurePcd is used in platform dsc file.
The patch is going to fixed this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-24 02:27:27 +00:00
Fan, ZhijuX 484b1534ed BaseTools:copy the common PcdValueCommon.c to output directory
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2568

PcdValueInit shares the same Edk2\BaseTools\Source\C\PcdValueCommon.c.
To avoid the conflict, it should copy this file to its output directory,
If so, PcdValueCommon.c file will be private for PcdValueInit

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-12 02:02:55 +00:00
Bob Feng 2be4828af1 BaseTools: Remove invalid leading space before !INCLUDE in Makefile
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2563

This patch is to fix a incremental build regression bug
which happen when using nmake. That's introduced by 818283de3f.

If there is white space before !INCLUDE instruction, nmake will not
process it. Source code's dependent header files are listed in
${deps_file} file, if it's not included successfully, nmake will
not detect the change of those header file.

This patch has been verified in Windows with VS2015 and Linux with GCC5.
The header file add/modify/delete can trig the incremental build with this fix.
There is no impact on the clean build.

Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
2020-03-02 02:36:25 +00:00
Liu, Zhiguang 58bccfa57c BaseTools: remove -DNO_MSABI_VA_FUNCS option in CLANGPDB tool chain
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2524

remove -DNO_MSABI_VA_FUNCS option in CLANGPDB tool chain After CLANGPDB is
switched to GNU mode, to use MS ABI version of GCC built-in macros for
variable argument lists as same as CLANG38 tool chain.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-13 05:53:20 +00:00
Fan, ZhijuX 422bf2725b BaseTools:build failure in CLANGPDB tool chain
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2520

Incremental build failure in CLANGPDB tool chain on Windows host
The build failure is like below when do incremental build.
The root cause is in generated deps_target file. It has one line ":".

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-13 03:42:36 +00:00
Fan, ZhijuX ec97412b7c BaseTools: Fixed build failure when using python38
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2304

SyntaxWarning: "is" with a literal. Did you mean "=="?
Using "is" instead of "==" is an irregular syntax

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Bob C Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 14:54:03 +00:00
Fan, ZhijuX 82af1cbf0d BaseTools:fix Ecc tool issue for check StructPcd
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2142

gAdvancedFeaturePkgTokenSpaceGuid.PcdSmbiosType0BiosInformation|
{0x0}|SMBIOS_TABLE_TYPE0|0x80010000 {
    <HeaderFiles>
      IndustryStandard/SmBios.h
    <Packages>
      MdePkg/MdePkg.dec
      AdvancedFeaturePkg/AdvancedFeaturePkg.dec
  }

If there's a PcdStructHF or PcdStructPKGs in StructPcd,
EccTool report error,IndexError: list index out of range

This patch is going to fix this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 09:26:22 +00:00
Pierre Gondois c44e0a896c BaseTools: Remove caret in NASM_INC macro
NASM_INC contains the list of directory to include when using
the nasm assembler.

In nmake makefiles, a trailing backslash escapes the newline char
and replaces it with a space ' '. To have a literal trailing
backslash, it must be escaped with a caret '^'. This is not
necessary for GNU makefiles.

On windows platforms, for the NASM_INC macro, a caret escaping
a trailing a backslash was appended to the last included
folder regardless of the makefile type.
For instance, "/Include/" was replaced by "/Include/^\".

This is causing a build failure on windows platforms using
GNU makefiles since the caret '^' doesn't escape any chars in
GNU makefiles and is thus conserved.
"/Include^\" was replaced by "/Include\/" in nmake makefiles,
but remained "/Include/^\" in GNU makefiles.

This patch removes the caret '^' on the build using GNU makefiles.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Pierre Gondois 818283de3f BaseTools: Rationalise makefile generation
The GenMake.py script tests the platform environment
to determine the type of makefile that needs to be
generated. If a Windows build host is detected, the
makefile generated is of Nmake type. Otherwise a
GNUmake type is generated.

Furthermore, the <TARGET>_<TAGNAME>_<ARCH>_MAKE_PATH
option in tools_def.template defines the make tool
to use.
E.g.: for VS2017 this is configured to use Nmake, cf.
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
while for GCC5 it is setup to use GNU make.
*_GCC5_*_MAKE_PATH = DEF(GCC_HOST_PREFIX)make

This prevents using the GCC compiler toolchain on a
Windows build host.

To address this issue this patch introduces 2 factors
to determine the generated makefile output.
  1. Platform -> to determine shell commands used
                 in makefile.
  2. MakeTool -> to determine the type of makefile
                 that needs to be generated.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Vitaly Cheptsov 69c135462d BaseTools: Switch to GNU mode for CLANGPDB
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2397

Signed-off-by: Vitaly Cheptsov <vit9696@protonmail.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-11 12:21:21 +00:00
Bob Feng b34ed98694 BaseTools: Fixed a Incremental build issue
The .map file is not update to FFS_OUTPUT_DIR folder
in the incremental build.

Signed-off-by: Guo Dong <guo.dong@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-09 09:29:02 +00:00
Michael D Kinney 0a5d2b505c BaseTools/Plugin: Add HostBasedUnitTestRunner plugin
https://bugzilla.tianocore.org/show_bug.cgi?id=2505

Add plugin to BaseTools to run host based unit tests.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
Acked-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 19:18:53 +00:00
Sean Brogan c6a6193d12 BaseTools/PcdValueCommon: Fix 64-bit host compiler error
https://bugzilla.tianocore.org/show_bug.cgi?id=2496

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Sean Brogan 3b4ad37ebe BaseTools/WindowsVsToolChain: Setup VS2017/VS2019 env
https://bugzilla.tianocore.org/show_bug.cgi?id=2495

Update the WindowsVsToolChain plugin to setup the VS2017
or VS2019 development environment.  This is required to
build BaseTools and Structured PCD host applications.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Sean Brogan d14feb6cb7 BaseTools/WindowsVsToolChain: Clean up Python source formatting
https://bugzilla.tianocore.org/show_bug.cgi?id=2495

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Kinney, Michael D 45b0be3840 BaseTools/Build: Do not use Common.lib in Structured PCD app
https://bugzilla.tianocore.org/show_bug.cgi?id=2496

Reduce the build and env dependencies for the Structured PCD
application by removing the dependency on Common.lib that
is only built when BaseTools is built which does not
happen if pre-compiled BaseToools are used.  Change the
makefile for the Structure PCD application to build all
files from sources which adds PcdValueCommon.c to the
makefile.  Also remove PcdValueCommon.c from Common.lib.

With the change to the makefile for the Structured PCD
application, multiple C files are compiled.  Only
PcdValueInit.c contains the extra information expected
by the error/warning message parser.  Only parse the
DSC line number into an error message if there is an
error/warning in PcdValueInit.c.  Errors/warnings in
other files should be passed through.  This fixes a build
failure with no useful log information that was observed
when there was a compiler error in PcdValueCommon.c.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00