Commit Graph

2959 Commits

Author SHA1 Message Date
PierreGondois 056b4bf74b BaseTools/Scripts/PatchCheck.py: Check for Change-id
Code review tools like gerrit might use a 'Change-id' tag to track
the evolution of patches. This tag should be removed before
submitting a patch to the mailing-list.
It has been observed that contributors sometimes forget to remove
this tag. Add a check in PatchCheck.py to automate this.

Also add a '--ignore-change-id' command line parameter to ignore
the above check.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Pedro Falcato <pedro.falcato@gmail.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-04 13:29:15 +00:00
levi.yun 40a45b5a2b Basetools: Include PCD declarations from Library Instance
The patch "[PATCH v3 1/2] StandaloneMmPkg: Make StandaloneMmCpu driver
architecture independent" (https://edk2.groups.io/g/devel/message/109178)
removed ArmPkg/ArmPkg.dec from the Packages section in the
INF file: StandaloneMmPkg/Drivers/StandaloneMmCpu/StandaloneMmCpu.inf

This change was done as part of making the StandaloneMmCpu driver
architecture independent.

Although this change is correct, it results in a side effect
here some platforms that utilise PCDs declared in ArmPkg.dec are
no longer declared.

An example of this issue can be seen when building
edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc

$ build -a AARCH64 -t GCC -p Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc
build.py...
/mnt/source/edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.fdf(23):
  error F001: PCD (gArmTokenSpaceGuid.PcdFdBaseAddress) used in
  FDF is not declared in DEC files.

As seen above, removing ArmPkg.dec from the Packages section in the
StandAloneMmCpu Driver Inf file triggers build failure.
Although, ArmPkg.dec is included in other Library Instances,
the build system does not include the declarations from
.dec files defined in Library instances.

The build system only includes the PCD declarations from DEC files
that are specified in INF files for Modules (components).

Therefore, extend the build system to include the Packages from
Library Instances so that the PCD declarations from the respective package
DEC files are included.

This patch can be seen on
    https://github.com/LeviYeoReum/edk2/tree/levi/2848_dec_check_on_library

Signed-off-by: levi.yun <yeoreum.yun@arm.com>
Tested-by: Pierre Gondois <pierre.gondois@arm.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Sami Mujawar <sami.mujawar@arm.com>
2024-01-31 10:58:05 +00:00
Joey Vagedes 6c488a2f39 BaseTools: Fix raw strings containing valid escape characters
Fixes raw regex strings that contain valid (and purposeful) escape
characters as they are being treated as individual characters rather
than the single escaped character they represent (i.e. '\t' is being
treated as a '\' and a 't' rather than a single tab character).

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-01-10 13:54:01 +00:00
Jake Garver 5d533bbc27 BaseTools/GenFw: Correct offset when relocating an ADR
When converting ELF to PE/COFF for the AArch64 target, we may encounter
an R_AARCH64_ADR_GOT_PAGE relocation that refers to an ADR instruction
instead of an ADRP instruction. This can happen when the toolchain is
working around Cortex-A53 erratum #843419.  If that's the case, be sure
to calculate the offset appropriately.

This resolves an issue experienced when building a StandaloneMm image
(which is built with -fpie) with stack protection enabled on GCC
compiled with "--enable-fix-cortex-a53-843419". In this case, the linker
may convert an ADRP instruction appearing at an offset of 0xff8 or 0xffc
modulo 4KiB into an ADR instruction, but will leave the original
R_AARCH64_ADR_GOT_PAGE relocation in place. (This is not a bug in the
linker, given that there is no other relocation type that it could
reasonably convert it into)

In this scenario, the following code is being generated by the
toolchain:

    # Load to set the stack canary
    2ffc:	10028020 	adr	x0, 8000 <mErrorString+0x1bc>
    3008:	f940d400 	ldr	x0, [x0, #424]

    # Load to check the stack canary
    30cc:	b0000020 	adrp	x0, 8000 <mErrorString+0x1bc>
    30d0:	f940d400 	ldr	x0, [x0, #424]

GenFw rewrote that to:

    # Load to set the stack canary
    2ffc:	10000480 	adr	x0, 0x308c
    3008:	912ec000 	add	x0, x0, #0xbb0

    # Load to check the stack canary
    30cc:	f0000460 	adrp	x0, 0x92000
    30d0:	912ec000 	add	x0, x0, #0xbb0

Note that we're now setting the stack canary from the wrong address,
resulting in an erroneous stack fault.

After this fix, the offset will be calculated correctly for an ADR and
the stack canary is set correctly. Note that there is a corner case
where this may cause the conversion to fail: if the original GOT entry
is just within -/+ 1 MiB of the reference, but the actual variable it
refers to is not, the resulting offset cannot be represented by the
immediate offset field in a ADR instruction. Given that this issue only
affects PIE executables, which are rare and usually tiny, this is
unlikely to cause problems in practice.

Ref: https://edk2.groups.io/g/devel/topic/102202314

[ardb: expand commit log, add reference]

Signed-off-by: Jake Garver <jake@nvidia.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 10:08:47 +00:00
Joey Vagedes via groups.io 9f0061a03b BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 00:33:31 +00:00
Rebecca Cran 89705ad6c6 BaseTools: FMMT GuidTool Auto Select Config file Enabling
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4624

Currently, Python FMMT tool does not support automatically
select FMMTConf.ini file which saves GuidTool settings.
This patch supports this features.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-20 22:59:03 +00:00
Yuwei Chen 5b5481526f BaseTools: fixing FMMT ShrinkFv issue
1. FvLength not change issue;
2. FileSystemGuid align with File Size;

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen 59f0d350a9 BaseTools: FMMT support ELF UPLD parser
FMMT add new function to support the .elf file parsing.
Using '-v' option, the UPLD info will be printed out.

'''
- UNIVERSAL_PAYLOAD_INFO
  - 4 bytes align (BOOLEAN)
    - Identifier
    - SpecRevision
    - Attribute
    - Revision
    - Capability
    - ProducerId
    - ImageId
UPLD Buffer
'''

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen 9627447625 BaseTools: FMMT replace output file is not generated successfully
For replace function, when target Ffs and new ffs are with
same size, the output file can not be generated successfully.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen b5f5106c1e BaseTools: FMMT replace new free space fixing in replace
In FMMT replace function, when newffs size <= targetffs size,
the new free space is calculated wrong as loss the pad data delta size.
That will cause invalid binary generated.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Michael Kubacki 4f99b5fb93 BaseTools/Conf/target.template: Use VS2019 as default tool chain
Updates the default tool chain from VS2015x86 to VS2019.

This is the VS tool chain used in CI and more likely to be installed
on developer's systems. This is used in stuart commands when a
toolchain is not explicitly specified.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2023-11-29 20:44:12 +00:00
Michael D Kinney 33deaa3b84 BaseTools/Scripts/GetMaintainer: Sort output addresses
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

Sort the list of output addresses alphabetically so this
script produces the same output even if the order of patches
in a patch series is modified such that that order of files
processed by this script changes.

Use set() logic instead of OrderedDict to accumulate the
list of unique addresses that are sorted alphabetically.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00
Michael D Kinney 706811819d BaseTools/Scripts/GetMaintainer: Handle reviewer only case
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

If a package only has reviewers and no maintainers, then also
return the <default> maintainers.

In order to detect this case, get_maintainers() is updated to
return maintainers, reviews, and lists separately instead of
a single merged list.  This also allows this module to be used
by other scripts that need to distinguish between maintainers,
reviewers, and lists.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00
Leif Lindholm 05f3c3f3d0 BaseTools/Scripts/GetMaintainer: refactor internal returns as dicts
To clean up interfaces, change the lookup functions to return dictionaries
rather than multiple values.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-11-11 02:31:13 +00:00
Michael D Kinney 1cb580be85 BaseTools/Scripts/GetMaintainer: Simplify logic
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

get_section_maintainers() either returns a list with
valid entries or an empty list.  It never returns None.
Simplify logic that accumulates maintainers and lists by
unconditionally appending lists returned from
get_section_maintainers().

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00
Michael D Kinney 431ead235f BaseTools/Scripts/GetMaintainer: Fix logic bug collecting maintainers
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

Fix logic bug where maintainers is incorrectly added to lists.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00
Yeping Song e077ccff6d BaseTools/tools_def: drop -mgeneral-regs-only for AArch64 CLANGDWARF
Commit 0df6c8c157 ("BaseTools/tools_def AARCH64:
avoid SIMD registers in XIP code")
adds -mgeneral-regs-only to GCC_AARCH64_CC_XIPFLAGS,
in order to avoid a bug present in certain versions of GCC.
This was never a problem for clang.
That's given the history of what the problem is.
Then we can describe how we fix it:
Change *_CLANGDWARF_AARCH64_CC_XIPFLAGS to set the required -mstrict-align
option instead of importing the whole GCC variable.

Signed-off-by: Yeping Song <quic_yepings@quicinc.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-09 19:04:39 +00:00
Michael Kubacki d461137e22 BaseTools/Plugin/CodeQL: Enable 30 queries
Updates the CodeQL queries opted into by edk2 to a set of queries from
the standard CodeQL query package `codeql/cpp-queries`.

After testing a large number of queries the included set here were
found to be the most useful with the least number of false positives.
Some queries had a number of issues that led to them being placed on
the exclusion list so that they are not considered in the future
without the notes there being taken into account.

General details about queries available in the pack are available here:
https://codeql.github.com/codeql-query-help/cpp/

The issues found by these queries will need to be fixed over time. In
the meantime, the results will show to those that have permission in
the repo's GitHub Code Scanning area. The build will not fail due to
CodeQL issues (since they are not all fixed) but that can be enabled in
the future.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Sean Brogan <sean.brogan@microsoft.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-11-07 03:19:26 +00:00
Michael Kubacki b531ca4bb3 BaseTools/Plugin/CodeQL: Add integration helpers
Adds a Python module to the CodeQL plugin directory that exports
functions commonly needed for Stuart-based platforms to easily
enable CodeQL in their platform build.

This functionality has already moved to edk2-pytool-extensions
https://github.com/tianocore/edk2-pytool-extensions in the
`edk2toolext/codeql.py` file but edk2 is too far behind to use that.

Additional integration changes are needed in edk2 and the series
to add those has not made it past review. In the meantime, the
functions are available locally in this commit and this commit can
be reverted after edk2-pytool-extensions 0.24.1 or greater is used
in edk2.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Sean Brogan <sean.brogan@microsoft.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-11-07 03:19:26 +00:00
Michael Kubacki 5464d0bed6 BaseTools/Plugin/CodeQL: Add CodeQL build plugin
Adds a CodeQL plugin that supports CodeQL in the build system.

1. CodeQlBuildPlugin - Generates a CodeQL database for a given build.
2. CodeQlAnalyzePlugin - Analyzes a CodeQL database and interprets
   results.
3. External dependencies - Assist with downloading the CodeQL CLI and
   making it available to the CodeQL plugins.
4. CodeQlQueries.qls - A C/C++ CodeQL query set run against the code.
5. Readme.md - A comprehensive readme file to help:
   - Platform integrators understand how to configure the plugin
   - Developers understand how to modify the plugin
   - Users understand how to use the plugin

Read Readme.md for additional details.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Sean Brogan <sean.brogan@microsoft.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-11-07 03:19:26 +00:00
Joey Vagedes da21991953 BaseTools: GenFw: auto-set nxcompat flag
Automatically set the nxcompat flag in the DLL Characteristics field of
the Optional Header of the PE32+ image. For this flag to be set
automatically, the section alignment must be evenly divisible
by 4K (EFI_PAGE_SIZE) and no section must be executable and writable.

Adds a command line flag to GenFw, --nonxcompat, to ensure the
IMAGE_DLLCHARACTERISTICS_NX_COMPAT bit is not set, even if all
requirements are met. Updates the manual for GenFw to include the new
flag.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
Acked-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-11-06 21:44:34 +00:00
Rebecca Cran ae02d487a3 BaseTools: Update PatchCheck.py to allow whitespace issues in .rtf files
Allow .rtf files created by applications such as Notepad to be committed
as-is without further manual editing by skipping the requirements for
CRLF, no tabs and no trailing whitespace.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Acked-by: Laszlo Ersek <lersek@redhat.com>
2023-11-06 20:25:57 +00:00
Joey Vagedes 7806713f00 BaseTools: Plugin: Integration of edk2-pytools
Performs Integration instructions necessary to upgrade edk2-pytool-library
to 0.19.3 and edk2-pytool-extensions to 0.25.1. This includes resolving the
deprecation of builder.mws and replacing it with builder.edk2path
functionality.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
Message-Id: <20231027151551.1043941-3-joeyvagedes@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@os.amperecomputing.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
2023-10-28 14:59:29 +00:00
Yuwei Chen b75d9f556d BaseTools: trim warning to error
As the error is changed to warning, Trim.py will skip the build
error when the source code have exactly issue.
This patch change warning to error to opens the checking.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-10-18 18:17:19 +00:00
Dongyan Qian c70d914428 BaseTools/GenFw: Add support for LOONGARCH64 relax relocation
Correct relax id from 99 to 100 and added relocation support up to 109

fix gcc14 adds new relocation, and the generated relocation
causes the build and compilation to fail.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4559

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Chao Li <lichao@loongson.cn>
Signed-off-by: Dongyan Qian <qiandongyan@loongson.cn>
Reviewed-by: Chao Li <lichao@loongson.cn>
2023-09-25 08:32:47 +00:00
Michael Kubacki cbcf0428e8 BaseTools/Plugin: Add DebugMacroCheck
Adds a plugin that finds debug macro formatting issues. These errors
often creep into debug prints in error conditions not frequently
executed and make debug more difficult when they are encountered.

The code can be as a standalone script which is useful to find
problems in a large codebase that has not been checked before or as
a build plugin that notifies a developer of an error right away.

The script was already used to find numerous issues in edk2 in the
past so there's not many code fixes in this change. More details
are available in the readme file:

.pytool\Plugin\DebugMacroCheck\Readme.md

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-09-19 01:20:27 +00:00
Zhiguang Liu 392456240a BaseTools: Remove logic to create AP waking vector in GenFv
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4494

Today for SEC core(not VTF-0), GenFv finds free 4K aligned space in
FV for AP waking vector and JMP to 4G-30h in the waking vector.
There is no usage of this today. Remove the logic to avoid confusing
and save spaces in reset vector.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
2023-09-18 02:39:25 +00:00
Leif Lindholm 14bf2cb411 BaseTools: drop tautological warning overrides for CLANGDWARF
The CLANGDWARF profile sets both -Wno-tautological-compare and
-Wno-tautological-constant-out-of-range-compare, but this prevents
compile-time detection of certain errors.
Drop these flags.

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-08-29 12:24:01 +00:00
Joey Vagedes fa789cc68a BaseTools: scan Edk2ToolsBuild.py make output
Adds edk2_logging.scan_compiler_output() to Edk2ToolsBuild.py to catch
some compilation errors and log them as an error.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>

Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-08-02 15:20:47 +00:00
Joey Vagedes 5cadb8ce21 BaseTools: BinToPcd: Resolve xdrlib deprecation
Removes the dependency on xdrlib and replaces it with custom logic to
pack a per the xdr requirements. Necessary as xdrlib is being deprecated
in python 3.13.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Joey Vagedes <joeyvagedes@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-08-02 05:23:00 +00:00
Sunil V L 3f49462558 BaseTools/tools_def: Add CLANGDWARF support for RISC-V
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4478

Add tools_def definitions to support CLANGDWARF toolchain
for RISC-V. This uses clang and the llvm LLD linker. This
helps people by not requiring to install multiple
cross compilers for different architectures.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>

Signed-off-by: Sunil V L <sunilvl@ventanamicro.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Ard Biesheuvel <ardb@kernel.org> # Debian clang version 14.0.6
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-07-31 12:37:12 +00:00
Sunil V L 1a39bdf2c5 BaseTools/tools_def: Add "-fno-unwind-tables" to GCC5_RISCV64_CC_FLAGS
gcc-13 for RISC-V enables unwind tables by default similar to ARM64.
This generates .eh_frame_hdr section which is not handled well by
GenFw causing failures.

Disable the unwind tables by adding -fno-unwind-tables flag similar
to [1].

[1] - https://github.com/tianocore/edk2/commit/cbf00651eda6

Signed-off-by: Sunil V L <sunilvl@ventanamicro.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Andrei Warkentin <andrei.warkentin@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-27 18:20:06 +00:00
Gerd Hoffmann 1df6658bcb BaseTools: remove duplicate includes: IndustryStandard/*.h
Use the MdePkg versions instead of maintaining a copy in BaseTools.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann 6101bd125a BaseTools: remove duplicate includes: IndustryStandard/PeImage.h
Use the MdePkg version instead of maintaining a copy in BaseTools.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Chao Li <lichao@loongson.cn>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann c15941a643 BaseTools: switch from EFI_IMAGE_MACHINE_* to IMAGE_FILE_MACHINE_*
Use the newer versions of the machine #defines.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Chao Li <lichao@loongson.cn>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann fdc6288f81 BaseTools: drop IMAGE_FILE_MACHINE_ARM hacks
The #define for IMAGE_FILE_MACHINE_ARM is not present in MdePkg,
this looks like a relic not used any more.  Remove.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann 26d6ef38e7 BaseTools: remove duplicate includes: IndustryStandard/Acpi*.h
Use the MdePkg version instead of maintaining a copy in BaseTools.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Sunil V L <sunilvl@ventanamicro.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann 69ac88cabd BaseTools: remove duplicate includes: <arch>/ProcessorBind.h
Use the MdePkg version instead of maintaining a copy in BaseTools.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Chao Li <lichao@loongson.cn>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann 6a07d9cacf BaseTools: remove WinNtInclude.h
Appears to be a relic for ancient windows / compiler versions,
windows builds in CI work just fine without it.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann bd035a066f BaseTools: add -fdirect-access-external-data to clang pie builds
Tell clang to not use external (via got) references for data access.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Liming Gao <gaoliming@byosoft.com.cn>
2023-06-01 10:53:35 +00:00
Gerd Hoffmann 7ffc433c2e BaseTools: make sure the got is empty
Add ASSERT()s that .got and .got.plt sections are empty
to linker scripts for gcc and clang.

Suggested-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-06-01 10:53:35 +00:00
Gua Guo 0f8323b44b BaseTools: toolsetup.bat always execute PYTHON_HOME
Ideally behavior is like below order that can support one local build
machine, clone multiple Edk2, some of edk2 repo use old tag and
some of edk2 repo use new tag, they can both support on one machine.

1. if defined PYTHON_COMMAND only
   - use PYTHON_COMMAND = user assigned
2. if not defined PYTHON_COMMAND, auto detect py -3
   - use PYTHON_COMMAND = py -3
3. if defined PYTHON_COMMAND and PYTHON_HOME, use PYTHON_COMMAND
   - use PYTHON_COMMAND = user assigned
4. if defined PYTHON_HOME only,
   - use PYTHON_COMMAND = %PYTHON_HOME%/python.exe

SCRIPT_ERROR should return for paraent batch file to consume
for error handle.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-05-29 01:34:28 +00:00
Ard Biesheuvel d322557712 BaseTools/tools_def: Disable overzealous unused variable warning on Clang
The warnings Clang emits when enabling -Wunneeded-internal-declaration
(which is part of -Wall) are generating false positives for variables
whose size gets taken but are not referenced beyond yet.

This may happen legitimately in debug code, so let's disable this
warning for Clang, rather than tiptoe around it in the code.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
2023-05-12 13:12:35 +00:00
Ard Biesheuvel e2607d3a78 BaseTools/tools_def: Drop ref to undefined CLANGDWARF_ARM_PREFIX
When using CLANGDWARF to build for the ARM architecture, objcopy is
references via the wrong environment variable, resulting in the wrong
llvm-objcopy to be used (if one exists), or the build to fail (if
CLANGDWARF_BIN points to the only available instance)

So use CLANGDWARF_BIN instead, which was what was intended.

Fixes: ecbc394365 ("BaseTools: Set CLANGDWARF RC path ...")
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-05-12 13:12:35 +00:00
Gua Guo 6dd64168ed BaseTools/Plugin: Too many execute files cause "cmd too long" failure
Windows command prompt have 8191 characters limitation,
enhance it to make command too long can be resloved.

Provide an example, if have too many cov files, it causes to run single
command over the 8191 characters limitation.
> OpenCppCoverage
>  --export_type binary:coverage.cov
>  --working_dir={workspace}Build
>  --input_coverage=AAA.cov
>  ...
>  --input_coverage=NNN.cov

The solution is passing many coverage files in single command line to
breaking it up into many command lines with one coverage file per
command line in order to prevent single line is over to 8191 characters.

- Command Line 1
> OpenCppCoverage
>  --export_type binary:coverage.cov
>  --working_dir={workspace}Build
>  --input_coverage=AAA.cov
>  --input_coverage=coverage.cov
...

- Command Line N
> OpenCppCoverage
>  --export_type. binary:coverage.cov
>  --working_dir={workspace}Build
>  --input_coverage=NNN.cov
>  --input_coverage=coverage.cov

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Michael Kubacki <mikuback@linux.microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2023-05-11 05:45:51 +00:00
Rebecca Cran 77f75c7fb8 BaseTools: Update Tests/TestTools.py to allow it to work on Windows
On Windows, executables have a '.exe' suffix which needs to be added for
them to be found in a path.

Also, files need to be explicitly opened as binary.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
2023-05-11 02:14:12 +00:00
Rebecca Cran b9bbb4ae93 BaseTools: only print the environment once in toolsetup.bat
Avoid printing %PATH% twice: move the printing of the environment down
to print_python_info.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
2023-05-11 02:14:12 +00:00
Rebecca Cran dd246227d6 BaseTools: Update toolsetup.bat to not use BASETOOLS_PYTHON_SOURCE
The BASETOOLS_PYTHON_SOURCE environment variable is only used temporarily to
set PYTHONPATH. Since it doesn't help improve clarity, remove it.

While here, make sure we set PYTHONPATH when we're using Pip BaseTools
so that build etc. can be found.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
2023-05-11 02:14:12 +00:00
Rebecca Cran f47415e031 BaseTools: Revert Set the CLANGDWARF OBJCOPY path in tools_def.template
This reverts commit 11f62f4cc0.

While GCC uses objcopy for the OBJCOPY command, it's not needed for the
CLANGDWARF toolchain and can be left as echo.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Ard BIesheuvel <ardb@kernel.org>
2023-05-11 00:37:05 +00:00
Rebecca Cran e6447d2a08 Remove bashisms from edksetup.sh and BaseTools/BuildEnv
Remove bashisms from edksetup.sh and BaseTools/BuildEnv. This allows any
POSIX shell to use those scripts, removing the dependency on bash.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2023-05-10 12:02:34 +00:00