Commit Graph

2997 Commits

Author SHA1 Message Date
Mikhail Krichanov 6e382e8bb0 BaseTools/Conf: Added EDK2_GCC_NOLTO build option. 2024-07-22 13:49:00 +03:00
Mikhail Krichanov f83c7b5c40 Build: Replaced GCC5 toolchain with GCC. 2024-07-22 13:49:00 +03:00
Mikhail Krichanov ecfa20d841 OvmfPkg, BaseTools: Fixed Amaranth compilation for gcc-12.3.0. 2024-07-22 13:47:18 +03:00
Mikhail Krichanov 047d1b2f88 BaseTools: Introduce new HII section design 2024-07-22 13:47:18 +03:00
Mikhail Krichanov c57cc69793 BaseTools/C: Consume UefiImageLib and ImageToolEmit 2024-07-22 13:47:18 +03:00
Marvin Häuser f69039282f BaseTools/GenFv: Remove .text and .data from FV map 2024-07-22 13:47:18 +03:00
Marvin Häuser 4f05d0cc12 BaseTools/build_rule: Drop deprecated ASM16 definition 2024-07-22 13:47:18 +03:00
Mikhail Krichanov 2ae98acd86 Drop support for the TE format 2024-07-22 13:47:18 +03:00
Mikhail Krichanov 76e055e723 BaseTools/CommonLib: Consume MemoryAllocationLib 2024-07-22 13:47:18 +03:00
Marvin Häuser 8632c62d0d BaseTools/GenFvInternalLib: Remove unused BaseOfCode parameter 2024-07-22 13:47:18 +03:00
Marvin Häuser 9482b7ddc9 BaseTools/GenFv: Remove duplicated alignment macros 2024-07-22 13:47:17 +03:00
Marvin Häuser 2df90a3027 BaseTools/DevicePath: Consume MdePkg/UefiDevicePathLib
BaseTools has been duplicating and adapting code that is defined in
MdePkg and MdeModulePkg. This leads to desync issues where the same
symbols may be backed by different functions with slightly different
semantics and also fixes that apply only to BaseTools or only to MdePkg
and MdeModulePkg.

To address these issues, update BaseTools/Source/C to utilize the code
from MdePkg and MdeModulePkg.

Signed-off-by: Marvin Häuser <mhaeuser@posteo.de>
2024-07-22 13:47:17 +03:00
Marvin Häuser 9163cd4925 BaseTools/CommonLib: Consume MdePkg and MdeModulePkg
BaseTools has been duplicating and adapting code that is defined in
MdePkg and MdeModulePkg. This leads to desync issues where the same
symbols may be backed by different functions with slightly different
semantics and also fixes that apply only to BaseTools or only to MdePkg
and MdeModulePkg.

To address these issues, update BaseTools/Source/C to utilize the code
from MdePkg and MdeModulePkg.

Signed-off-by: Marvin Häuser <mhaeuser@posteo.de>
2024-07-22 13:47:17 +03:00
Mike Beaton e19d725d4c GenSec: Support TianoCompress
Allows .fdf files to define a compressed .ffs section which can be read
by legacy EFI firmware (such as found on Apple Mac).
Use `COMPRESS TIANO`.
2024-07-22 13:47:17 +03:00
Mikhail Krichanov a98a96336f Build: Renamed XIPFLAGS as SECPEIFLAGS, added comment in build_rule.template on its usage. 2024-07-22 13:47:17 +03:00
Mikhail Krichanov ba869df56a BaseTools: Removed GCC48/49 toolchains, refactored tools_def.template. 2024-07-22 13:47:17 +03:00
Mikhail Krichanov 12ba277a2d BaseTools, MdeModulePkg: Update brotli submodule
Update the brotli submodule to the latest commit (ed1995b6bda1)
so that the build isn't broken in GCC 12 compilers.

Signed-off-by: Savva Mitrofanov <savvamtr@gmail.com>
2024-07-22 13:47:17 +03:00
Mikhail Krichanov 7b10859b36 BaseTools: Replaced GenFw with ImageTool and MicroTool. 2024-07-22 13:45:44 +03:00
Mikhail Krichanov a955e8811b SecurePE: Replaced old PE loader with Secure one. 2024-07-22 13:42:14 +03:00
Mikhail Krichanov 443162427a Build: Fixed missing HII for X64, IA32 CLANGDWARF DEBUG, NOOPT. 2024-07-22 12:55:02 +03:00
Vitaly Cheptsov ad7e33255b BaseTools: Allow .text relocations in CLANGDWARF toolchain
UefiCpuPkg modules like ExceptionHandlerAsm.nasm require relocations
to .text or PIC as they use instructions like mov or bt against
external symbols:

1. mov     rax, ASM_PFX(CommonInterruptEntry)
2. mov     rax, HookAfterStubHeaderEnd
3. bt      [ASM_PFX(mErrorCodeFlag)], ecx
4. mov     [rcx + (@VectorNum - HookAfterStubHeaderBegin)], al

It is not easily possible to rewrite 3/4 cases without involving
more registers or more commands, and EDK II has never targeted
read-only .text at load time, only at runtime.

Change the defaults to let OVMF compile by CLANGDWARF as per:
https://lists.llvm.org/pipermail/llvm-dev/2017-March/111441.html

Signed-off-by: Vitaly Cheptsov <cheptsov@ispras.ru>
2024-07-22 12:45:14 +03:00
Michael Kubacki 7142e64841 CodeQL: Update from 2.16.1 to 2.17.3
This fixes an issue where the CodeQL queries currently fetched in the
pipeline are incompatible with the current executable used.

Update to pick up functional and security fixes. See the following
comparison for detailed differences:

https://github.com/github/codeql-cli-binaries/compare/v2.16.1...v2.17.3

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2024-05-21 02:24:08 +00:00
Gua Guo c0dfe3ec1f BaseTools/GetUtcDateTime.py: Python 3.12 support
Ref to https://docs.python.org/3/whatsnew/3.12.html
utcnow() and utcfromtimestamp() are deprecated
Prevent use it cause build error.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-26 03:01:55 +00:00
Michael Kubacki 90b6725562 Update to CodeQL 2.16.1
Updates CodeQL to work with the latest queries. Includes functional
and security fixes within the CodeQL CLI binary.

For more information on release details see:

https://github.com/github/codeql-cli-binaries/releases

For changes between the previous version (2.14.5) and 2.16.1 see:

https://github.com/github/codeql-cli-binaries/compare/v2.14.5...v2.16.1

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Joey Vagedes <joey.vagedes@gmail.com>
2024-04-24 23:11:19 +00:00
Gua Guo d97f964f7c BaseTools/Fmmt.py: Python 3.12 support
Ref to https://docs.python.org/3/whatsnew/3.12.html
A backslash-character pair that is not a valid
escape sequence now generates

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-23 21:36:22 +00:00
Taylor Beebe 5ba3602e45 BaseTools: Use Stronger Matching for NULL Linked Libraries
To prevent the possibility that a library with a name like
NULLTestLib is interpreted as a NULL linked library, use
more explicit pattern matching to ensure that the library
name follows the pattern NULL%d.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-15 19:04:08 +00:00
Taylor Beebe d77efa2ebe BaseTools: Don't Recurse NULL Includes Not Linked to Module
When collecting the required library instances for modules and
libraries, included libraries will be recursed to ensure the module is
built with all the libraries directly linked to it and indirectly
linked to it via included libraries.

Using the following scenario as an example:

[LibraryClasses.common.DXE_CORE]
NULL|Path/To/Library1.inf // Includes DebugLib

[LibraryClasses.common.DXE_DRIVER]
NULL|Path/To/Library2.inf // Includes DebugLib

[LibraryClasses.common.DXE_CORE, LibraryClasses.common.DXE_DRIVER]
DebugLib|MdePkg/Library/BaseDebugLibSerialPort/BaseDebugLibSerialPort.inf

[Components]
MdeModulePkg/Core/Dxe/DxeMain.inf // Includes DebugLib

The DXE_CORE NULL library will be assigned a fake library class like
NULL1 and the DXE_DRIVER will be assigned NULL2. The recursion logic
will see NULL1 as a directly linked and will add an instance of it to
the list of libraries which need to be included in the module. When
DebugLib is evaluated, the recursion logic will add the libraries
DebugLib depends on to the queue which includes both NULL1 and NULL2.
When NULL2 is unqueued, an instance of it will also be added to the
list of libraries needed to build DxeMain which now means that both
NULL1 and NULL2 have been linked.

NULL includes outside of module overrides are not supported according
to the spec, but we do it anyways so this seems like a case which
should be fixed. This change updates the recursion logic to skip
evaluating NULL libraries unless they are linked directly to the
module/library being evaluated.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-15 19:04:08 +00:00
Igniculus Fu 5572b43c67 BaseTools/GenFds: Apply OEM_CAPSULE_FLAGS during Capsule generation.
Bugzilla ticket 4633

FdfParser.py has defined a key named OEM_CAPSULE_FLAGS to set the
lower 16 bits of EFI_CAPSULE_HEADER.Flags. However, this key is totally
"forgotten" in Capsule.py, making it impossible to set lower 16 bits of
this field, and leading to an always FALSE when comparing to
gEfiMdeModulePkgTokenSpaceGuid.PcdSystemRebootAfterCapsuleProcessFlag
in MdeModulePkg/Library/DxeCapsuleLibFmp/DxeCapsuleProcessLib.c:
ProcessTheseCapsules().

This patch ORs the value of OEM_CAPSULE_FLAGS with previously calculated
CAPSULE_FLAGS value, making the lower 16 bits of value being correctly set.

Signed-off-by: Igniculus Fu <igniculus.fu@amd.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Abner Chang <abner.chang@amd.com>
Cc: Eric Xing <eric.xing@amd.com>
Cc: Abdul Lateef Attar <abdattar@amd.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-03-13 10:53:30 +00:00
Michael Kubacki bff9815b61 BaseTools/GenFds: Resolve absolute workspace INF paths
Currently, if an INF path is an absolute path on Linux (begins with
"/"), the "/" character will be removed. If the path is an absolute
system path, this creates an invalid path.

An example of when this may be an issue is in external dependencies
where an INF is within the external dependency, the `set_build_var`
flag is set, and DSC files refer to files by its build variable
(e.g. `$(SHARED_BINARIES)/Module.inf`). INFs in a binary distribution
like this example may contain a [Binaries] section and refer to
different section files that can be used by a platform to compose an
FFS file. For example, the PE32 (.efi) and DEPEX (.depex) files.

In this case, `$(SHARED_BINARIES)` will be an absolute path to the
ext dep directory and `FfsInfStatement.__InfParse__` will remove the
leading "/" character so the path is invalid.

This change first checks if the absolute path will resolve into the
current workspace. If it does (as will happen in the shared crypto
ext dep example above), it modifies the path to be relative to the
workspace so later logic dependent on relative paths can operate on
it. If the absolute path is not within the current workspace, it
follows previous behavior for backward compatibility to that
scenario.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-03-04 16:23:58 +00:00
Laszlo Ersek bac9c74080 BaseTools/AutoGen: declare ProcessLibraryConstructorList() for SEC modules
Most module types have standardized entry point function prototypes. They
are declared in headers like

- MdePkg/Include/Library/PeiCoreEntryPoint.h
- MdePkg/Include/Library/PeimEntryPoint.h
- MdePkg/Include/Library/DxeCoreEntryPoint.h
- MdePkg/Include/Library/UefiDriverEntryPoint.h
- MdePkg/Include/Library/UefiApplicationEntryPoint.h

These header files also declare matching ProcessLibraryConstructorList()
prototypes.

The SEC module type does not have a standardized entry point prototype
(aka parameter list), therefore no header file like the above ones exists
for SEC. Consequently, no header file *declares*
ProcessLibraryConstructorList() for SEC modules, even though AutoGen
always *defines* ProcessLibraryConstructorList() with the same, empty,
parameter list (i.e., just (VOID)).

The lack of a central declaration is a problem because in SEC code,
ProcessLibraryConstructorList() needs to be called manually, and those
calls need a prototype. Most SEC modules in edk2 get around this by
declaring ProcessLibraryConstructorList() manually, while some others use
an incorrect (PEIM) prototype.

Liming suggested in
<https://bugzilla.tianocore.org/show_bug.cgi?id=991#c2> that AutoGen
provide the declaration as well; implement that in this patch.

Mike suggested that the feature be gated with INF_VERSION, for
compatibility reasons. (INF_VERSION >= 1.30) reflects that the latest
(draft) version of the INF specification, as of this writing, is commit
a31e3c842bee / version 1.29.

For example, if we modify "OvmfPkg/Sec/SecMain.inf" as follows:

> diff --git a/OvmfPkg/Sec/SecMain.inf b/OvmfPkg/Sec/SecMain.inf
> index 3c47a664a95d..dca932a474ee 100644
> --- a/OvmfPkg/Sec/SecMain.inf
> +++ b/OvmfPkg/Sec/SecMain.inf
> @@ -8,7 +8,7 @@
>  ##
>
>  [Defines]
> -  INF_VERSION                    = 0x00010005
> +  INF_VERSION                    = 1.30
>    BASE_NAME                      = SecMain
>    FILE_GUID                      = df1ccef6-f301-4a63-9661-fc6030dcc880
>    MODULE_TYPE                    = SEC

then the patch produces the following difference in
"Build/OvmfX64/NOOPT_GCC5/X64/OvmfPkg/Sec/SecMain/DEBUG/AutoGen.h":

> --- AutoGen.h.orig      2024-02-06 23:10:23.469535345 +0100
> +++ AutoGen.h   2024-02-07 00:00:57.361294055 +0100
> @@ -220,6 +220,13 @@
>
>  // Definition of PCDs used in libraries is in AutoGen.c
>
> +// ProcessLibraryConstructorList() declared here because SEC has no standard entry point.
> +VOID
> +EFIAPI
> +ProcessLibraryConstructorList (
> +  VOID
> +  );
> +
>
>  #ifdef __cplusplus
>  }

which presently (as of edk2 commit edc6681206) triggers the following
build error:

> In file included from OvmfPkg/Sec/SecMain.c:14:
> MdePkg/Include/Library/PeimEntryPoint.h:74:1: error: conflicting types for
> ‘ProcessLibraryConstructorList’; have ‘void(void *, const
> EFI_PEI_SERVICES **)’ {aka ‘void(void *, const struct _EFI_PEI_SERVICES
> **)’}
>    74 | ProcessLibraryConstructorList (
>       | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> In file included from <command-line>:
> Build/OvmfX64/NOOPT_GCC5/X64/OvmfPkg/Sec/SecMain/DEBUG/AutoGen.h:226:1: note:
> previous declaration of ‘ProcessLibraryConstructorList’ with type
> ‘void(void)’
>   226 | ProcessLibraryConstructorList (
>       | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

That's a genuine bug in OvmfPkg that needs to be fixed, but we keep
compatibility with existent SEC modules until/unless they upgrade
INF_VERSION to 1.30+.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=991
Suggested-by: Liming Gao <gaoliming@byosoft.com.cn>
Suggested-by: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20240224210504.41873-1-lersek@redhat.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-02-29 09:56:38 +00:00
Jayaprakash, N 3e91e42136 BaseTools: Syntax warning invalid escape sequence \C
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4666

This commit fixes the issue reported through BZ4666.
The Syntax warning related to invalid escape sequence
for \C is seen on Windows OS based builds of edk2 sources.
On Windows the path seperator needs to prefixed with \
so essentially we need to use \\ as path seperator.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Jayaprakash N <n.jayaprakash@intel.com>
Signed-off-by: Jayaprakash N <n.jayaprakash@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-28 00:23:58 +00:00
Michael D Kinney 6d571c0070 BaseTools/Scripts/PatchCheck: Error if commit modifies multiple packages
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4679

Update PatchCheck.py to evaluate all the files modified in each commit
and generate an error if:
* A commit adds/modifies files in multiple package directories
* A commit adds/modifies files in multiple non-package directories
* A commit adds/modifies files in both a package and a non-package
  directory
* A commit deletes files from multiple package directories
* A commit deletes files from multiple non-package directories
* A commit deletes files from both a package and a non-package
  directory

Modifications to files in the root of the repository are not
evaluated.

This check is skipped if PatchCheck.py is run on a patch file or
input from stdin because this multiple package commit check depends
on information from a git repository.

If --ignore-multi-package option is set, then reduce the multiple
package commit check from an error to a warning for all commits in
the commit range provided to PatchCheck.py.

Add check for a 'Continuous-integration-options:' commit message
tag that allows one or more options to be specified at the individual
commit scope to enable/disable continuous integration checks. This
tag must start at the beginning of a commit message line and may
appear more than once in a commit message.

Add support for a Continuous-integration-options tag value of
'PatchCheck.ignore-multi-package' that reduces the multiple package
commit check from an error to a warning for the specific commits that
specify this option.  Example:

  Continuous-integration-options: PatchCheck.ignore-multi-package

The set of packages are found by searching for DEC files in a git
repository. The list of DEC files in a git repository is collected
with the following git command:

  git ls-files *.dec

The set of files added/modified by each commit is found using the
following git command:

  git diff-tree --no-commit-id --name-only --diff-filter=AM -r <commit>

The set of files deleted by each commit is found using the
following git command:

  git diff-tree --no-commit-id --name-only --diff-filter=D -r <commit>

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-27 19:28:19 +00:00
Michael D Kinney 0bbec15b54 BaseTools/Scripts/PatchCheck: Error if no Cc tags are present
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4694

If no Cc tags are detected in a commit message, then generate an
error. All patches sent for review are required to provide the set
of maintainers and reviewers responsible for the directories/files
modified. The set of maintainers and reviewers are documented in
Maintainers.txt and can be retrieved using the script
BaseTools/Scripts/GetMaintainer.py.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-27 19:28:19 +00:00
Michael D Kinney 45ad13bb64 BaseTools/Scripts/PatchCheck: Return CommitMessageCheck errors
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4693

Commit signatures are checked and error messages are logged but
errors are not captured and returned from find_signatures() in the
CommitMessageCheck class. This causes signature errors to be
silently ignored by CI.

Update logic in CommitMessageCheck class to return errors
detected in commit message signatures.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-27 19:28:19 +00:00
Michael D Kinney dae8c29dab BaseTools/Scripts/PatchCheck: Update Author checks
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4680

* Reject patches that match Author email "devel@edk2.groups.io"
* Update the current check for " via Groups.Io" to perform a
  case insensitive match. It appears that groups.io has changed the
  format of this string to use all lower case.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
2024-02-27 19:28:19 +00:00
Yi Li 44fdc4f398 BaseTools: Update keybaord map based on UEFI spec 2.10
REF: UEFI SPEC 2.10 34.8.10 EFI_KEY

Add EfiKeyIntl0-9.

Signed-off-by: Yi Li <yi1.li@intel.com>

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-02-26 15:32:09 +00:00
Ashraf Ali S e32b58ab5a BaseTools: Remove Duplicate sets of SkuName and SkuId from allskuset
Currently when the platform has many SKUs then allskuset will be having
so many duplicate. and while parsing the allskuset will take longer
time while assigning Pcd.SkuInfoList.
This patch is to eliminate those duplicate entries to reduce the
build time

Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Amy Chan <amy.chan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2024-02-08 19:27:18 +00:00
devel@edk2.groups.io 8f316e99ec BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs
During the Incremental build GenerateByteArrayValue used to generate the
ByteArrayValue even when there is no change in the PCD/VPDs. which is
time consuming API based on the number of PCD/VPDs and SKU IDs.

The optimization is that GenerateByteArrayValue is used to store the
StructuredPcdsData in a JSON file for each of the arch. and during the
Incremental build this API will check, if there is any change in the
Structured PCD/VPDs then rest of the flow remains the same.
if there is no change then it will return the provious build data.

Flow:
during the 1st build StructuredPcdsData.json is not exists,
StructuredPcdsData will be dumped to json file. and it will copy the
output.txt as well.
Note: as the output.txt are different for different Arch, so it will be
stored in the Arch folder.
During the Incremental build check if there is any change in Structured
PCD/VPD. if there is a change in Structured VPD/PCD then recreate the
StructuredPcdsData.json, and rest of the flow remains same.
if there is no change in VPD/PCD read the output.txt and return the data

Unit Test:
Test1: Modified the Structured Pcds default from DEC file. current flow
is executing.
Test2: Override the default value of the PCD from DEC file. current flow
is executing.
Test3: Modified/Override the PCD from DSC file. current flow executing
Test4: Modified/Override the FDF from DSC file. current flow executing
Test5: update the default value from Command Line.current flow executing
Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the
proposed changes will be executing, and the return data remains the same
with and without the changes.
Test7: Build with and without modified the include headers of Structured
PCDs. if there is any change in those Structured PCD header then
current flow will be executed.

With these changes it's helping to save around ~2.5min to ~3.5min of
Incremental build time in my build environment.

Sample PR: https://github.com/tianocore/edk2-basetools/pull/113

Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Cc: Digant H Solanki <digant.h.solanki@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2024-02-08 04:08:38 +00:00
PierreGondois 056b4bf74b BaseTools/Scripts/PatchCheck.py: Check for Change-id
Code review tools like gerrit might use a 'Change-id' tag to track
the evolution of patches. This tag should be removed before
submitting a patch to the mailing-list.
It has been observed that contributors sometimes forget to remove
this tag. Add a check in PatchCheck.py to automate this.

Also add a '--ignore-change-id' command line parameter to ignore
the above check.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Acked-by: Pedro Falcato <pedro.falcato@gmail.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-04 13:29:15 +00:00
levi.yun 40a45b5a2b Basetools: Include PCD declarations from Library Instance
The patch "[PATCH v3 1/2] StandaloneMmPkg: Make StandaloneMmCpu driver
architecture independent" (https://edk2.groups.io/g/devel/message/109178)
removed ArmPkg/ArmPkg.dec from the Packages section in the
INF file: StandaloneMmPkg/Drivers/StandaloneMmCpu/StandaloneMmCpu.inf

This change was done as part of making the StandaloneMmCpu driver
architecture independent.

Although this change is correct, it results in a side effect
here some platforms that utilise PCDs declared in ArmPkg.dec are
no longer declared.

An example of this issue can be seen when building
edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc

$ build -a AARCH64 -t GCC -p Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc
build.py...
/mnt/source/edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.fdf(23):
  error F001: PCD (gArmTokenSpaceGuid.PcdFdBaseAddress) used in
  FDF is not declared in DEC files.

As seen above, removing ArmPkg.dec from the Packages section in the
StandAloneMmCpu Driver Inf file triggers build failure.
Although, ArmPkg.dec is included in other Library Instances,
the build system does not include the declarations from
.dec files defined in Library instances.

The build system only includes the PCD declarations from DEC files
that are specified in INF files for Modules (components).

Therefore, extend the build system to include the Packages from
Library Instances so that the PCD declarations from the respective package
DEC files are included.

This patch can be seen on
    https://github.com/LeviYeoReum/edk2/tree/levi/2848_dec_check_on_library

Signed-off-by: levi.yun <yeoreum.yun@arm.com>
Tested-by: Pierre Gondois <pierre.gondois@arm.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Sami Mujawar <sami.mujawar@arm.com>
2024-01-31 10:58:05 +00:00
Joey Vagedes 6c488a2f39 BaseTools: Fix raw strings containing valid escape characters
Fixes raw regex strings that contain valid (and purposeful) escape
characters as they are being treated as individual characters rather
than the single escaped character they represent (i.e. '\t' is being
treated as a '\' and a 't' rather than a single tab character).

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-01-10 13:54:01 +00:00
Jake Garver 5d533bbc27 BaseTools/GenFw: Correct offset when relocating an ADR
When converting ELF to PE/COFF for the AArch64 target, we may encounter
an R_AARCH64_ADR_GOT_PAGE relocation that refers to an ADR instruction
instead of an ADRP instruction. This can happen when the toolchain is
working around Cortex-A53 erratum #843419.  If that's the case, be sure
to calculate the offset appropriately.

This resolves an issue experienced when building a StandaloneMm image
(which is built with -fpie) with stack protection enabled on GCC
compiled with "--enable-fix-cortex-a53-843419". In this case, the linker
may convert an ADRP instruction appearing at an offset of 0xff8 or 0xffc
modulo 4KiB into an ADR instruction, but will leave the original
R_AARCH64_ADR_GOT_PAGE relocation in place. (This is not a bug in the
linker, given that there is no other relocation type that it could
reasonably convert it into)

In this scenario, the following code is being generated by the
toolchain:

    # Load to set the stack canary
    2ffc:	10028020 	adr	x0, 8000 <mErrorString+0x1bc>
    3008:	f940d400 	ldr	x0, [x0, #424]

    # Load to check the stack canary
    30cc:	b0000020 	adrp	x0, 8000 <mErrorString+0x1bc>
    30d0:	f940d400 	ldr	x0, [x0, #424]

GenFw rewrote that to:

    # Load to set the stack canary
    2ffc:	10000480 	adr	x0, 0x308c
    3008:	912ec000 	add	x0, x0, #0xbb0

    # Load to check the stack canary
    30cc:	f0000460 	adrp	x0, 0x92000
    30d0:	912ec000 	add	x0, x0, #0xbb0

Note that we're now setting the stack canary from the wrong address,
resulting in an erroneous stack fault.

After this fix, the offset will be calculated correctly for an ADR and
the stack canary is set correctly. Note that there is a corner case
where this may cause the conversion to fail: if the original GOT entry
is just within -/+ 1 MiB of the reference, but the actual variable it
refers to is not, the resulting offset cannot be represented by the
immediate offset field in a ADR instruction. Given that this issue only
affects PIE executables, which are rare and usually tiny, this is
unlikely to cause problems in practice.

Ref: https://edk2.groups.io/g/devel/topic/102202314

[ardb: expand commit log, add reference]

Signed-off-by: Jake Garver <jake@nvidia.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 10:08:47 +00:00
Joey Vagedes via groups.io 9f0061a03b BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 00:33:31 +00:00
Rebecca Cran 89705ad6c6 BaseTools: FMMT GuidTool Auto Select Config file Enabling
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4624

Currently, Python FMMT tool does not support automatically
select FMMTConf.ini file which saves GuidTool settings.
This patch supports this features.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-20 22:59:03 +00:00
Yuwei Chen 5b5481526f BaseTools: fixing FMMT ShrinkFv issue
1. FvLength not change issue;
2. FileSystemGuid align with File Size;

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen 59f0d350a9 BaseTools: FMMT support ELF UPLD parser
FMMT add new function to support the .elf file parsing.
Using '-v' option, the UPLD info will be printed out.

'''
- UNIVERSAL_PAYLOAD_INFO
  - 4 bytes align (BOOLEAN)
    - Identifier
    - SpecRevision
    - Attribute
    - Revision
    - Capability
    - ProducerId
    - ImageId
UPLD Buffer
'''

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen 9627447625 BaseTools: FMMT replace output file is not generated successfully
For replace function, when target Ffs and new ffs are with
same size, the output file can not be generated successfully.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen b5f5106c1e BaseTools: FMMT replace new free space fixing in replace
In FMMT replace function, when newffs size <= targetffs size,
the new free space is calculated wrong as loss the pad data delta size.
That will cause invalid binary generated.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Michael Kubacki 4f99b5fb93 BaseTools/Conf/target.template: Use VS2019 as default tool chain
Updates the default tool chain from VS2015x86 to VS2019.

This is the VS tool chain used in CI and more likely to be installed
on developer's systems. This is used in stuart commands when a
toolchain is not explicitly specified.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2023-11-29 20:44:12 +00:00
Michael D Kinney 33deaa3b84 BaseTools/Scripts/GetMaintainer: Sort output addresses
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4593

Sort the list of output addresses alphabetically so this
script produces the same output even if the order of patches
in a patch series is modified such that that order of files
processed by this script changes.

Use set() logic instead of OrderedDict to accumulate the
list of unique addresses that are sorted alphabetically.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
2023-11-11 02:31:13 +00:00