1322 Commits

Author SHA1 Message Date
Mikhail Krichanov
ba9aad0b56 ImageTool: Rework PeEmit with dynamically-growing buffers 2025-04-07 12:24:27 +03:00
Mikhail Krichanov
7193c4e313 BaseTools: Introduce new HII section design 2025-04-07 12:18:23 +03:00
Marvin Häuser
544c7bdda8 BaseTools/build_rule: Drop deprecated ASM16 definition 2025-04-07 12:18:22 +03:00
Mikhail Krichanov
bc87d7b430 Drop support for the TE format 2025-04-07 12:18:22 +03:00
Mike Beaton
850cf3b628 GenSec: Support TianoCompress
Allows .fdf files to define a compressed .ffs section which can be read
by legacy EFI firmware (such as found on Apple Mac).
Use `COMPRESS TIANO`.
2025-04-07 12:13:58 +03:00
Mikhail Krichanov
5d894921a3 BaseTools: Replaced GenFw with ImageTool and MicroTool. 2025-04-07 12:13:57 +03:00
Mikhail Krichanov
09a0c067d0 SecurePE: Replaced old PE loader with Secure one. 2025-04-07 12:12:35 +03:00
Rebecca Cran
259e1f04c6 BaseTools: Add @verbatim as allowed Doxygen tag
Per the discussion in https://github.com/tianocore/edk2/pull/6476,
update Ecc code to allow the @verbatim Doxygen tag.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
2025-02-04 00:12:00 +00:00
Michael D Kinney
62b0698309 BaseTools/AutoGen: GenMake response file quotes strings
If command line options are moved into a response file
of a GCC family build, then the file path separators are
converted from '\' to '/'. However, this can corrupt
command line options that are quoted strings.

Update GenMake to no convert '\' to '/' in quoted strings.

Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
2025-02-02 19:22:42 +00:00
Ashraf Ali S
6d2143f685 BaseTools: Fix NoneType parent reference in FMMT operations
This patch addresses an issue in the FMMT operations where the parent
reference is not checked for NoneType. This oversight can lead to an
AttributeError: 'NoneType' object has no attribute 'Name' when
attempting to access the parent reference. The fix involves adding a
check for NoneType before accessing the parent reference to ensure that
the operations handle such cases gracefully.

The affected functions include:
- AddNewFfs
- ReplaceFfs
- ExtractFfs

These functions now include proper checks to prevent the AttributeError.

Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
2025-01-27 16:26:18 +00:00
Yang Gang
3ac092cf72 BaseTools: Clean up os.path.normcase and os.path.normpath usage
Refer to the docs of python, `os.path.normcase(path)` function:
"Normalize the case of a pathname. On Windows, convert all characters in
the pathname to lowercase, and also convert forward slashes to backward
slashes. On other operating systems, return the path unchanged."

`os.path.normpath(path)` also convert forward slashes to backward slashes.

So call `os.path.normcase` after `os.path.normpath` just convert path to
lowercase on Windows(only).

And Windows is case-insensitive but case-preserving.

So the usage of `os.path.normcase(os.path.normpath(path))` can be
simplified to `os.path.normpath(path)`. Then we can use case-preserving
paths rather than lowercase paths in compile_commands.json file
or build log.

But this patch continue to use `os.path.normcase`
when comparing/searching paths.

Signed-off-by: Yang Gang <yanggang@byosoft.com.cn>
2025-01-09 07:25:45 +00:00
Ashraf Ali
1cc78814cd Remove dependancy on re.T
Signed-off-by: Ashraf Ali <ashraf.ali.s@intel.com>
2024-12-24 07:32:42 +00:00
Dionna Glaze
74ac8cc0e8 BaseTools: Typo fixes
Some in error messages, some in local variable names.

Signed-off-by: Dionna Glaze <dionnaglaze@google.com>
2024-12-11 09:00:21 +08:00
Rebecca Cran
5b2d55533b BaseTools: Improve error messages from UefiCapsuleHeader.py
Instead of throwing ValueErrors with no explanation, add a message
explaining what went wrong.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
2024-12-06 13:02:18 +00:00
Mike Beaton
6820004b3e BaseTools: Fix multiple 'invalid escape sequence' warnings in tests
In Python 3.12 invalid escape sequences in strings moved from
DeprecationWarning to SyntaxWarning
(ref https://docs.python.org/3/whatsnew/changelog.html#python-3-12-0-final
and search for gh-98401). In a future Python version this will become
SyntaxError.

Multiple instances of these SyntaxWarnings are currently printed when
running the BaseTools tests using Python 3.12 (though without actually
failing the affected tests).

This commit updates all lines which were causing this type of warning.

Typical examples which needed fixing are:

- "BaseTools\Source\Python" representing a path: "\S" and "\P" are invalid
escape sequences, therefore left unchanged, therefore the test works
(with a warning in Python 3.12). r"BaseTools\Source\Python" represents
the same string, but with escapes turned off completely thus no warning.

- Where '\t\s' is used as a regex pattern, then chr(9) + '\\s' is sent
to the regex parser (with a warning in Python 3.12) since '\s' is not a
valid Python escape sequence. This works correctly, though arguably for
the wrong reasons. r'\t\s' sends the same as '\\t\\s', as originally
intended and with no warning.

(Note that ' and " are not fundamentally different in Python.)

Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
2024-09-23 04:55:53 +00:00
Taylor Beebe
cac0955658 BaseTools: Update Stack Cookie Logic
This patch updates the GenC logic to generate a random stack cookie value
for the stack check libraries. These random values improve security
for modules which cannot update the global intrinsics.

If the stack cookie value is randomized in the AutoGen.h file each
build, the build system will determine the module/library must be
rebuilt causing effectively a clean build every time. This also makes
binary reproducibility impossible.

This patch updates the early build scripts to create 32 and 64-bit JSON
files in the build output directory which each contain 100 randomized
stack cookie values for each bitwidth. If the JSON files are already
present, then they are not recreated which allows them to be stored and
moved to other builds for binary reproducibility. Because they are in
the build directory, a clean build will cause the values to be
regenerated.

The logic which creates AutoGen.h will read these JSON files and use a
hash of the module GUID (the hash seed is fixed in Basetools) to index
into the array of stack cookie values for the module bitwidth. This
model is necessary because there isn't thread-consistent data so we
cannot use a locking mechanism to ensure only one thread is writing to
the stack cookie files at a time. With this model, the build threads
only need to read from the files.

Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
2024-09-13 03:58:46 +00:00
Oliver Smith-Denny
61f9695f20 BaseTools: Remove Pip BaseTools
BaseTools was moved out to a separate repo and consumed as a pip
module by edk2 CI. This process has not led to the desired goals
of doing so, so this patch removes the pip based BaseTools from
edk2 CI.

The original goal of moving BaseTools to a pip module was
primarily to speed up the development process, as the old edk2
mailing list was slow. However, with edk2 moving to PRs, it now
actually slows the BaseTools development process to have to do
a PR in another repo, publish the module, and then make a PR
in edk2 to consume the new BaseTools. It also holds up using
the features in a new BaseTools in other PRs.

There were other goals of moving, such as allowing projects to
use the BaseTools outside of edk2. This can still be accomplished
outside of this PR, this PR simply stops edk2 CI from using the
pip module.

Continuous-integration-options: PatchCheck.ignore-multi-package

Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
2024-09-10 00:41:53 +00:00
Ken Lautner
50871ee0ec BaseTools: GenMake: FIx missing logs from GenMake.py
EdkLogger logs were not showing up as part of the build log output.
Adding the EdkLogger import to GenMake.py fixes the missing log prints.

Signed-off-by: Kenneth Lautner <kenlautner3@gmail.com>
2024-09-02 06:32:06 +00:00
Joey Vagedes
95ee7f3ef7 BaseTools: Trim: Add header/footer for ASL include
When including one ASL file in another, add a header / footer to the
included file to easily tell where the included file starts and ends.

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
2024-08-31 04:58:54 +00:00
Jason1 Lin
68b4c4b481 BaseTools/Capsule: Support Different Hash Algorithm for Payload Digest
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4821

- The capsule payload digest got hardcoded inside the GenerateCapsule
  script as "sha256".

- It would be hard for the caller to change the supported hash algorithm
  which supported on OpenSSL or Windows signtool program and platform.

- Capsule payload digest signed data is followed by the PKCS#7 standard,
  in EDK-II CryptoPkg "Pkcs7Verify ()" is supported to validate with
  several hash algorithms naturally.
  (md5, sha1, sha256, sha384, and sha512)

- Deliver below changes within this patch,
  (1) Introduce an optional argument "--hash-algorithm" to assign
      the caller expected one and leave the default value "sha256"
      to support the backward compatibility.

  (2) Add the double quotes to put the string of certificate's
      subject name inside it.

  (3) Set "Open" argument of "SignToolSubjectName" into "False".

  (4) Set "Convert" argument of "SignToolSubjectName: into "str".

  (5) Correct the actual name of the "--subject-name" flag.

  (6) Add back correct number of arguments for PayloadDescriptor
      class object initializing.

Note:
- Platform needs to support the correspond hash algorithm to validate
  the digital signature or the failure would be observed.

- Set the md5 and sha1 algorithm as EOL based on the CryptoPkg supported
  table and reject the capsule creation.

Signed-off-by: Jason1 Lin <jason1.lin@intel.com>
2024-08-05 05:49:16 +00:00
Leif Lindholm
5366def8d0 BaseTools: drop GeneralCheckNonAscii() from ECC
The GeneralCheckNonAscii() function is a sledgehammer rejecting any file
containing any character outside of the 7-bit ASCII encoding space, as
well as the DEL character (which seems unrelated).

This conflicts with basic stuff like correctly spelling certain proper
nouns in comments (like copyright statements), or string literals (for
example in multi-language driver binding ComponentNames).

So rip it out, to be replaced by more fine-grained checks to be added as
identified and needed.

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
2024-07-15 13:27:00 +00:00
Leif Lindholm
8ade6d7bd1 BaseTools: fix consistent Ecc misspelling of ASCII
Ecc concistently referred to ASCII/Ascii as ACSII/Acsii, which
bugged me to no end when trying to figure out how those tests
worked. Fix all instances.

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
2024-07-15 13:27:00 +00:00
Michael Kubacki
6c061c4715 BaseTools/Ecc: Allow static as a modifier
Currently, `STATIC` is allowed as a function modifier but `static`
results in the below ECC errors:

```
*Error code: 5001
*Return type of a function should exist and in the first line
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
*[UefiDebugLibDebugPortProtocolExitBootServicesCallback] Return
 Type should appear at the start of line

EFI coding style error
*Error code: 5002
*Any optional functional modifiers should exist and next to the
 return type
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
```

This is because `GetDataTypeFromModifier()` will return both `static`
and the return type (e.g. `VOID`) whereas for a modifier in the list
(e.g. `STATIC`) it will return only the return type allowing logic in
Ecc/c.py to process the modifier and return type with current logic.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
2024-07-10 14:32:08 +00:00
Sergii Dmytruk
26bc42f1e3 BaseTools/GenerateCapsule.py: Fix checking for DepExp presence
struct.unpack() returns a tuple even for a single-element pack,
resulting in signature verification being evaluated to false even when
the signature is there.

This fixes --decode and --dump-info actions incorrectly reporting issues
with parsing capsule dependencies when there are none.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk
eeddb86aaa BaseTools/GenerateCapsule.py: Fix inconsistent error formatting
Just add a space between colon and a more detailed error message in two
places.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk
47c1078175 BaseTools/GenerateCapsule.py: Require --output for --decode
--decode unconditionally uses args.OutputFile.name as a prefix for
output files that it creates and fails in a non-pretty way without
--output option.

This doesn't address creation/truncation of the file specified via
--output, but at least you're able to decode a capsule.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk
822ff966c6 BaseTools/GenerateCapsule.py: Better error message on --decode failure
Print error text from the exception.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk
3be79ece37 BaseTools/GenerateCapsule.py: Disallow UpdateImageIndex == 0 on --encode
This field seems to be one-based according UEFI specification, default
value is 1 and --decode of GenerateCapsule.py errors upon seeing
UpdateImageIndex less than 1.  So align --encode behaviour to enforce a
value within the 1..255 range.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk
8e7bd66dc1 BaseTools/GenerateCapsule.py: Fix --decode operation
Commit b68d566439683d0ebe60d52c85ff0e90331db740 added support for
input subject name with signtool and broke --decode operation by using
incorrect identifier in one place (could be an incomplete rename during
review).

It's `args.SignToolSubjectName`, not `args.SignSubjectName`.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk
5a4a7172bc BaseTools/FmpCapsuleHeader.py: Explain error when throwing exceptions
This gives a caller a chance to report a meaningful error to the user.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Joey Vagedes
6b9307192b BaseTools: InfBuildData: Fix Private value retrieval
Update retrieval of private guids, protocols, or ppis from a package's
declaration file to use the original path of the module's INF file
rather than the current path. When building the same module multiple
times in the same INF (by override the define's FILE_GUID), a temporary
instance of the module is generated outside the package, causing the
retrieval of private values to fail as the check to access private
values is done by verifying the module to build, is inside the package.

Signed-off-by: Joey Vagedes <Joey.Vagedes@gmail.com>

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
2024-07-04 11:55:25 +00:00
Jeff Brasen
aa99d36be9 BaseTools/BuildReport: Improve compile_commands generation
This produces output that matches CodeChecker log command

- Set directory to build output path
- Set build destination to the object created instead of the path
- Add recursive macro support
- Add lookup in module.Macros dictionary
- Add leading include flag to include list
- Add source file to compile commands

Signed-off-by: Jeff Brasen <jbrasen@nvidia.com>
2024-06-15 11:07:28 +00:00
Gua Guo
d97f964f7c BaseTools/Fmmt.py: Python 3.12 support
Ref to https://docs.python.org/3/whatsnew/3.12.html
A backslash-character pair that is not a valid
escape sequence now generates

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-23 21:36:22 +00:00
Taylor Beebe
5ba3602e45 BaseTools: Use Stronger Matching for NULL Linked Libraries
To prevent the possibility that a library with a name like
NULLTestLib is interpreted as a NULL linked library, use
more explicit pattern matching to ensure that the library
name follows the pattern NULL%d.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-15 19:04:08 +00:00
Taylor Beebe
d77efa2ebe BaseTools: Don't Recurse NULL Includes Not Linked to Module
When collecting the required library instances for modules and
libraries, included libraries will be recursed to ensure the module is
built with all the libraries directly linked to it and indirectly
linked to it via included libraries.

Using the following scenario as an example:

[LibraryClasses.common.DXE_CORE]
NULL|Path/To/Library1.inf // Includes DebugLib

[LibraryClasses.common.DXE_DRIVER]
NULL|Path/To/Library2.inf // Includes DebugLib

[LibraryClasses.common.DXE_CORE, LibraryClasses.common.DXE_DRIVER]
DebugLib|MdePkg/Library/BaseDebugLibSerialPort/BaseDebugLibSerialPort.inf

[Components]
MdeModulePkg/Core/Dxe/DxeMain.inf // Includes DebugLib

The DXE_CORE NULL library will be assigned a fake library class like
NULL1 and the DXE_DRIVER will be assigned NULL2. The recursion logic
will see NULL1 as a directly linked and will add an instance of it to
the list of libraries which need to be included in the module. When
DebugLib is evaluated, the recursion logic will add the libraries
DebugLib depends on to the queue which includes both NULL1 and NULL2.
When NULL2 is unqueued, an instance of it will also be added to the
list of libraries needed to build DxeMain which now means that both
NULL1 and NULL2 have been linked.

NULL includes outside of module overrides are not supported according
to the spec, but we do it anyways so this seems like a case which
should be fixed. This change updates the recursion logic to skip
evaluating NULL libraries unless they are linked directly to the
module/library being evaluated.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-15 19:04:08 +00:00
Igniculus Fu
5572b43c67 BaseTools/GenFds: Apply OEM_CAPSULE_FLAGS during Capsule generation.
Bugzilla ticket 4633

FdfParser.py has defined a key named OEM_CAPSULE_FLAGS to set the
lower 16 bits of EFI_CAPSULE_HEADER.Flags. However, this key is totally
"forgotten" in Capsule.py, making it impossible to set lower 16 bits of
this field, and leading to an always FALSE when comparing to
gEfiMdeModulePkgTokenSpaceGuid.PcdSystemRebootAfterCapsuleProcessFlag
in MdeModulePkg/Library/DxeCapsuleLibFmp/DxeCapsuleProcessLib.c:
ProcessTheseCapsules().

This patch ORs the value of OEM_CAPSULE_FLAGS with previously calculated
CAPSULE_FLAGS value, making the lower 16 bits of value being correctly set.

Signed-off-by: Igniculus Fu <igniculus.fu@amd.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Abner Chang <abner.chang@amd.com>
Cc: Eric Xing <eric.xing@amd.com>
Cc: Abdul Lateef Attar <abdattar@amd.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-03-13 10:53:30 +00:00
Michael Kubacki
bff9815b61 BaseTools/GenFds: Resolve absolute workspace INF paths
Currently, if an INF path is an absolute path on Linux (begins with
"/"), the "/" character will be removed. If the path is an absolute
system path, this creates an invalid path.

An example of when this may be an issue is in external dependencies
where an INF is within the external dependency, the `set_build_var`
flag is set, and DSC files refer to files by its build variable
(e.g. `$(SHARED_BINARIES)/Module.inf`). INFs in a binary distribution
like this example may contain a [Binaries] section and refer to
different section files that can be used by a platform to compose an
FFS file. For example, the PE32 (.efi) and DEPEX (.depex) files.

In this case, `$(SHARED_BINARIES)` will be an absolute path to the
ext dep directory and `FfsInfStatement.__InfParse__` will remove the
leading "/" character so the path is invalid.

This change first checks if the absolute path will resolve into the
current workspace. If it does (as will happen in the shared crypto
ext dep example above), it modifies the path to be relative to the
workspace so later logic dependent on relative paths can operate on
it. If the absolute path is not within the current workspace, it
follows previous behavior for backward compatibility to that
scenario.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-03-04 16:23:58 +00:00
Laszlo Ersek
bac9c74080 BaseTools/AutoGen: declare ProcessLibraryConstructorList() for SEC modules
Most module types have standardized entry point function prototypes. They
are declared in headers like

- MdePkg/Include/Library/PeiCoreEntryPoint.h
- MdePkg/Include/Library/PeimEntryPoint.h
- MdePkg/Include/Library/DxeCoreEntryPoint.h
- MdePkg/Include/Library/UefiDriverEntryPoint.h
- MdePkg/Include/Library/UefiApplicationEntryPoint.h

These header files also declare matching ProcessLibraryConstructorList()
prototypes.

The SEC module type does not have a standardized entry point prototype
(aka parameter list), therefore no header file like the above ones exists
for SEC. Consequently, no header file *declares*
ProcessLibraryConstructorList() for SEC modules, even though AutoGen
always *defines* ProcessLibraryConstructorList() with the same, empty,
parameter list (i.e., just (VOID)).

The lack of a central declaration is a problem because in SEC code,
ProcessLibraryConstructorList() needs to be called manually, and those
calls need a prototype. Most SEC modules in edk2 get around this by
declaring ProcessLibraryConstructorList() manually, while some others use
an incorrect (PEIM) prototype.

Liming suggested in
<https://bugzilla.tianocore.org/show_bug.cgi?id=991#c2> that AutoGen
provide the declaration as well; implement that in this patch.

Mike suggested that the feature be gated with INF_VERSION, for
compatibility reasons. (INF_VERSION >= 1.30) reflects that the latest
(draft) version of the INF specification, as of this writing, is commit
a31e3c842bee / version 1.29.

For example, if we modify "OvmfPkg/Sec/SecMain.inf" as follows:

> diff --git a/OvmfPkg/Sec/SecMain.inf b/OvmfPkg/Sec/SecMain.inf
> index 3c47a664a95d..dca932a474ee 100644
> --- a/OvmfPkg/Sec/SecMain.inf
> +++ b/OvmfPkg/Sec/SecMain.inf
> @@ -8,7 +8,7 @@
>  ##
>
>  [Defines]
> -  INF_VERSION                    = 0x00010005
> +  INF_VERSION                    = 1.30
>    BASE_NAME                      = SecMain
>    FILE_GUID                      = df1ccef6-f301-4a63-9661-fc6030dcc880
>    MODULE_TYPE                    = SEC

then the patch produces the following difference in
"Build/OvmfX64/NOOPT_GCC5/X64/OvmfPkg/Sec/SecMain/DEBUG/AutoGen.h":

> --- AutoGen.h.orig      2024-02-06 23:10:23.469535345 +0100
> +++ AutoGen.h   2024-02-07 00:00:57.361294055 +0100
> @@ -220,6 +220,13 @@
>
>  // Definition of PCDs used in libraries is in AutoGen.c
>
> +// ProcessLibraryConstructorList() declared here because SEC has no standard entry point.
> +VOID
> +EFIAPI
> +ProcessLibraryConstructorList (
> +  VOID
> +  );
> +
>
>  #ifdef __cplusplus
>  }

which presently (as of edk2 commit edc6681206c1) triggers the following
build error:

> In file included from OvmfPkg/Sec/SecMain.c:14:
> MdePkg/Include/Library/PeimEntryPoint.h:74:1: error: conflicting types for
> ‘ProcessLibraryConstructorList’; have ‘void(void *, const
> EFI_PEI_SERVICES **)’ {aka ‘void(void *, const struct _EFI_PEI_SERVICES
> **)’}
>    74 | ProcessLibraryConstructorList (
>       | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> In file included from <command-line>:
> Build/OvmfX64/NOOPT_GCC5/X64/OvmfPkg/Sec/SecMain/DEBUG/AutoGen.h:226:1: note:
> previous declaration of ‘ProcessLibraryConstructorList’ with type
> ‘void(void)’
>   226 | ProcessLibraryConstructorList (
>       | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

That's a genuine bug in OvmfPkg that needs to be fixed, but we keep
compatibility with existent SEC modules until/unless they upgrade
INF_VERSION to 1.30+.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=991
Suggested-by: Liming Gao <gaoliming@byosoft.com.cn>
Suggested-by: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20240224210504.41873-1-lersek@redhat.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-02-29 09:56:38 +00:00
Jayaprakash, N
3e91e42136 BaseTools: Syntax warning invalid escape sequence \C
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4666

This commit fixes the issue reported through BZ4666.
The Syntax warning related to invalid escape sequence
for \C is seen on Windows OS based builds of edk2 sources.
On Windows the path seperator needs to prefixed with \
so essentially we need to use \\ as path seperator.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Jayaprakash N <n.jayaprakash@intel.com>
Signed-off-by: Jayaprakash N <n.jayaprakash@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-28 00:23:58 +00:00
Ashraf Ali S
e32b58ab5a BaseTools: Remove Duplicate sets of SkuName and SkuId from allskuset
Currently when the platform has many SKUs then allskuset will be having
so many duplicate. and while parsing the allskuset will take longer
time while assigning Pcd.SkuInfoList.
This patch is to eliminate those duplicate entries to reduce the
build time

Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Amy Chan <amy.chan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2024-02-08 19:27:18 +00:00
devel@edk2.groups.io
8f316e99ec BaseTools: Optimize GenerateByteArrayValue and CollectPlatformGuids APIs
During the Incremental build GenerateByteArrayValue used to generate the
ByteArrayValue even when there is no change in the PCD/VPDs. which is
time consuming API based on the number of PCD/VPDs and SKU IDs.

The optimization is that GenerateByteArrayValue is used to store the
StructuredPcdsData in a JSON file for each of the arch. and during the
Incremental build this API will check, if there is any change in the
Structured PCD/VPDs then rest of the flow remains the same.
if there is no change then it will return the provious build data.

Flow:
during the 1st build StructuredPcdsData.json is not exists,
StructuredPcdsData will be dumped to json file. and it will copy the
output.txt as well.
Note: as the output.txt are different for different Arch, so it will be
stored in the Arch folder.
During the Incremental build check if there is any change in Structured
PCD/VPD. if there is a change in Structured VPD/PCD then recreate the
StructuredPcdsData.json, and rest of the flow remains same.
if there is no change in VPD/PCD read the output.txt and return the data

Unit Test:
Test1: Modified the Structured Pcds default from DEC file. current flow
is executing.
Test2: Override the default value of the PCD from DEC file. current flow
is executing.
Test3: Modified/Override the PCD from DSC file. current flow executing
Test4: Modified/Override the FDF from DSC file. current flow executing
Test5: update the default value from Command Line.current flow executing
Test6: Build without change in PCD in DSC, FDF, DEC and Command Line the
proposed changes will be executing, and the return data remains the same
with and without the changes.
Test7: Build with and without modified the include headers of Structured
PCDs. if there is any change in those Structured PCD header then
current flow will be executed.

With these changes it's helping to save around ~2.5min to ~3.5min of
Incremental build time in my build environment.

Sample PR: https://github.com/tianocore/edk2-basetools/pull/113

Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Amy Chan <amy.chan@intel.com>
Cc: Sai Chaganty <rangasai.v.chaganty@intel.com>
Cc: Digant H Solanki <digant.h.solanki@intel.com>
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2024-02-08 04:08:38 +00:00
levi.yun
40a45b5a2b Basetools: Include PCD declarations from Library Instance
The patch "[PATCH v3 1/2] StandaloneMmPkg: Make StandaloneMmCpu driver
architecture independent" (https://edk2.groups.io/g/devel/message/109178)
removed ArmPkg/ArmPkg.dec from the Packages section in the
INF file: StandaloneMmPkg/Drivers/StandaloneMmCpu/StandaloneMmCpu.inf

This change was done as part of making the StandaloneMmCpu driver
architecture independent.

Although this change is correct, it results in a side effect
here some platforms that utilise PCDs declared in ArmPkg.dec are
no longer declared.

An example of this issue can be seen when building
edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc

$ build -a AARCH64 -t GCC -p Platform/ARM/SgiPkg/PlatformStandaloneMm.dsc
build.py...
/mnt/source/edk2-platforms/Platform/ARM/SgiPkg/PlatformStandaloneMm.fdf(23):
  error F001: PCD (gArmTokenSpaceGuid.PcdFdBaseAddress) used in
  FDF is not declared in DEC files.

As seen above, removing ArmPkg.dec from the Packages section in the
StandAloneMmCpu Driver Inf file triggers build failure.
Although, ArmPkg.dec is included in other Library Instances,
the build system does not include the declarations from
.dec files defined in Library instances.

The build system only includes the PCD declarations from DEC files
that are specified in INF files for Modules (components).

Therefore, extend the build system to include the Packages from
Library Instances so that the PCD declarations from the respective package
DEC files are included.

This patch can be seen on
    https://github.com/LeviYeoReum/edk2/tree/levi/2848_dec_check_on_library

Signed-off-by: levi.yun <yeoreum.yun@arm.com>
Tested-by: Pierre Gondois <pierre.gondois@arm.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Sami Mujawar <sami.mujawar@arm.com>
2024-01-31 10:58:05 +00:00
Joey Vagedes
6c488a2f39 BaseTools: Fix raw strings containing valid escape characters
Fixes raw regex strings that contain valid (and purposeful) escape
characters as they are being treated as individual characters rather
than the single escaped character they represent (i.e. '\t' is being
treated as a '\' and a 't' rather than a single tab character).

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-01-10 13:54:01 +00:00
Joey Vagedes via groups.io
9f0061a03b BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-21 00:33:31 +00:00
Rebecca Cran
89705ad6c6 BaseTools: FMMT GuidTool Auto Select Config file Enabling
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4624

Currently, Python FMMT tool does not support automatically
select FMMTConf.ini file which saves GuidTool settings.
This patch supports this features.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-12-20 22:59:03 +00:00
Yuwei Chen
5b5481526f BaseTools: fixing FMMT ShrinkFv issue
1. FvLength not change issue;
2. FileSystemGuid align with File Size;

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen
59f0d350a9 BaseTools: FMMT support ELF UPLD parser
FMMT add new function to support the .elf file parsing.
Using '-v' option, the UPLD info will be printed out.

'''
- UNIVERSAL_PAYLOAD_INFO
  - 4 bytes align (BOOLEAN)
    - Identifier
    - SpecRevision
    - Attribute
    - Revision
    - Capability
    - ProducerId
    - ImageId
UPLD Buffer
'''

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen
9627447625 BaseTools: FMMT replace output file is not generated successfully
For replace function, when target Ffs and new ffs are with
same size, the output file can not be generated successfully.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen
b5f5106c1e BaseTools: FMMT replace new free space fixing in replace
In FMMT replace function, when newffs size <= targetffs size,
the new free space is calculated wrong as loss the pad data delta size.
That will cause invalid binary generated.
This patch fixes this issue.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2023-12-11 10:12:52 +00:00
Yuwei Chen
b75d9f556d BaseTools: trim warning to error
As the error is changed to warning, Trim.py will skip the build
error when the source code have exactly issue.
This patch change warning to error to opens the checking.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2023-10-18 18:17:19 +00:00