Commit Graph

3011 Commits

Author SHA1 Message Date
Oliver Smith-Denny 61f9695f20 BaseTools: Remove Pip BaseTools
BaseTools was moved out to a separate repo and consumed as a pip
module by edk2 CI. This process has not led to the desired goals
of doing so, so this patch removes the pip based BaseTools from
edk2 CI.

The original goal of moving BaseTools to a pip module was
primarily to speed up the development process, as the old edk2
mailing list was slow. However, with edk2 moving to PRs, it now
actually slows the BaseTools development process to have to do
a PR in another repo, publish the module, and then make a PR
in edk2 to consume the new BaseTools. It also holds up using
the features in a new BaseTools in other PRs.

There were other goals of moving, such as allowing projects to
use the BaseTools outside of edk2. This can still be accomplished
outside of this PR, this PR simply stops edk2 CI from using the
pip module.

Continuous-integration-options: PatchCheck.ignore-multi-package

Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
2024-09-10 00:41:53 +00:00
Ashraf Ali bfb33c0e09 BaseTools: Disable MSVC volatileMetadata for VS2019 and VS2022 for X64
Starting with Visual Studio 2019 version 16.10, the /volatileMetadata
option is enabled by default when generating x64 code.
This patch disables the /volatileMetadata option for x64 builds in both
VS2019 and VS2022.

We observed a slight increase in used space for the Firmware volumes in
VS2019. Upon investigation, we found that VS2019 version 16.10 enabled
this feature by default. Disabling /volatileMetadata helps reduce the
used space by approximately 3.5KB by considering the 2 Firmware volumes
(2KB uncompressed FV and 1.5KB of compressed FV)

Signed-off-by: Ashraf Ali <ashraf.ali.s@intel.com>
2024-09-06 01:54:53 +00:00
Ken Lautner 50871ee0ec BaseTools: GenMake: FIx missing logs from GenMake.py
EdkLogger logs were not showing up as part of the build log output.
Adding the EdkLogger import to GenMake.py fixes the missing log prints.

Signed-off-by: Kenneth Lautner <kenlautner3@gmail.com>
2024-09-02 06:32:06 +00:00
Joey Vagedes 95ee7f3ef7 BaseTools: Trim: Add header/footer for ASL include
When including one ASL file in another, add a header / footer to the
included file to easily tell where the included file starts and ends.

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
2024-08-31 04:58:54 +00:00
Jason1 Lin 68b4c4b481 BaseTools/Capsule: Support Different Hash Algorithm for Payload Digest
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4821

- The capsule payload digest got hardcoded inside the GenerateCapsule
  script as "sha256".

- It would be hard for the caller to change the supported hash algorithm
  which supported on OpenSSL or Windows signtool program and platform.

- Capsule payload digest signed data is followed by the PKCS#7 standard,
  in EDK-II CryptoPkg "Pkcs7Verify ()" is supported to validate with
  several hash algorithms naturally.
  (md5, sha1, sha256, sha384, and sha512)

- Deliver below changes within this patch,
  (1) Introduce an optional argument "--hash-algorithm" to assign
      the caller expected one and leave the default value "sha256"
      to support the backward compatibility.

  (2) Add the double quotes to put the string of certificate's
      subject name inside it.

  (3) Set "Open" argument of "SignToolSubjectName" into "False".

  (4) Set "Convert" argument of "SignToolSubjectName: into "str".

  (5) Correct the actual name of the "--subject-name" flag.

  (6) Add back correct number of arguments for PayloadDescriptor
      class object initializing.

Note:
- Platform needs to support the correspond hash algorithm to validate
  the digital signature or the failure would be observed.

- Set the md5 and sha1 algorithm as EOL based on the CryptoPkg supported
  table and reject the capsule creation.

Signed-off-by: Jason1 Lin <jason1.lin@intel.com>
2024-08-05 05:49:16 +00:00
Joey Vagedes 159f1aee56 BaseTools/WinRcPath: Improve Performance.
WinRcPath generally takes about 2 seconds to run, due to calling
multiple .bat files behind the scenes. This change reduces this time to
~0 seconds due to the following changes:

1. It will attempt to load the path from the cache, which is located a
$(WORKSPACE)/Conf/.rc_path. If the loading is a success and the rc_path
still exists, it will use it.

2. If the cache did not exist, or the path provided by the cache does
not exist, it will find the rc path via the .bat files. If that
succeeds, it will write the path to the cache.

Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
2024-08-04 07:30:59 +00:00
Antaeus Kleinert-Strand 71b9bda1ac BaseTools/Scripts/BinToPcd.py: Update regex strings to use raw strings.
With Python 3.12 invalid escape sequences now generate warning messages.
This change fixes the problem exposed by the warning message.

```
BaseTools/Scripts\BinToPcd.py:40: SyntaxWarning: invalid escape sequence
BaseTools\Scripts\BinToPcd.py:46: SyntaxWarning: invalid escape sequence
```

Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
2024-08-01 11:04:09 +00:00
Michael Kubacki eed43245df CodeQlQueries.qls: Pin to the 1.1.0 codeq/cpp-queries pack
The codeql/cpp-queries pack used in CodeQlQueries.qls was versioned
1.1.0 for the CodeQL CLI v2.18.1 release currently used.

https://github.com/github/codeql/blob/codeql-cli/v2.18.1/cpp/ql/src/qlpack.yml

This change pins that pack version to prevent the CodeQL CLI and
pack from getting out of sync until explicitly updated.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
2024-07-31 23:02:15 +00:00
wilson_chen 91a822749a BaseTools: fix build error with TOOL_CHAIN_TAG VS2015 & VS2015x86
Start the build with TOOL_CHAIN_TAG VS2015 by launch:
  Build -t VS2015

ERROR: Would get following build error message:
'c:\Program' is not recognized as an internal or external command,
operable program or batch file.
NMAKE : fatal error U1077: '"c:\Program Files\Windows Kits\8.1\bin\x86\\rc.exe' : return code '0x1'
Stop.

Fix the build error,
Tested :
  TOOL_CHAIN_TAG = VS2015    (>Build -t VS2015)
  TOOL_CHAIN_TAG = VS2015x86 (>Build -t VS2015x86)

Signed-off-by: wilson_chen <wilson_chen@phoenix.com>
2024-07-30 11:15:27 +00:00
Ard Biesheuvel ecb1d67775 BaseTools/tools_def CLANGDWARF: Always use -Oz in RELEASE mode
GCC5 and CLANGDWARF for IA32/X64 use -Os or -Oz as the optimization
level, which agressively optimizes for the smallest possible object
code.

On AARCH64, RISCV64 and ARM, we use -O3 instead, which results in
considerable image bloat, to the point where the Raspberry Pi 4 build in
edk2-platforms does not even build with -D SECURE_BOOT_ENABLE.

So let's align CLANGDWARF across all architectures, and use -Oz
throughout.

Note that O3 is still used for the linker, which build in LTO mode and
therefore performs some code generation as well. This is deliberate: LLD
does not support the Os/Oz optimization levels at all, and using Oz for
the compile pass is sufficient to reduce the code size substantially.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
2024-07-28 20:45:13 +00:00
Michael Kubacki 6589843cc6 BaseTools/codeql: Update to CodeQL 2.18.1
Updates to the latest CodeQL version to resolve query dependencies.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
2024-07-26 08:17:11 +00:00
Dmitry Antipov 41426040da BaseTools: Move MEMORY_TYPE_* Defines to EFI_MEMORY_TYPE Enum
Per TCBZ2372, clang on Linux emits a warning if an enum-typed variable
is compared with a constant outside of the range of the enum. Such
comparisons are performed in multiple locations in DXE core on
variables of type EFI_MEMORY_TYPE. This patch moves the OEM and OS
reserved types into the EFI_MEMORY_TYPE enum itself to resolve this
issue and improve readability. This commit does this for the BaseTools
copy of this enum.

Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
2024-07-26 03:08:20 +00:00
Michael Kubacki 3f0c4cee94 BaseTools/GetMaintainer.py: Add GitHub username argument
Adds a new `-g` parameter so that output will also include the GitHub
username.

This change uses a simple regular expression as opposed to directly
returning the original line from the file to make the extraction of
GitHub usernames more robust to other changes on the line in the
maintainers text file.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
2024-07-25 02:28:49 +00:00
Oliver Smith-Denny da591416ee BaseTools: Move GnuNoteBti.bin to BaseTools
This patch moves GnuNoteBti.bin from ArmPkg to BaseTools as it
is used during the build by GCC. This removes an unnecessary
dependency on ArmPkg from BaseTools and keeps build related
files in BaseTools.

Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
2024-07-23 15:07:41 +00:00
Oliver Smith-Denny 990bc4e562 BaseTools: Move GccLto Files to BaseTools
This moves the GccLto files from ArmPkg to BaseTools as they
are files that are only used in the build. This removes an
artificial dependency on ArmPkg from BaseTools and keeps build
related files in BaseTools.

Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
2024-07-23 15:07:41 +00:00
Michael Kubacki 1bb9f47739 BaseTools/CodeQL: Removed unused static function query
This query seems to produce a rate of false positives with some
common patterns in edk2 like passing function pointers for callback.

Due to the usage of `STATIC` instead of `static` particularly for
functions, this query was rarely used in the past. It is removed
here to prevent future false positives.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
2024-07-16 04:04:33 +00:00
v-bhavanisu 8e6ba0dcae BaseTools/HostBasedUnitTestRunner: Promote Unittest error to CI fail.
Some unit tests would fail to execute or execute and not produce any output
logs. In these cases, the only output would be in the CI Log as
`UnitTest Execution Error`.

A UnitTest Execution Error should be considered the same as a unit tests
test failing.

Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
2024-07-15 16:50:00 +00:00
Leif Lindholm 5366def8d0 BaseTools: drop GeneralCheckNonAscii() from ECC
The GeneralCheckNonAscii() function is a sledgehammer rejecting any file
containing any character outside of the 7-bit ASCII encoding space, as
well as the DEL character (which seems unrelated).

This conflicts with basic stuff like correctly spelling certain proper
nouns in comments (like copyright statements), or string literals (for
example in multi-language driver binding ComponentNames).

So rip it out, to be replaced by more fine-grained checks to be added as
identified and needed.

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
2024-07-15 13:27:00 +00:00
Leif Lindholm 8ade6d7bd1 BaseTools: fix consistent Ecc misspelling of ASCII
Ecc concistently referred to ASCII/Ascii as ACSII/Acsii, which
bugged me to no end when trying to figure out how those tests
worked. Fix all instances.

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
2024-07-15 13:27:00 +00:00
Chao Li b92e16d5c3 BaseTools: Remove fno-plt from LoongArch CC flags
Static relocation types have been handled in GenFw if using the PIC, and
the CC flags not enable `fno-pic` by default.
The option `fno-plt` is not necessary, as is not created by defualt in
edk2(static linking) regardless of wether `fplt` is used or not, so
remove this option from the LoongArch common CC flags.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Chao Li <lichao@loongson.cn>
2024-07-12 10:11:17 +00:00
Michael Kubacki 6c061c4715 BaseTools/Ecc: Allow `static` as a modifier
Currently, `STATIC` is allowed as a function modifier but `static`
results in the below ECC errors:

```
*Error code: 5001
*Return type of a function should exist and in the first line
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
*[UefiDebugLibDebugPortProtocolExitBootServicesCallback] Return
 Type should appear at the start of line

EFI coding style error
*Error code: 5002
*Any optional functional modifiers should exist and next to the
 return type
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
```

This is because `GetDataTypeFromModifier()` will return both `static`
and the return type (e.g. `VOID`) whereas for a modifier in the list
(e.g. `STATIC`) it will return only the return type allowing logic in
Ecc/c.py to process the modifier and return type with current logic.

Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
2024-07-10 14:32:08 +00:00
Leif Lindholm 426b69830e BaseTools: change non-ASCII characters in LinuxGcc5ToolChain.py
Commit ea56fa3d47 ("BaseTools: Enable RISC-V architecture for RISC-V
EDK2 CI") introduced a UTF-8 NBSP (0xc2a0) inside a comment block
otherwise copied identically from pre-existing architectures.
This was clearly unintentional and confuses things when looking for
which files contain UTF-8 encodings, so change it to good old 0x20.

Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
2024-07-09 13:38:55 +00:00
Matthew Carlson 95a6892aac BaseTools: Add VS2022 support.
Adding tools_def for VS2022.
Update WindowsVsToolChain to support VS2022.
Update set_vsPrefix_envs and toolsetup and edksetup to support VS2022.

Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
2024-07-08 16:50:21 +00:00
Sergii Dmytruk 26bc42f1e3 BaseTools/GenerateCapsule.py: Fix checking for DepExp presence
struct.unpack() returns a tuple even for a single-element pack,
resulting in signature verification being evaluated to false even when
the signature is there.

This fixes --decode and --dump-info actions incorrectly reporting issues
with parsing capsule dependencies when there are none.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk eeddb86aaa BaseTools/GenerateCapsule.py: Fix inconsistent error formatting
Just add a space between colon and a more detailed error message in two
places.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk 47c1078175 BaseTools/GenerateCapsule.py: Require --output for --decode
--decode unconditionally uses args.OutputFile.name as a prefix for
output files that it creates and fails in a non-pretty way without
--output option.

This doesn't address creation/truncation of the file specified via
--output, but at least you're able to decode a capsule.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk 822ff966c6 BaseTools/GenerateCapsule.py: Better error message on --decode failure
Print error text from the exception.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk 3be79ece37 BaseTools/GenerateCapsule.py: Disallow UpdateImageIndex == 0 on --encode
This field seems to be one-based according UEFI specification, default
value is 1 and --decode of GenerateCapsule.py errors upon seeing
UpdateImageIndex less than 1.  So align --encode behaviour to enforce a
value within the 1..255 range.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk 8e7bd66dc1 BaseTools/GenerateCapsule.py: Fix --decode operation
Commit b68d566439 added support for
input subject name with signtool and broke --decode operation by using
incorrect identifier in one place (could be an incomplete rename during
review).

It's `args.SignToolSubjectName`, not `args.SignSubjectName`.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Sergii Dmytruk 5a4a7172bc BaseTools/FmpCapsuleHeader.py: Explain error when throwing exceptions
This gives a caller a chance to report a meaningful error to the user.

Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
2024-07-05 19:33:50 +00:00
Joey Vagedes 6b9307192b BaseTools: InfBuildData: Fix Private value retrieval
Update retrieval of private guids, protocols, or ppis from a package's
declaration file to use the original path of the module's INF file
rather than the current path. When building the same module multiple
times in the same INF (by override the define's FILE_GUID), a temporary
instance of the module is generated outside the package, causing the
retrieval of private values to fail as the check to access private
values is done by verifying the module to build, is inside the package.

Signed-off-by: Joey Vagedes <Joey.Vagedes@gmail.com>

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
2024-07-04 11:55:25 +00:00
Joey Vagedes 8c09d862bf BaseTools: BinToPcd: Remove xdrlib dependency
The xdrlib dependency was removed in commit
5cadb8ce21 but the actual import of the
module was not removed. This commit removes the import of xdrlib and
sorts the imports.

Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
2024-06-30 23:45:58 +00:00
Jeff Brasen aa99d36be9 BaseTools/BuildReport: Improve compile_commands generation
This produces output that matches CodeChecker log command

- Set directory to build output path
- Set build destination to the object created instead of the path
- Add recursive macro support
- Add lookup in module.Macros dictionary
- Add leading include flag to include list
- Add source file to compile commands

Signed-off-by: Jeff Brasen <jbrasen@nvidia.com>
2024-06-15 11:07:28 +00:00
Michael D Kinney f9c2f2fa0f BaseTools/Scripts: Fix PatchCheck commit range
Fix logic error that changes the commit range checked depending
on the verbosity level set.

Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
2024-06-07 09:15:15 +08:00
Michael D Kinney 0e3189d406 BaseTools/Scripts: Remove Cc: tag check from PatchCheck.py
The commit message format requirements have been updated for
GitHub PR based code reviews and no longer required Cc: tags
for the maintainers and reviewers.  Remove the Cc: tag check
from PatchCheck.py.

Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
2024-05-28 20:23:14 +00:00
Michael Kubacki 7142e64841 CodeQL: Update from 2.16.1 to 2.17.3
This fixes an issue where the CodeQL queries currently fetched in the
pipeline are incompatible with the current executable used.

Update to pick up functional and security fixes. See the following
comparison for detailed differences:

https://github.com/github/codeql-cli-binaries/compare/v2.16.1...v2.17.3

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2024-05-21 02:24:08 +00:00
Gua Guo c0dfe3ec1f BaseTools/GetUtcDateTime.py: Python 3.12 support
Ref to https://docs.python.org/3/whatsnew/3.12.html
utcnow() and utcfromtimestamp() are deprecated
Prevent use it cause build error.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-26 03:01:55 +00:00
Michael Kubacki 90b6725562 Update to CodeQL 2.16.1
Updates CodeQL to work with the latest queries. Includes functional
and security fixes within the CodeQL CLI binary.

For more information on release details see:

https://github.com/github/codeql-cli-binaries/releases

For changes between the previous version (2.14.5) and 2.16.1 see:

https://github.com/github/codeql-cli-binaries/compare/v2.14.5...v2.16.1

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Joey Vagedes <joey.vagedes@gmail.com>
2024-04-24 23:11:19 +00:00
Gua Guo d97f964f7c BaseTools/Fmmt.py: Python 3.12 support
Ref to https://docs.python.org/3/whatsnew/3.12.html
A backslash-character pair that is not a valid
escape sequence now generates

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-23 21:36:22 +00:00
Taylor Beebe 5ba3602e45 BaseTools: Use Stronger Matching for NULL Linked Libraries
To prevent the possibility that a library with a name like
NULLTestLib is interpreted as a NULL linked library, use
more explicit pattern matching to ensure that the library
name follows the pattern NULL%d.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-15 19:04:08 +00:00
Taylor Beebe d77efa2ebe BaseTools: Don't Recurse NULL Includes Not Linked to Module
When collecting the required library instances for modules and
libraries, included libraries will be recursed to ensure the module is
built with all the libraries directly linked to it and indirectly
linked to it via included libraries.

Using the following scenario as an example:

[LibraryClasses.common.DXE_CORE]
NULL|Path/To/Library1.inf // Includes DebugLib

[LibraryClasses.common.DXE_DRIVER]
NULL|Path/To/Library2.inf // Includes DebugLib

[LibraryClasses.common.DXE_CORE, LibraryClasses.common.DXE_DRIVER]
DebugLib|MdePkg/Library/BaseDebugLibSerialPort/BaseDebugLibSerialPort.inf

[Components]
MdeModulePkg/Core/Dxe/DxeMain.inf // Includes DebugLib

The DXE_CORE NULL library will be assigned a fake library class like
NULL1 and the DXE_DRIVER will be assigned NULL2. The recursion logic
will see NULL1 as a directly linked and will add an instance of it to
the list of libraries which need to be included in the module. When
DebugLib is evaluated, the recursion logic will add the libraries
DebugLib depends on to the queue which includes both NULL1 and NULL2.
When NULL2 is unqueued, an instance of it will also be added to the
list of libraries needed to build DxeMain which now means that both
NULL1 and NULL2 have been linked.

NULL includes outside of module overrides are not supported according
to the spec, but we do it anyways so this seems like a case which
should be fixed. This change updates the recursion logic to skip
evaluating NULL libraries unless they are linked directly to the
module/library being evaluated.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-04-15 19:04:08 +00:00
Igniculus Fu 5572b43c67 BaseTools/GenFds: Apply OEM_CAPSULE_FLAGS during Capsule generation.
Bugzilla ticket 4633

FdfParser.py has defined a key named OEM_CAPSULE_FLAGS to set the
lower 16 bits of EFI_CAPSULE_HEADER.Flags. However, this key is totally
"forgotten" in Capsule.py, making it impossible to set lower 16 bits of
this field, and leading to an always FALSE when comparing to
gEfiMdeModulePkgTokenSpaceGuid.PcdSystemRebootAfterCapsuleProcessFlag
in MdeModulePkg/Library/DxeCapsuleLibFmp/DxeCapsuleProcessLib.c:
ProcessTheseCapsules().

This patch ORs the value of OEM_CAPSULE_FLAGS with previously calculated
CAPSULE_FLAGS value, making the lower 16 bits of value being correctly set.

Signed-off-by: Igniculus Fu <igniculus.fu@amd.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Abner Chang <abner.chang@amd.com>
Cc: Eric Xing <eric.xing@amd.com>
Cc: Abdul Lateef Attar <abdattar@amd.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-03-13 10:53:30 +00:00
Michael Kubacki bff9815b61 BaseTools/GenFds: Resolve absolute workspace INF paths
Currently, if an INF path is an absolute path on Linux (begins with
"/"), the "/" character will be removed. If the path is an absolute
system path, this creates an invalid path.

An example of when this may be an issue is in external dependencies
where an INF is within the external dependency, the `set_build_var`
flag is set, and DSC files refer to files by its build variable
(e.g. `$(SHARED_BINARIES)/Module.inf`). INFs in a binary distribution
like this example may contain a [Binaries] section and refer to
different section files that can be used by a platform to compose an
FFS file. For example, the PE32 (.efi) and DEPEX (.depex) files.

In this case, `$(SHARED_BINARIES)` will be an absolute path to the
ext dep directory and `FfsInfStatement.__InfParse__` will remove the
leading "/" character so the path is invalid.

This change first checks if the absolute path will resolve into the
current workspace. If it does (as will happen in the shared crypto
ext dep example above), it modifies the path to be relative to the
workspace so later logic dependent on relative paths can operate on
it. If the absolute path is not within the current workspace, it
follows previous behavior for backward compatibility to that
scenario.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-03-04 16:23:58 +00:00
Laszlo Ersek bac9c74080 BaseTools/AutoGen: declare ProcessLibraryConstructorList() for SEC modules
Most module types have standardized entry point function prototypes. They
are declared in headers like

- MdePkg/Include/Library/PeiCoreEntryPoint.h
- MdePkg/Include/Library/PeimEntryPoint.h
- MdePkg/Include/Library/DxeCoreEntryPoint.h
- MdePkg/Include/Library/UefiDriverEntryPoint.h
- MdePkg/Include/Library/UefiApplicationEntryPoint.h

These header files also declare matching ProcessLibraryConstructorList()
prototypes.

The SEC module type does not have a standardized entry point prototype
(aka parameter list), therefore no header file like the above ones exists
for SEC. Consequently, no header file *declares*
ProcessLibraryConstructorList() for SEC modules, even though AutoGen
always *defines* ProcessLibraryConstructorList() with the same, empty,
parameter list (i.e., just (VOID)).

The lack of a central declaration is a problem because in SEC code,
ProcessLibraryConstructorList() needs to be called manually, and those
calls need a prototype. Most SEC modules in edk2 get around this by
declaring ProcessLibraryConstructorList() manually, while some others use
an incorrect (PEIM) prototype.

Liming suggested in
<https://bugzilla.tianocore.org/show_bug.cgi?id=991#c2> that AutoGen
provide the declaration as well; implement that in this patch.

Mike suggested that the feature be gated with INF_VERSION, for
compatibility reasons. (INF_VERSION >= 1.30) reflects that the latest
(draft) version of the INF specification, as of this writing, is commit
a31e3c842bee / version 1.29.

For example, if we modify "OvmfPkg/Sec/SecMain.inf" as follows:

> diff --git a/OvmfPkg/Sec/SecMain.inf b/OvmfPkg/Sec/SecMain.inf
> index 3c47a664a95d..dca932a474ee 100644
> --- a/OvmfPkg/Sec/SecMain.inf
> +++ b/OvmfPkg/Sec/SecMain.inf
> @@ -8,7 +8,7 @@
>  ##
>
>  [Defines]
> -  INF_VERSION                    = 0x00010005
> +  INF_VERSION                    = 1.30
>    BASE_NAME                      = SecMain
>    FILE_GUID                      = df1ccef6-f301-4a63-9661-fc6030dcc880
>    MODULE_TYPE                    = SEC

then the patch produces the following difference in
"Build/OvmfX64/NOOPT_GCC5/X64/OvmfPkg/Sec/SecMain/DEBUG/AutoGen.h":

> --- AutoGen.h.orig      2024-02-06 23:10:23.469535345 +0100
> +++ AutoGen.h   2024-02-07 00:00:57.361294055 +0100
> @@ -220,6 +220,13 @@
>
>  // Definition of PCDs used in libraries is in AutoGen.c
>
> +// ProcessLibraryConstructorList() declared here because SEC has no standard entry point.
> +VOID
> +EFIAPI
> +ProcessLibraryConstructorList (
> +  VOID
> +  );
> +
>
>  #ifdef __cplusplus
>  }

which presently (as of edk2 commit edc6681206) triggers the following
build error:

> In file included from OvmfPkg/Sec/SecMain.c:14:
> MdePkg/Include/Library/PeimEntryPoint.h:74:1: error: conflicting types for
> ‘ProcessLibraryConstructorList’; have ‘void(void *, const
> EFI_PEI_SERVICES **)’ {aka ‘void(void *, const struct _EFI_PEI_SERVICES
> **)’}
>    74 | ProcessLibraryConstructorList (
>       | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> In file included from <command-line>:
> Build/OvmfX64/NOOPT_GCC5/X64/OvmfPkg/Sec/SecMain/DEBUG/AutoGen.h:226:1: note:
> previous declaration of ‘ProcessLibraryConstructorList’ with type
> ‘void(void)’
>   226 | ProcessLibraryConstructorList (
>       | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

That's a genuine bug in OvmfPkg that needs to be fixed, but we keep
compatibility with existent SEC modules until/unless they upgrade
INF_VERSION to 1.30+.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=991
Suggested-by: Liming Gao <gaoliming@byosoft.com.cn>
Suggested-by: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20240224210504.41873-1-lersek@redhat.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-02-29 09:56:38 +00:00
Jayaprakash, N 3e91e42136 BaseTools: Syntax warning invalid escape sequence \C
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4666

This commit fixes the issue reported through BZ4666.
The Syntax warning related to invalid escape sequence
for \C is seen on Windows OS based builds of edk2 sources.
On Windows the path seperator needs to prefixed with \
so essentially we need to use \\ as path seperator.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Jayaprakash N <n.jayaprakash@intel.com>
Signed-off-by: Jayaprakash N <n.jayaprakash@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-28 00:23:58 +00:00
Michael D Kinney 6d571c0070 BaseTools/Scripts/PatchCheck: Error if commit modifies multiple packages
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4679

Update PatchCheck.py to evaluate all the files modified in each commit
and generate an error if:
* A commit adds/modifies files in multiple package directories
* A commit adds/modifies files in multiple non-package directories
* A commit adds/modifies files in both a package and a non-package
  directory
* A commit deletes files from multiple package directories
* A commit deletes files from multiple non-package directories
* A commit deletes files from both a package and a non-package
  directory

Modifications to files in the root of the repository are not
evaluated.

This check is skipped if PatchCheck.py is run on a patch file or
input from stdin because this multiple package commit check depends
on information from a git repository.

If --ignore-multi-package option is set, then reduce the multiple
package commit check from an error to a warning for all commits in
the commit range provided to PatchCheck.py.

Add check for a 'Continuous-integration-options:' commit message
tag that allows one or more options to be specified at the individual
commit scope to enable/disable continuous integration checks. This
tag must start at the beginning of a commit message line and may
appear more than once in a commit message.

Add support for a Continuous-integration-options tag value of
'PatchCheck.ignore-multi-package' that reduces the multiple package
commit check from an error to a warning for the specific commits that
specify this option.  Example:

  Continuous-integration-options: PatchCheck.ignore-multi-package

The set of packages are found by searching for DEC files in a git
repository. The list of DEC files in a git repository is collected
with the following git command:

  git ls-files *.dec

The set of files added/modified by each commit is found using the
following git command:

  git diff-tree --no-commit-id --name-only --diff-filter=AM -r <commit>

The set of files deleted by each commit is found using the
following git command:

  git diff-tree --no-commit-id --name-only --diff-filter=D -r <commit>

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-27 19:28:19 +00:00
Michael D Kinney 0bbec15b54 BaseTools/Scripts/PatchCheck: Error if no Cc tags are present
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4694

If no Cc tags are detected in a commit message, then generate an
error. All patches sent for review are required to provide the set
of maintainers and reviewers responsible for the directories/files
modified. The set of maintainers and reviewers are documented in
Maintainers.txt and can be retrieved using the script
BaseTools/Scripts/GetMaintainer.py.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-27 19:28:19 +00:00
Michael D Kinney 45ad13bb64 BaseTools/Scripts/PatchCheck: Return CommitMessageCheck errors
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4693

Commit signatures are checked and error messages are logged but
errors are not captured and returned from find_signatures() in the
CommitMessageCheck class. This causes signature errors to be
silently ignored by CI.

Update logic in CommitMessageCheck class to return errors
detected in commit message signatures.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
2024-02-27 19:28:19 +00:00
Michael D Kinney dae8c29dab BaseTools/Scripts/PatchCheck: Update Author checks
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4680

* Reject patches that match Author email "devel@edk2.groups.io"
* Update the current check for " via Groups.Io" to perform a
  case insensitive match. It appears that groups.io has changed the
  format of this string to use all lower case.

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
2024-02-27 19:28:19 +00:00
Yi Li 44fdc4f398 BaseTools: Update keybaord map based on UEFI spec 2.10
REF: UEFI SPEC 2.10 34.8.10 EFI_KEY

Add EfiKeyIntl0-9.

Signed-off-by: Yi Li <yi1.li@intel.com>

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2024-02-26 15:32:09 +00:00