Subsequent to updating the 'null' DEBUG macro to explicitly
discard its expression, it is possible to remove this warning
suppression from CLANGDWARF and still successfully compile
its RELEASE build.
Note that CLANGPDB did and does not have this warning suppressed,
and so before updating the 'null' DEBUG macro, CLANGPDB RELEASE
was not building successfully in recent versions of clang,
but was stopping with the error:
.../edk2/OvmfPkg/VirtioSerialDxe/VirtioSerial.c:28:22: error:
variable 'EventNames' is not needed and will not be
emitted [-Werror,-Wunneeded-internal-declaration]
STATIC CONST CHAR8 *EventNames[] = {
^
This change makes the two CLANG variants match with respect
to this warning, and leaves the warning enabled which is
considered a benefit as it has the potential to catch real
coding errors
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
For the CodeQl `AuditOnly` flag,
prioritize Plugin settings over global settings.
This patch adjusts the logic for the global `AuditOnly` setting,
placing it before the Plugin setting code.
This ensures that Plugin settings take precedence over global settings.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Abdul Lateef Attar <AbdulLateef.Attar@amd.com>
The GCC5 TOOL_CHAIN_TAG is being deprecated in favor of
GCC. LinuxGcc5ToolChain.py needs to be updated to find
the correct ARM/AARCH64/RISCV/LOONGARCH64 compilers for
the GCC TOOL_CHAIN_TAG, as well, otherwise it defaults
to the system GCC, which is typically X64 based.
In order to keep this backward and forward compatible,
the plugin now checks for the substring "GCC" in the
TOOL_CHAIN_TAG and will set either the "GCC5" or "GCC"
env variables used by tools_def.template to find the
GCC compiler for the requested architecture.
This plugin is also renamed to drop the old GCC5.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
This is part of a sequence of commits to restore build on the XCODE5
toolchain.
The definition is required on other toolchains, but on XCODE5 results
in a macro redefined error (from the existing value 255) from
/usr/include/stdint.h.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
In Python 3.12 invalid escape sequences in strings moved from
DeprecationWarning to SyntaxWarning
(ref https://docs.python.org/3/whatsnew/changelog.html#python-3-12-0-final
and search for gh-98401). In a future Python version this will become
SyntaxError.
Multiple instances of these SyntaxWarnings are currently printed when
running the BaseTools tests using Python 3.12 (though without actually
failing the affected tests).
This commit updates all lines which were causing this type of warning.
Typical examples which needed fixing are:
- "BaseTools\Source\Python" representing a path: "\S" and "\P" are invalid
escape sequences, therefore left unchanged, therefore the test works
(with a warning in Python 3.12). r"BaseTools\Source\Python" represents
the same string, but with escapes turned off completely thus no warning.
- Where '\t\s' is used as a regex pattern, then chr(9) + '\\s' is sent
to the regex parser (with a warning in Python 3.12) since '\s' is not a
valid Python escape sequence. This works correctly, though arguably for
the wrong reasons. r'\t\s' sends the same as '\\t\\s', as originally
intended and with no warning.
(Note that ' and " are not fundamentally different in Python.)
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
This patch is to sync RETURN_ERROR macro with the
MdePkg/Include/Base.h
Ref: 1a89d9887f MdePkg:Update Return Error Macro in Base.h
Fixing RETURN_ERROR macro.
It is causing problem in Coverity Static analysis tool
as we are directly converting the UINT value to INTN.
Changing value from UINT to INTN might cause problema
Here we know that the values would not be in loss of data.
To increase the code quality and increase the static tool
analysis score we have to change it
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Abdul Lateef Attar <AbdulLateef.Attar@amd.com>
Clang insists on emitting a movt/movw pair into the function
pro/epilogues to load the stack protector reference value from memory,
and this movt/movw pair may turn out non-consecutively in the
instruction stream.
The resulting symbol reference cannot be fixed up by GenFw, as PE/COFF
always treats movt/movw as a pair, and the ELF-to-PE conversion will
therefore fail.
Just disable the stack protector when using CLANGDWARF.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
This patch updates the GenC logic to generate a random stack cookie value
for the stack check libraries. These random values improve security
for modules which cannot update the global intrinsics.
If the stack cookie value is randomized in the AutoGen.h file each
build, the build system will determine the module/library must be
rebuilt causing effectively a clean build every time. This also makes
binary reproducibility impossible.
This patch updates the early build scripts to create 32 and 64-bit JSON
files in the build output directory which each contain 100 randomized
stack cookie values for each bitwidth. If the JSON files are already
present, then they are not recreated which allows them to be stored and
moved to other builds for binary reproducibility. Because they are in
the build directory, a clean build will cause the values to be
regenerated.
The logic which creates AutoGen.h will read these JSON files and use a
hash of the module GUID (the hash seed is fixed in Basetools) to index
into the array of stack cookie values for the module bitwidth. This
model is necessary because there isn't thread-consistent data so we
cannot use a locking mechanism to ensure only one thread is writing to
the stack cookie files at a time. With this model, the build threads
only need to read from the files.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
BaseTools was moved out to a separate repo and consumed as a pip
module by edk2 CI. This process has not led to the desired goals
of doing so, so this patch removes the pip based BaseTools from
edk2 CI.
The original goal of moving BaseTools to a pip module was
primarily to speed up the development process, as the old edk2
mailing list was slow. However, with edk2 moving to PRs, it now
actually slows the BaseTools development process to have to do
a PR in another repo, publish the module, and then make a PR
in edk2 to consume the new BaseTools. It also holds up using
the features in a new BaseTools in other PRs.
There were other goals of moving, such as allowing projects to
use the BaseTools outside of edk2. This can still be accomplished
outside of this PR, this PR simply stops edk2 CI from using the
pip module.
Continuous-integration-options: PatchCheck.ignore-multi-package
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
Starting with Visual Studio 2019 version 16.10, the /volatileMetadata
option is enabled by default when generating x64 code.
This patch disables the /volatileMetadata option for x64 builds in both
VS2019 and VS2022.
We observed a slight increase in used space for the Firmware volumes in
VS2019. Upon investigation, we found that VS2019 version 16.10 enabled
this feature by default. Disabling /volatileMetadata helps reduce the
used space by approximately 3.5KB by considering the 2 Firmware volumes
(2KB uncompressed FV and 1.5KB of compressed FV)
Signed-off-by: Ashraf Ali <ashraf.ali.s@intel.com>
EdkLogger logs were not showing up as part of the build log output.
Adding the EdkLogger import to GenMake.py fixes the missing log prints.
Signed-off-by: Kenneth Lautner <kenlautner3@gmail.com>
When including one ASL file in another, add a header / footer to the
included file to easily tell where the included file starts and ends.
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4821
- The capsule payload digest got hardcoded inside the GenerateCapsule
script as "sha256".
- It would be hard for the caller to change the supported hash algorithm
which supported on OpenSSL or Windows signtool program and platform.
- Capsule payload digest signed data is followed by the PKCS#7 standard,
in EDK-II CryptoPkg "Pkcs7Verify ()" is supported to validate with
several hash algorithms naturally.
(md5, sha1, sha256, sha384, and sha512)
- Deliver below changes within this patch,
(1) Introduce an optional argument "--hash-algorithm" to assign
the caller expected one and leave the default value "sha256"
to support the backward compatibility.
(2) Add the double quotes to put the string of certificate's
subject name inside it.
(3) Set "Open" argument of "SignToolSubjectName" into "False".
(4) Set "Convert" argument of "SignToolSubjectName: into "str".
(5) Correct the actual name of the "--subject-name" flag.
(6) Add back correct number of arguments for PayloadDescriptor
class object initializing.
Note:
- Platform needs to support the correspond hash algorithm to validate
the digital signature or the failure would be observed.
- Set the md5 and sha1 algorithm as EOL based on the CryptoPkg supported
table and reject the capsule creation.
Signed-off-by: Jason1 Lin <jason1.lin@intel.com>
WinRcPath generally takes about 2 seconds to run, due to calling
multiple .bat files behind the scenes. This change reduces this time to
~0 seconds due to the following changes:
1. It will attempt to load the path from the cache, which is located a
$(WORKSPACE)/Conf/.rc_path. If the loading is a success and the rc_path
still exists, it will use it.
2. If the cache did not exist, or the path provided by the cache does
not exist, it will find the rc path via the .bat files. If that
succeeds, it will write the path to the cache.
Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
With Python 3.12 invalid escape sequences now generate warning messages.
This change fixes the problem exposed by the warning message.
```
BaseTools/Scripts\BinToPcd.py:40: SyntaxWarning: invalid escape sequence
BaseTools\Scripts\BinToPcd.py:46: SyntaxWarning: invalid escape sequence
```
Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
The codeql/cpp-queries pack used in CodeQlQueries.qls was versioned
1.1.0 for the CodeQL CLI v2.18.1 release currently used.
https://github.com/github/codeql/blob/codeql-cli/v2.18.1/cpp/ql/src/qlpack.yml
This change pins that pack version to prevent the CodeQL CLI and
pack from getting out of sync until explicitly updated.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Start the build with TOOL_CHAIN_TAG VS2015 by launch:
Build -t VS2015
ERROR: Would get following build error message:
'c:\Program' is not recognized as an internal or external command,
operable program or batch file.
NMAKE : fatal error U1077: '"c:\Program Files\Windows Kits\8.1\bin\x86\\rc.exe' : return code '0x1'
Stop.
Fix the build error,
Tested :
TOOL_CHAIN_TAG = VS2015 (>Build -t VS2015)
TOOL_CHAIN_TAG = VS2015x86 (>Build -t VS2015x86)
Signed-off-by: wilson_chen <wilson_chen@phoenix.com>
GCC5 and CLANGDWARF for IA32/X64 use -Os or -Oz as the optimization
level, which agressively optimizes for the smallest possible object
code.
On AARCH64, RISCV64 and ARM, we use -O3 instead, which results in
considerable image bloat, to the point where the Raspberry Pi 4 build in
edk2-platforms does not even build with -D SECURE_BOOT_ENABLE.
So let's align CLANGDWARF across all architectures, and use -Oz
throughout.
Note that O3 is still used for the linker, which build in LTO mode and
therefore performs some code generation as well. This is deliberate: LLD
does not support the Os/Oz optimization levels at all, and using Oz for
the compile pass is sufficient to reduce the code size substantially.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Per TCBZ2372, clang on Linux emits a warning if an enum-typed variable
is compared with a constant outside of the range of the enum. Such
comparisons are performed in multiple locations in DXE core on
variables of type EFI_MEMORY_TYPE. This patch moves the OEM and OS
reserved types into the EFI_MEMORY_TYPE enum itself to resolve this
issue and improve readability. This commit does this for the BaseTools
copy of this enum.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
Adds a new `-g` parameter so that output will also include the GitHub
username.
This change uses a simple regular expression as opposed to directly
returning the original line from the file to make the extraction of
GitHub usernames more robust to other changes on the line in the
maintainers text file.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
This patch moves GnuNoteBti.bin from ArmPkg to BaseTools as it
is used during the build by GCC. This removes an unnecessary
dependency on ArmPkg from BaseTools and keeps build related
files in BaseTools.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
This moves the GccLto files from ArmPkg to BaseTools as they
are files that are only used in the build. This removes an
artificial dependency on ArmPkg from BaseTools and keeps build
related files in BaseTools.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
This query seems to produce a rate of false positives with some
common patterns in edk2 like passing function pointers for callback.
Due to the usage of `STATIC` instead of `static` particularly for
functions, this query was rarely used in the past. It is removed
here to prevent future false positives.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Some unit tests would fail to execute or execute and not produce any output
logs. In these cases, the only output would be in the CI Log as
`UnitTest Execution Error`.
A UnitTest Execution Error should be considered the same as a unit tests
test failing.
Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
The GeneralCheckNonAscii() function is a sledgehammer rejecting any file
containing any character outside of the 7-bit ASCII encoding space, as
well as the DEL character (which seems unrelated).
This conflicts with basic stuff like correctly spelling certain proper
nouns in comments (like copyright statements), or string literals (for
example in multi-language driver binding ComponentNames).
So rip it out, to be replaced by more fine-grained checks to be added as
identified and needed.
Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Ecc concistently referred to ASCII/Ascii as ACSII/Acsii, which
bugged me to no end when trying to figure out how those tests
worked. Fix all instances.
Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Static relocation types have been handled in GenFw if using the PIC, and
the CC flags not enable `fno-pic` by default.
The option `fno-plt` is not necessary, as is not created by defualt in
edk2(static linking) regardless of wether `fplt` is used or not, so
remove this option from the LoongArch common CC flags.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Chao Li <lichao@loongson.cn>
Currently, `STATIC` is allowed as a function modifier but `static`
results in the below ECC errors:
```
*Error code: 5001
*Return type of a function should exist and in the first line
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
*[UefiDebugLibDebugPortProtocolExitBootServicesCallback] Return
Type should appear at the start of line
EFI coding style error
*Error code: 5002
*Any optional functional modifiers should exist and next to the
return type
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
```
This is because `GetDataTypeFromModifier()` will return both `static`
and the return type (e.g. `VOID`) whereas for a modifier in the list
(e.g. `STATIC`) it will return only the return type allowing logic in
Ecc/c.py to process the modifier and return type with current logic.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Commit ea56fa3d47 ("BaseTools: Enable RISC-V architecture for RISC-V
EDK2 CI") introduced a UTF-8 NBSP (0xc2a0) inside a comment block
otherwise copied identically from pre-existing architectures.
This was clearly unintentional and confuses things when looking for
which files contain UTF-8 encodings, so change it to good old 0x20.
Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Adding tools_def for VS2022.
Update WindowsVsToolChain to support VS2022.
Update set_vsPrefix_envs and toolsetup and edksetup to support VS2022.
Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
struct.unpack() returns a tuple even for a single-element pack,
resulting in signature verification being evaluated to false even when
the signature is there.
This fixes --decode and --dump-info actions incorrectly reporting issues
with parsing capsule dependencies when there are none.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
--decode unconditionally uses args.OutputFile.name as a prefix for
output files that it creates and fails in a non-pretty way without
--output option.
This doesn't address creation/truncation of the file specified via
--output, but at least you're able to decode a capsule.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
This field seems to be one-based according UEFI specification, default
value is 1 and --decode of GenerateCapsule.py errors upon seeing
UpdateImageIndex less than 1. So align --encode behaviour to enforce a
value within the 1..255 range.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
Commit b68d566439 added support for
input subject name with signtool and broke --decode operation by using
incorrect identifier in one place (could be an incomplete rename during
review).
It's `args.SignToolSubjectName`, not `args.SignSubjectName`.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
Update retrieval of private guids, protocols, or ppis from a package's
declaration file to use the original path of the module's INF file
rather than the current path. When building the same module multiple
times in the same INF (by override the define's FILE_GUID), a temporary
instance of the module is generated outside the package, causing the
retrieval of private values to fail as the check to access private
values is done by verifying the module to build, is inside the package.
Signed-off-by: Joey Vagedes <Joey.Vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
The xdrlib dependency was removed in commit
5cadb8ce21 but the actual import of the
module was not removed. This commit removes the import of xdrlib and
sorts the imports.
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
This produces output that matches CodeChecker log command
- Set directory to build output path
- Set build destination to the object created instead of the path
- Add recursive macro support
- Add lookup in module.Macros dictionary
- Add leading include flag to include list
- Add source file to compile commands
Signed-off-by: Jeff Brasen <jbrasen@nvidia.com>
Fix logic error that changes the commit range checked depending
on the verbosity level set.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
The commit message format requirements have been updated for
GitHub PR based code reviews and no longer required Cc: tags
for the maintainers and reviewers. Remove the Cc: tag check
from PatchCheck.py.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
This fixes an issue where the CodeQL queries currently fetched in the
pipeline are incompatible with the current executable used.
Update to pick up functional and security fixes. See the following
comparison for detailed differences:
https://github.com/github/codeql-cli-binaries/compare/v2.16.1...v2.17.3
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Updates CodeQL to work with the latest queries. Includes functional
and security fixes within the CodeQL CLI binary.
For more information on release details see:
https://github.com/github/codeql-cli-binaries/releases
For changes between the previous version (2.14.5) and 2.16.1 see:
https://github.com/github/codeql-cli-binaries/compare/v2.14.5...v2.16.1
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Joey Vagedes <joey.vagedes@gmail.com>
Ref to https://docs.python.org/3/whatsnew/3.12.html
A backslash-character pair that is not a valid
escape sequence now generates
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
To prevent the possibility that a library with a name like
NULLTestLib is interpreted as a NULL linked library, use
more explicit pattern matching to ensure that the library
name follows the pattern NULL%d.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Mike Beaton
Nhi Pham
Abdul Lateef Attar
Oliver Smith-Denny
Ard Biesheuvel
Taylor Beebe
Oliver Smith-Denny
Ashraf Ali
Ken Lautner
Joey Vagedes
Jason1 Lin
Joey Vagedes
Antaeus Kleinert-Strand
Michael Kubacki
wilson_chen
Dmitry Antipov
v-bhavanisu
Leif Lindholm
Chao Li
Matthew Carlson
Sergii Dmytruk
Jeff Brasen
Michael D Kinney
Gua Guo