BaseTools has been duplicating and adapting code that is defined in
MdePkg and MdeModulePkg. This leads to desync issues where the same
symbols may be backed by different functions with slightly different
semantics and also fixes that apply only to BaseTools or only to MdePkg
and MdeModulePkg.
To address these issues, update BaseTools/Source/C to utilize the code
from MdePkg and MdeModulePkg.
Signed-off-by: Marvin Häuser <mhaeuser@posteo.de>
BaseTools has been duplicating and adapting code that is defined in
MdePkg and MdeModulePkg. This leads to desync issues where the same
symbols may be backed by different functions with slightly different
semantics and also fixes that apply only to BaseTools or only to MdePkg
and MdeModulePkg.
To address these issues, update BaseTools/Source/C to utilize the code
from MdePkg and MdeModulePkg.
Signed-off-by: Marvin Häuser <mhaeuser@posteo.de>
Update the brotli submodule to the latest commit (ed1995b6bda1)
so that the build isn't broken in GCC 12 compilers.
Signed-off-by: Savva Mitrofanov <savvamtr@gmail.com>
If command line options are moved into a response file
of a GCC family build, then the file path separators are
converted from '\' to '/'. However, this can corrupt
command line options that are quoted strings.
Update GenMake to no convert '\' to '/' in quoted strings.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
The prehistoric code base doesn't build with ISO C23. Set the C
standard to C11 (for both clang and gcc) so it continues to build with
gcc 15 (which uses C23 by default).
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This patch addresses an issue in the FMMT operations where the parent
reference is not checked for NoneType. This oversight can lead to an
AttributeError: 'NoneType' object has no attribute 'Name' when
attempting to access the parent reference. The fix involves adding a
check for NoneType before accessing the parent reference to ensure that
the operations handle such cases gracefully.
The affected functions include:
- AddNewFfs
- ReplaceFfs
- ExtractFfs
These functions now include proper checks to prevent the AttributeError.
Signed-off-by: Ashraf Ali S <ashraf.ali.s@intel.com>
Refer to the docs of python, `os.path.normcase(path)` function:
"Normalize the case of a pathname. On Windows, convert all characters in
the pathname to lowercase, and also convert forward slashes to backward
slashes. On other operating systems, return the path unchanged."
`os.path.normpath(path)` also convert forward slashes to backward slashes.
So call `os.path.normcase` after `os.path.normpath` just convert path to
lowercase on Windows(only).
And Windows is case-insensitive but case-preserving.
So the usage of `os.path.normcase(os.path.normpath(path))` can be
simplified to `os.path.normpath(path)`. Then we can use case-preserving
paths rather than lowercase paths in compile_commands.json file
or build log.
But this patch continue to use `os.path.normcase`
when comparing/searching paths.
Signed-off-by: Yang Gang <yanggang@byosoft.com.cn>
This change added the build script to cross compile the base tool
binaries for Linux ARM/AARCH64 systems.
The needed libuuid system library is pulled from source file and rebuilt
to support the corresponding library dependencies. Individual tools'
makefiles are also updated to link the cross compiled library as well.
The EDK2 base tool build script was also updated to support such change.
This was tested functional on Linux ARM host system.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Kun Qin <kun.qin@microsoft.com>
This change adds the support of crossbuilding basetool for Windows ARM/
ARM64 systems, which will enable the generally available pipeline agents
to build binary tools and make releases as they see fit.
The EDK2 base tools build script is also updated to support cross
compilation using this script.
The crossbuilt binary output is tested on Windows ARM based hardware
systems.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Kun Qin <kun.qin@microsoft.com>
This change focuses on the support of building basetool natively for
Windows ARM/ARM64 host system, which will enable the ARM based platforms
to build UEFI and unit tests.
Note that the warnings due to integer conversions are suppressed for
this specific target to avoid too much local changes carried in MU. The
formal change should drop all these binaries and move to pythonic
scripts.
The binary output is tested on Windows ARM based hardware systems.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Kun Qin <kun.qin@microsoft.com>
GenFw relies on static ELF relocation tables emitted by the linker (via
the --emit-relocs command line switch). These are different from the
dynamic relocations that a dynamic loader uses: static relocations are
emitted by the compiler/assembler, and consumed by the linker to
construct the executable. Only when the load address is a priori unknown
are dynamic relocations emitted, by the linker, in a format that the
dynamic loader can consume.
This distinction is relevant because only dynamic relocations cover the
GOT, and so GOT based indirections are better avoided. Unfortunately,
there are cases where the toolchain insists on emitting GOT based symbol
references, and so we have to deal with them in one of 2 ways:
- replace GOT based symbol references with direct references, so that
the GOT entries themselves are no longer used, and can be ignored when
generating the PE/COFF relocation tables (AARCH64 and RISCV64 take
this approach);
- infer the locations of the GOT slots from the references appearing in
the code, and emit PE/COFF relocations for them so that their contents
will be fixed up appropriately.
The latter is the approach taken by GenFw for x86_64, which is the only
feasible approach for its ISA, given that GOT slots can be used as
memory operands in many different types of instructions, not all of
which can be converted straight-forwardly.
E.g.,
movq foo@GOTPCREL(%rip), %rax
can always be converted into
leaq foo(%rip), %rax
whereas
cmpq foo@GOTPCREL(%rip), %rax
can only be converted under the 32-bit position dependent code model,
into
cmpq $foo, %rax
and so the GOT references cannot be elided when generating position
independent code, which is what GenFw requires.
To remove the need for the linker to guess where the instructions start,
the ELF psABI for x86_64 specifies a couple of relaxable alternatives
for GOTPCREL, which are used to annotate particular classes of GOT
referencing instructions that may be relaxed to their non-GOT
counterparts.
There is no specification for what --emit-relocs is supposed to produce,
or whether or not its output is supposed to reflect such relaxations.
ld.bfd and LLD behave differently in this regard, and the latter may
emit R_X86_64_REX_GOTPCRELX relocations for MOV instructions that it
already has relaxed into LEA instructions. This means the displacement
in the instruction no longer refers to the GOT slot, but directly to the
object itself, and emitting a relocation is not only unnecessary, but
also harmful as the PE/COFF loader will corrupt the object when it
applies the relocations at startup.
Under the position independent code model, the only relaxation that the
linker could have applied for a R_X86_64_REX_GOTPCRELX relocation is MOV
to LEA, so detect whether the instruction is already LEA, and ignore the
relocation if that is the case.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
This is part of a sequence of commits to restore build on the XCODE5
toolchain.
The definition is required on other toolchains, but on XCODE5 results
in a macro redefined error (from the existing value 255) from
/usr/include/stdint.h.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
In Python 3.12 invalid escape sequences in strings moved from
DeprecationWarning to SyntaxWarning
(ref https://docs.python.org/3/whatsnew/changelog.html#python-3-12-0-final
and search for gh-98401). In a future Python version this will become
SyntaxError.
Multiple instances of these SyntaxWarnings are currently printed when
running the BaseTools tests using Python 3.12 (though without actually
failing the affected tests).
This commit updates all lines which were causing this type of warning.
Typical examples which needed fixing are:
- "BaseTools\Source\Python" representing a path: "\S" and "\P" are invalid
escape sequences, therefore left unchanged, therefore the test works
(with a warning in Python 3.12). r"BaseTools\Source\Python" represents
the same string, but with escapes turned off completely thus no warning.
- Where '\t\s' is used as a regex pattern, then chr(9) + '\\s' is sent
to the regex parser (with a warning in Python 3.12) since '\s' is not a
valid Python escape sequence. This works correctly, though arguably for
the wrong reasons. r'\t\s' sends the same as '\\t\\s', as originally
intended and with no warning.
(Note that ' and " are not fundamentally different in Python.)
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
This patch is to sync RETURN_ERROR macro with the
MdePkg/Include/Base.h
Ref: 1a89d9887f MdePkg:Update Return Error Macro in Base.h
Fixing RETURN_ERROR macro.
It is causing problem in Coverity Static analysis tool
as we are directly converting the UINT value to INTN.
Changing value from UINT to INTN might cause problema
Here we know that the values would not be in loss of data.
To increase the code quality and increase the static tool
analysis score we have to change it
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Abdul Lateef Attar <AbdulLateef.Attar@amd.com>
This patch updates the GenC logic to generate a random stack cookie value
for the stack check libraries. These random values improve security
for modules which cannot update the global intrinsics.
If the stack cookie value is randomized in the AutoGen.h file each
build, the build system will determine the module/library must be
rebuilt causing effectively a clean build every time. This also makes
binary reproducibility impossible.
This patch updates the early build scripts to create 32 and 64-bit JSON
files in the build output directory which each contain 100 randomized
stack cookie values for each bitwidth. If the JSON files are already
present, then they are not recreated which allows them to be stored and
moved to other builds for binary reproducibility. Because they are in
the build directory, a clean build will cause the values to be
regenerated.
The logic which creates AutoGen.h will read these JSON files and use a
hash of the module GUID (the hash seed is fixed in Basetools) to index
into the array of stack cookie values for the module bitwidth. This
model is necessary because there isn't thread-consistent data so we
cannot use a locking mechanism to ensure only one thread is writing to
the stack cookie files at a time. With this model, the build threads
only need to read from the files.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
BaseTools was moved out to a separate repo and consumed as a pip
module by edk2 CI. This process has not led to the desired goals
of doing so, so this patch removes the pip based BaseTools from
edk2 CI.
The original goal of moving BaseTools to a pip module was
primarily to speed up the development process, as the old edk2
mailing list was slow. However, with edk2 moving to PRs, it now
actually slows the BaseTools development process to have to do
a PR in another repo, publish the module, and then make a PR
in edk2 to consume the new BaseTools. It also holds up using
the features in a new BaseTools in other PRs.
There were other goals of moving, such as allowing projects to
use the BaseTools outside of edk2. This can still be accomplished
outside of this PR, this PR simply stops edk2 CI from using the
pip module.
Continuous-integration-options: PatchCheck.ignore-multi-package
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
EdkLogger logs were not showing up as part of the build log output.
Adding the EdkLogger import to GenMake.py fixes the missing log prints.
Signed-off-by: Kenneth Lautner <kenlautner3@gmail.com>
When including one ASL file in another, add a header / footer to the
included file to easily tell where the included file starts and ends.
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4821
- The capsule payload digest got hardcoded inside the GenerateCapsule
script as "sha256".
- It would be hard for the caller to change the supported hash algorithm
which supported on OpenSSL or Windows signtool program and platform.
- Capsule payload digest signed data is followed by the PKCS#7 standard,
in EDK-II CryptoPkg "Pkcs7Verify ()" is supported to validate with
several hash algorithms naturally.
(md5, sha1, sha256, sha384, and sha512)
- Deliver below changes within this patch,
(1) Introduce an optional argument "--hash-algorithm" to assign
the caller expected one and leave the default value "sha256"
to support the backward compatibility.
(2) Add the double quotes to put the string of certificate's
subject name inside it.
(3) Set "Open" argument of "SignToolSubjectName" into "False".
(4) Set "Convert" argument of "SignToolSubjectName: into "str".
(5) Correct the actual name of the "--subject-name" flag.
(6) Add back correct number of arguments for PayloadDescriptor
class object initializing.
Note:
- Platform needs to support the correspond hash algorithm to validate
the digital signature or the failure would be observed.
- Set the md5 and sha1 algorithm as EOL based on the CryptoPkg supported
table and reject the capsule creation.
Signed-off-by: Jason1 Lin <jason1.lin@intel.com>
Per TCBZ2372, clang on Linux emits a warning if an enum-typed variable
is compared with a constant outside of the range of the enum. Such
comparisons are performed in multiple locations in DXE core on
variables of type EFI_MEMORY_TYPE. This patch moves the OEM and OS
reserved types into the EFI_MEMORY_TYPE enum itself to resolve this
issue and improve readability. This commit does this for the BaseTools
copy of this enum.
Signed-off-by: Oliver Smith-Denny <osde@linux.microsoft.com>
The GeneralCheckNonAscii() function is a sledgehammer rejecting any file
containing any character outside of the 7-bit ASCII encoding space, as
well as the DEL character (which seems unrelated).
This conflicts with basic stuff like correctly spelling certain proper
nouns in comments (like copyright statements), or string literals (for
example in multi-language driver binding ComponentNames).
So rip it out, to be replaced by more fine-grained checks to be added as
identified and needed.
Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Ecc concistently referred to ASCII/Ascii as ACSII/Acsii, which
bugged me to no end when trying to figure out how those tests
worked. Fix all instances.
Signed-off-by: Leif Lindholm <quic_llindhol@quicinc.com>
Currently, `STATIC` is allowed as a function modifier but `static`
results in the below ECC errors:
```
*Error code: 5001
*Return type of a function should exist and in the first line
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
*[UefiDebugLibDebugPortProtocolExitBootServicesCallback] Return
Type should appear at the start of line
EFI coding style error
*Error code: 5002
*Any optional functional modifiers should exist and next to the
return type
*file: D:\src\edk2\Build\.pytool\Plugin\EccCheck\MdePkg\Library\UefiDebugLibDebugPortProtocol\DebugLibConstructor.c
*Line number: 37
```
This is because `GetDataTypeFromModifier()` will return both `static`
and the return type (e.g. `VOID`) whereas for a modifier in the list
(e.g. `STATIC`) it will return only the return type allowing logic in
Ecc/c.py to process the modifier and return type with current logic.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
struct.unpack() returns a tuple even for a single-element pack,
resulting in signature verification being evaluated to false even when
the signature is there.
This fixes --decode and --dump-info actions incorrectly reporting issues
with parsing capsule dependencies when there are none.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
--decode unconditionally uses args.OutputFile.name as a prefix for
output files that it creates and fails in a non-pretty way without
--output option.
This doesn't address creation/truncation of the file specified via
--output, but at least you're able to decode a capsule.
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
