Commit Graph

20 Commits

Author SHA1 Message Date
Chen, Christine a64b944942 BaseTools: Add FMMT Python Tool
The FMMT python tool is used for firmware files operation, which has
the Fv/FFs-based 'View'&'Add'&'Delete'&'Replace' operation function:

1.Parse a FD(Firmware Device) / FV(Firmware Volume) / FFS(Firmware Files)
2.Add a new FFS into a FV file (both included in a FD file or not)
3.Replace an FFS in a FV file with a new FFS file
4.Delete an FFS in a FV file (both included in a FD file or not)
5.Extract the FFS from a FV file (both included in a FD file or not)

This version of FMMT Python tool does not support PEIM rebase feature,
this feature will be added in future update.

Currently the FMMT C tool is saved in edk2-staging repo, but its
quality and coding style can't meet the Edk2 quality, which is hard to
maintain (Hard/Duplicate Code; Regression bugs; Restrict usage).

The new Python version keeps same functions with origin C version. It
has higher quality and better coding style, and it is much easier to
extend new functions and to maintain.

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1847
RFC Link: https://edk2.groups.io/g/devel/message/82877
Staging Link: https://github.com/tianocore/edk2-staging/tree/PyFMMT

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Liming Gao <gaoliming@byosoft.com.cn>
2022-05-06 04:22:21 +00:00
Matthew Carlson 1534b6228b BaseTools: Use pip module if available, CI uses it by default
Use the new edk2-basetools pip module.
Includes a helpful message in setup to let users know which has been
selected.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei <Chen<yuwei.chen@intel.com>

Signed-off-by: Matthew Carlson <matthewfcarlson@gmail.com>
2021-02-09 03:56:37 +00:00
Bob Feng 5b4a97bbc3 BaseTools: Convert Split tool to python
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3165

There are 2 reasons to convert Split tool from C to Python.
1. We are in the process of moving the Basetools Python code
to a separate repository. But there still are many C tools under
edk2/BaseTools. To make all Basetools be in the separate repo,
we can convert the C tools to Python tools.
2. The original Split tool is very slow. This python tool can reduce
90% time.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2021-01-21 10:19:09 +00:00
Yunhua Feng 662b42db76 BaseTools: Fix BrotliCompress tool issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2946

This is the regression issue in BaseTools BrotliCompress after Brotli
is changed to submodule. BrotliCompress should store the source size
and scratch buffer size into the header of the compressed binary data.
But now, BrotliCompress doesn't store them. So, BrotliDecompress
can't work.

To fix this issue, BrotliCompress tool main() function should be provided.
It needs to support the options of -e, -d, -o file, -g gap, -q level.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Yunhua Feng <fengyunhua@byosoft.com.cn>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-11-13 06:23:56 +00:00
Pierre Gondois 88228db38e BaseTools: Rename AmlToHex script to AmlToC
The AmlToHex script and Posix/WindowsLike wrappers convert
an AML file to a .hex file, containing a C array storing
AML bytecode. This ".hex" file can then be included in a
C file, allowing to access the AML bytecode from this C
file.

The EDK2 build system doesn't allow to a depict dependency
orders between files of different languages. For instance,
in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a
".c" file since this later ".hex" file may or may not
have been created yet.

This patch renames the script as AmlToC. It is posted as
a separate patch to prevent git from seeing the renaming
as a deletion plus addition of a new file.
The ending line of the posix-like bin-wrapper script has
also been corrected.

This is a first step toward generating a C file containing
the AML bytecode from an ASL file. This C file will then
be handled by the EDK2 build system to generate an object
file.
Thus, no file inclusion will be required anymore. The C file
requiring the AML bytecode as a C array, and the ASL file,
will be compiled independently. The C array must be defined
as an external symbol. The linker is resolving the
reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;
 -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 1549651da6 BaseTools: Script for converting .aml to .hex
The "-tc" option of the iasl compiler allows to generate a
.hex file containing a C array storing AML bytecode.

An online discussion suggested that this "-tc" option
was specific to the iasl compiler and it shouldn't be relied
on. This conversation is available at:
https://edk2.groups.io/g/devel/topic/39786201#49659

A way to address this issue is to implement a compiler
independent script that takes an AML file as input, and
generates a .hex file.

This patch implements a Python script that converts an AML
file to a .hex file, containing a C array storing AML bytecode.
This scipt has been tested with the AML output from the
following compilers supported by the EDKII implementation:
  * Intel ASL compiler
  * Microsoft ASL compiler

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-06 13:52:11 +00:00
Sean Brogan f7978bb258 BaseTools: Add YAML files with path env and tool extdeps
https://bugzilla.tianocore.org/show_bug.cgi?id=2315

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-11-11 13:01:53 -08:00
Michael D Kinney 2e351cbe8e BaseTools: Replace BSD License with BSD+Patent License
https://bugzilla.tianocore.org/show_bug.cgi?id=1373

Replace BSD 2-Clause License with BSD+Patent License.  This change is
based on the following emails:

  https://lists.01.org/pipermail/edk2-devel/2019-February/036260.html
  https://lists.01.org/pipermail/edk2-devel/2018-October/030385.html

RFCs with detailed process for the license change:

  V3: https://lists.01.org/pipermail/edk2-devel/2019-March/038116.html
  V2: https://lists.01.org/pipermail/edk2-devel/2019-March/037669.html
  V1: https://lists.01.org/pipermail/edk2-devel/2019-March/037500.html

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-09 09:10:20 -07:00
Liming Gao 7aef7b7cbf BaseTools: Update PYTHON env to PYTHON_COMMAND
Update PYTHON env to PYTHON_COMMAND.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:23 +08:00
Zhijux Fan 9c2d68c0a2 BaseTools: Update windows and linux run scripts file to use Python3
Modify windows script, PosixLike script, edksetup.sh, edksetup.bat to
use Python3 based on PYTHON3_ENABLE environment.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:22 +08:00
Liming Gao 1ccc4d895d Revert BaseTools: PYTHON3 migration
This reverts commit 6693f359b3c213513c5096a06c6f67244a44dc52..
678f851312.

Python3 migration is the fundamental change. It requires every developer
to install Python3. Before this migration, the well communication and wide
verification must be done. But now, most people is not aware of this change,
and not try it. So, Python3 migration is reverted and be moved to edk2-staging
Python3 branch for the edk2 user evaluation.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
2018-10-15 08:29:14 +08:00
Yunhua Feng 04c47cd401 BaseTools: Update windows and linux run scripts file to use Python3
Modify windows script, PosixLike script, edksetup.sh, edksetup.bat to
use Python3

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:57:14 +08:00
Liming Gao dd4f667e70 BaseTools: Update Brotli Compress to the latest one 1.0.6
https://bugzilla.tianocore.org/show_bug.cgi?id=1201
Update Brotli to the latest version 1.0.6
https://github.com/google/brotli
Verify VS2017, GCC5 build.
Verify Decompression boot functionality.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
2018-10-09 22:27:32 +08:00
hchen30 855698fb69 BaseTools/Ecc: Fix import issues
1. Complete the full path for import statement. Use "EccMain" to
replace "Ecc" for the absolute path support.
2. Fix some issues on configuration file.
3. Fix an issue of RaiseError not working in EdkLogger.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Hess Chen <hesheng.chen@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-08-16 09:42:55 +08:00
Kinney, Michael D 8b63877aca BaseTools/Capsule: Add Capsule Generation Tools
https://bugzilla.tianocore.org/show_bug.cgi?id=945

Based on content from the following branch

https://github.com/Microsoft/MS_UEFI/tree/share/beta/CapsuleTools

* Convert C tools to Python
* Add common python modules to:
    BaseTools/Source/Python/Common/Uefi/Capsule
    BaseTools/Source/Python/Common/Edk2/Capsule
* Add GenerateCapsule.py to BaseTools/Source/Python/Capsule
* Add Windows and Posix wrappers for GenerateCapsule.py

usage: GenerateCapsule [-h] [-o OUTPUTFILE] (-e | -d | --dump-info)
                       [--capflag {PersistAcrossReset,PopulateSystemTable,InitiateReset}]
                       [--capoemflag CAPSULEOEMFLAG] [--guid GUID]
                       [--hardware-instance HARDWAREINSTANCE]
                       [--monotonic-count MONOTONICCOUNT]
                       [--fw-version FWVERSION] [--lsv LOWESTSUPPORTEDVERSION]
                       [--pfx-file SIGNTOOLPFXFILE]
                       [--signer-private-cert OPENSSLSIGNERPRIVATECERTFILE]
                       [--other-public-cert OPENSSLOTHERPUBLICCERTFILE]
                       [--trusted-public-cert OPENSSLTRUSTEDPUBLICCERTFILE]
                       [--signing-tool-path SIGNINGTOOLPATH] [--version] [-v]
                       [-q] [--debug [0-9]]
                       InputFile

Generate a capsule. Copyright (c) 2018, Intel Corporation. All rights
reserved.

positional arguments:
  InputFile             Input binary payload filename.

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUTFILE, --output OUTPUTFILE
                        Output filename.
  -e, --encode          Encode file
  -d, --decode          Decode file
  --dump-info           Display FMP Payload Header information
  --capflag {PersistAcrossReset,PopulateSystemTable,InitiateReset}
                        Capsule flag can be PersistAcrossReset, or
                        PopulateSystemTable or InitiateReset or not set
  --capoemflag CAPSULEOEMFLAG
                        Capsule OEM Flag is an integer between 0x0000 and
                        0xffff.
  --guid GUID           The FMP/ESRT GUID in registry format. Required for
                        encode operations.
  --hardware-instance HARDWAREINSTANCE
                        The 64-bit hardware instance. The default is
                        0x0000000000000000
  --monotonic-count MONOTONICCOUNT
                        64-bit monotonic count value in header. Default is
                        0x0000000000000000.
  --fw-version FWVERSION
                        The 32-bit version of the binary payload (e.g.
                        0x11223344 or 5678).
  --lsv LOWESTSUPPORTEDVERSION
                        The 32-bit lowest supported version of the binary
                        payload (e.g. 0x11223344 or 5678).
  --pfx-file SIGNTOOLPFXFILE
                        signtool PFX certificate filename.
  --signer-private-cert OPENSSLSIGNERPRIVATECERTFILE
                        OpenSSL signer private certificate filename.
  --other-public-cert OPENSSLOTHERPUBLICCERTFILE
                        OpenSSL other public certificate filename.
  --trusted-public-cert OPENSSLTRUSTEDPUBLICCERTFILE
                        OpenSSL trusted public certificate filename.
  --signing-tool-path SIGNINGTOOLPATH
                        Path to signtool or OpenSSL tool. Optional if path to
                        tools are already in PATH.
  --version             show program's version number and exit
  -v, --verbose         Turn on verbose output with informational messages
                        printed, including capsule headers and warning
                        messages.
  -q, --quiet           Disable all messages except fatal errors.
  --debug [0-9]         Set debug level

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-02 14:35:20 -07:00
Gary Lin e52b39ab8e BaseTools: Treat Ecc.py as a python module
Since Ecc.py import modules from its own directory, add "-m" to the
python parameters so that they can import its own modules after adopting
absolute import.

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-07-16 11:22:16 +08:00
Gary Lin 9f3594782d BaseTools: Treat BPDG.py as a python module
Since BPDG.py import modules from its own directory, add "-m" to the
python parameters so that they can import its own modules after adopting
absolute import.

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-07-16 11:22:15 +08:00
Gary Lin 15e2022825 BaseTools: Treat GenFds.py as a python module
Since GenFds.py import modules from its own directory, add "-m" to the
python parameters so that they can import its own modules after adopting
absolute import.

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-07-16 11:22:12 +08:00
Yonghong Zhu 83c6c3bfe2 BaseTools: Correct bin wrappers for GenDepex.py path
The WindowsLike and PosixLike bin wrappers for GenDepex.py do not use
correct path. GenDepex.py is not in a GenDepex directory. Instead, it is
in the AutoGen directory.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=292
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2016-12-20 09:15:38 +08:00
Liming Gao bf069759ef BaseTools: Add Windows batch files to run python tool from Source
Add 13 windows batch files for every python tool.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>
Cc: Erik Bjorge <erik.c.bjorge@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Erik Bjorge <erik.c.bjorge@intel.com>
2016-09-21 10:49:12 +08:00