Commit Graph

95 Commits

Author SHA1 Message Date
Fan, ZhijuX 401507c786 BaseTools:Add DetectNotUsedItem.py to Edk2\BaseTools\Scripts
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1850

This script is used to Detect unreferenced PCD and GUID/Protocols/PPIs.
The input parameters are Dec file and package directory.

This script can be run in both Py2 and Py3.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-06-25 09:23:48 +08:00
Leif Lindholm 4eb0acb1e2 BaseTools: add script to configure local git options
Patch contribution and review is greatly simplified by following the
steps described in "Laszlo's unkempt guide":
https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
but there are a lot of tedious manual steps in there, so here is a
python script that configures all options I am aware of
*for the repository the script is executed from*.

Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-12 10:10:11 +08:00
Fan, ZhijuX cfb29d2bda BaseTools:update ConvertFceToStructurePcd.py with the char order PCD name.
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1718

BaseTools\Scripts\ConvertFceToStructurePcd.py
Update script to sort the PCD order base on PcdName,
then base on Pcd field name.

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-04-18 08:55:21 +08:00
Michael D Kinney a281361014 BaseTools/PatchCheck: Generate error if Contributed-under found
https://bugzilla.tianocore.org/show_bug.cgi?id=1655

With the change to BSD+Patent License, the TianoCore Contributor's
Agreement has been removed and as a result, a Contributed-under
tag is no longer appropriate in patches.  Remove the check for
the TianoCore Contributor's Agreement and instead, generate an
error if a patch contains a Contributed-under tag in the commit
message.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2019-04-09 11:26:28 -07:00
Michael D Kinney 2e351cbe8e BaseTools: Replace BSD License with BSD+Patent License
https://bugzilla.tianocore.org/show_bug.cgi?id=1373

Replace BSD 2-Clause License with BSD+Patent License.  This change is
based on the following emails:

  https://lists.01.org/pipermail/edk2-devel/2019-February/036260.html
  https://lists.01.org/pipermail/edk2-devel/2018-October/030385.html

RFCs with detailed process for the license change:

  V3: https://lists.01.org/pipermail/edk2-devel/2019-March/038116.html
  V2: https://lists.01.org/pipermail/edk2-devel/2019-March/037669.html
  V1: https://lists.01.org/pipermail/edk2-devel/2019-March/037500.html

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-09 09:10:20 -07:00
Fan, ZhijuX 8ef3a6ec1f BaseTools:Run packagedoc_cli.py to generate doc failed
The reason for this problem is that the file was opened incorrectly.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-03-06 09:17:33 +08:00
Fan, ZhijuX 9ab4ec5188 BaseTools:PackageDocumentTools import lib error occurs.
Steps:
 1. Download edk2 tree
 2. Build BaseTools
 3. Go to edk2\BaseTools\Scripts\PackageDocumentTools
    to run packagedoc_cli.py

An error occurs if relative imports are used when running
a file alone

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-26 09:38:39 +08:00
Antoine Coeur fb0b35e05f BaseTools: Various typo
Various typo in BaseTools.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Coeur <coeur@gmx.fr>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-14 10:02:28 +08:00
Zhijux Fan 7cc7e054f8 BaseTools/Scripts: Porting PackageDocumentTools code to use Python3
Porting PackageDocumentTools code to support python2 and python3

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:19 +08:00
Liming Gao 591a44c02d BaseTools Script: Update ConvertFceToStructurePcd to report warning messages
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1297
When the header files are not found for the used C structure, this script will
report the warning, let user know there is no header file to define C structure.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Wang BinX A <binx.a.wang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-11-29 08:30:52 +08:00
Liming Gao ce283fd609 BaseTools ConvertFceToStructurePcd: Fix the array value with empty string
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-10-31 22:07:57 +08:00
Liming Gao 7b500c606a BaseTools: Remove the step to freeze python tool
https://bugzilla.tianocore.org/show_bug.cgi?id=1257
Binary python tool is not supported anymore. So, the freeze python tool
step is not required.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-10-19 08:21:47 +08:00
Liming Gao 1ccc4d895d Revert BaseTools: PYTHON3 migration
This reverts commit 6693f359b3c213513c5096a06c6f67244a44dc52..
678f851312.

Python3 migration is the fundamental change. It requires every developer
to install Python3. Before this migration, the well communication and wide
verification must be done. But now, most people is not aware of this change,
and not try it. So, Python3 migration is reverted and be moved to edk2-staging
Python3 branch for the edk2 user evaluation.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
2018-10-15 08:29:14 +08:00
Yunhua Feng 9d7e1e56a7 BaseTools/Scripts: Porting PackageDocumentTools code to use Python3
Porting PackageDocumentTools code to Python3
DoxyGen 1.8.6
wxpython 4.0.3

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:57:16 +08:00
Dandan Bi 6460513aa6 BaseTools/PatchCheck.py: Fix error when run with Python3
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1145

Currently run PatchCheck.py with Python3 will meet
following error:
.....
File "PatchCheck.py", line 554, in run_git
return Result[0].decode('utf-8', 'ignore') if Result[0] and
Result[0].find("fatal")!=0 else None
TypeError: a bytes-like object is required, not 'str'

This issue was introduce by commit:5ac4548cdf654.

This patch is to convert the str object of "fatal" to
byte object to fix this failure.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Dandan Bi <dandan.bi@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-09-06 13:41:35 +08:00
Jaben Carsey 5ac4548cdf PatchCheck - add error message for invalid parameter
Currently if an invalid parameter is passed, it gives a stack trace.
This changes it to an error message.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
2018-08-08 11:41:28 +08:00
Kinney, Michael D cd3a42640a BaseTools/BinToPcd: Open output file as text file
https://bugzilla.tianocore.org/show_bug.cgi?id=1069

Undo changes from following commit:

83964ebc5e

Change the open mode for the output file from 'wb' to 'w' so the
output file is written as a text file and not a binary file.

This resolves the issue where the text file was not writable from
Python 3.x and also removes b'' from output file when the string
was encoded as a bytearray.

Cc: YanYan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Tested-by:YanYan Sun <yanyan.sun@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-06 10:48:35 +08:00
Kinney, Michael D 83964ebc5e BaseTools/BinToPcd: Encode string returned from ByteArray()
https://bugzilla.tianocore.org/show_bug.cgi?id=1069

The ByteArray() method returns a string with the hex bytes of
a PCD value.  Make sure the string is always encoded as a string,
so it can be used to build a complete PCD statement string and be
written out to a file.  This change is required for Python 3.x
compatibility.

Cc: YanYan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-02 18:42:27 -07:00
Kinney, Michael D 76c09700ed BaseTools/BinToPcd: Fix Python 2.7.x compatibility issue
https://bugzilla.tianocore.org/show_bug.cgi?id=1042

Convert Buffer to type bytearray before converting to a
string of hex byte values so the type of items in Buffer is
consistent for both Python 2.7.x and Python 3.x.

Cc: YanYan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-01 15:27:56 -07:00
Liming Gao ef529e6ab7 BaseTools Script: Add the script to generate Structure PCD setting
Here is this script usage.
1. Build one platform.
2. Use FCE tool to read HII config from the generated FD image.
FCE read -i Platform.fd > Config.txt
3. Call the script to generate StructurePcd setting.
ConvertFceToStructurePcd.py -p Build\PlatformPkg\DEBUG_VS2015x86 \
-c Config.txt -o OutputDir
OutputDir directory has StructurePcd.dec, StructurePcd.dsc, StructurePcd.inf.
4. Refer to wiki https://github.com/lgao4/edk2/wiki/StructurePcd-Enable-Steps
to enable structure pcd in this platform.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-07-25 11:12:18 +08:00
Gary Lin 2add3cffb0 BaseTools: Use absolute import in Scripts
Based on "futurize -f libfuturize.fixes.fix_absolute_import

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-07-16 11:22:19 +08:00
Liming Gao f7496d7173 BaseTools: Clean up source files
1. Do not use tab characters
2. No trailing white space in one line
3. All files must end with CRLF

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-07-09 10:25:47 +08:00
Gary Lin 86379ac48b BaseTools: Replace StringIO.StringIO with io.BytesIO
Replace StringIO.StringIO with io.BytesIO to be compatible with python3.
This commit also removes "import StringIO" from those python scripts
that don't really use it.

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:28 +08:00
Gary Lin 0d1f5b2b5d BaseTools: Fix old python2 idioms
Based on "futurize -f lib2to3.fixes.fix_idioms"

* Change some type comparisons to isinstance() calls:
    type(x) == T -> isinstance(x, T)
    type(x) is T -> isinstance(x, T)
    type(x) != T -> not isinstance(x, T)
    type(x) is not T -> not isinstance(x, T)

* Change "while 1:" into "while True:".

* Change both

    v = list(EXPR)
    v.sort()
    foo(v)

and the more general

    v = EXPR
    v.sort()
    foo(v)

into

    v = sorted(EXPR)
    foo(v)

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:27 +08:00
Gary Lin ccaa7754a2 BaseTools: Adjust the spaces around commas and colons
Based on "futurize -f lib2to3.fixes.fix_ws_comma"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:25 +08:00
Gary Lin 27c4ceb41c BaseTools: Remove the deprecated hash_key()
Replace "has_key()" with "in" to be compatible with python3.
Based on "futurize -f lib2to3.fixes.fix_has_key"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:23 +08:00
Gary Lin 72443dd250 BaseTools: Refactor python print statements
Refactor print statements to be compatible with python 3.
Based on "futurize -f libfuturize.fixes.fix_print_with_import"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:21 +08:00
Gary Lin 5b0671c1e5 BaseTools: Refactor python except statements
Convert "except ... ," to "except ... as" to be compatible with python3.
Based on "futurize -f lib2to3.fixes.fix_except"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:31:30 +08:00
Gary Lin 00eb12a2c7 BaseTools: Fix a typo in ini.py
"if mis not None:" => "if m is not None:"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Jaben Carsey <jaben.carsey@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-06-27 16:30:31 +08:00
Kinney, Michael D 0c805f4f8f BaseTools/BinToPcd: Follow PEP-8 indent of 4 spaces
https://www.python.org/dev/peps/pep-0008/

Cc: Yanyan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-14 15:43:40 -07:00
Kinney, Michael D d0f946f30d BaseTools/BinToPcd: Update for Python 3 compatibility
Update to be compatible with both Python 2.x and Python 3.x.
Also return error code 1 when an error is detected to support
use of this tool in scripts.

Cc: Yanyan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-14 15:43:30 -07:00
Kinney, Michael D 8e96580999 BaseTools/BinToPcd: --offset must be 8-byte aligned
https://bugzilla.tianocore.org/show_bug.cgi?id=974
https://bugzilla.tianocore.org/show_bug.cgi?id=965

Update help to state that --offset must be 8-byte aligned.
Verify that --offset is 8-byte aligned and print an error
message if it is not 8-byte aligned.

Cc: Yanyan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-14 15:43:22 -07:00
Kinney, Michael D b2e043b689 BaseTools/BinToPcd: Clarify error message for --type HII
https://bugzilla.tianocore.org/show_bug.cgi?id=963

Update error message for --type HII.  If either --variable-guid
or --variable-name is missing, then print an error message that
states that both --variable-guid and --variable-name are required.

Cc: Yanyan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-14 15:43:15 -07:00
Kinney, Michael D 1d79b72ee6 BaseTools/BinToPcd: Fix typo in error messages
https://bugzilla.tianocore.org/show_bug.cgi?id=962

Change "PcdToBin" to "BinToPcd"

Cc: Yanyan Sun <yanyan.sun@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-14 15:43:06 -07:00
Liming Gao c30084fbac BaseTools Script: Formalize source files to follow DOS format
V3:
support exclude dir and file by name while traversing the directory.
remove close in with statement.

V2:
add version,description,copyright.
add flag -v,-q,--append-extensions,--override-extensions,--debug.
-q will omit default output,-v and --debug are not implemented.
add default file extensions.
support input of file path.
support muliple input path.
simplify comment.
change 'pattern'.encode() to b'pattern',I think this will be better.
change naming of variable and function to keep the same with BinToPcd.py

V1:
FormatDosFiles.py is added to clean up dos source files. It bases on
the rules defined in EDKII C Coding Standards Specification.
5.1.2 Do not use tab characters
5.1.6 Only use CRLF (Carriage Return Line Feed) line endings.
5.1.7 All files must end with CRLF
No trailing white space in one line. (To be added in spec)

The source files in edk2 project with the below postfix are dos format.
.h .c .nasm .nasmb .asm .S .inf .dec .dsc .fdf .uni .asl .aslc .vfr .idf
.txt .bat .py

The package maintainer can use this script to clean up all files in his
package. The prefer way is to create one patch per one package.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Signed-off-by: Dongao Guo <dongao.guo@intel.com>
2018-06-13 13:03:16 +08:00
Carsey, Jaben 98120f5fe2 BaseTools: refactor and remove out of date use of .keys()
this is no longer required to make dictionary objects iterable.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-04-10 10:05:16 +08:00
Carsey, Jaben 4231a8193e BaseTools: Remove equality operator with None
replace "== None" with "is None" and "!= None" with "is not None"

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-03-30 08:25:13 +08:00
Kinney, Michael D aedd1559cc BaseTools/BinToPcd: Add support for multiple binary input files
https://bugzilla.tianocore.org/show_bug.cgi?id=890

There are use cases where a VOID * PCD needs to be generated from multiple
binary input files.  This can be in the form of an array of fixed size
elements or a set of variable sized elements.

Update BinToPcd to support multiple one or more -i INPUTFILE arguments.
By default, the contents of each binary input file are concatenated in
the order provided.  This supports generating a PCD that is an array of
fixed size elements

Add -x, --xdr flags to BinToPcd  to encodes the PCD using the
Variable-Length Opaque Data of RFC 4506 External Data Representation
Standard (XDR).

    https://tools.ietf.org/html/rfc4506
    https://tools.ietf.org/html/rfc4506#section-4.10

The data format from RFC 4506 meets the requirements for a PCD that is a
set of variable sized elements in the Variable-Length Opaque Data format.
The overhead of this format is a 32-bit length and 0 to 3 bytes of padding
to align the next element at a 32-bit boundary.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-03-28 08:51:15 -07:00
Yonghong Zhu 7ccc9c954c BaseTools: Add PackageDocumentTools into Scripts folder
This tool is used to generate the document for edk2 packages. The
generated document will be in UDK release. For example, UDK2017
document can be found in:
https://github.com/tianocore/tianocore.github.io/wiki/UDK2017#documentation

Cc: Star Zeng <star.zeng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
2018-03-19 09:19:36 +08:00
Michael D Kinney ef190542b4 BaseTools/Scripts: Add sample makefile for use with RunMakefile.py
https://bugzilla.tianocore.org/show_bug.cgi?id=670

Add sample makefile that can be used to test RunMakefile.py
script and can also be used as a template to start a new
PREBUILD/POSTBUILD makefile.

This makefile contains TAB characters instead of spaces on
purpose to maximize compatibility with make utilities.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2017-08-11 11:28:35 -07:00
Michael D Kinney c8d385265f BaseTools/Scripts: Add python script to run a makefile
https://bugzilla.tianocore.org/show_bug.cgi?id=670

Add the python script RunMakefile.py that can be used
in a PREBUILD/POSTBUIILD action to invoke a makefile
passing in context as makefile defines.  The command
line arguments passed into RunMakefile.py are converted
to the following set of defines.

* ACTIVE_PLATFORM
* TARGET_ARCH
* TOOL_CHAIN_TAG
* CONF_DIRECTORY
* TARGET
* EXTRA_FLAGS

In addition, a makefile can access the system environment
variables including WORKSPACE and PACKAGES_PATH.

The makefile target from the following set is also passed
into the makefile.  If no target is passed into build, then
the 'all' target is used.

[all|fds|genc|genmake|clean|cleanall|cleanlib|modules|libraries|run]

A platform DSC file can use a statements in the [Defines]
section of the following form to use this script.  MAKEFILE
is a WORKSPACE or PACKAGES_PATH relative path to the makefile
to run.

[Defines]
  PREBUILD  = python BaseTools/Script/RunMakefile.py --makefile MAKEFILE
  POSTBUILD = python BaseTools/Script/RunMakefile.py --makefile MAKEFILE

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2017-08-11 11:21:04 -07:00
Michael D Kinney f2d1b866df BaseTools/PatchCheck: Support Contribution Agreement 1.1
https://bugzilla.tianocore.org/show_bug.cgi?id=628

Update PatchCheck.py to support either
"Contributed-under: TianoCore Contribution Agreement 1.0"
or "Contributed-under: TianoCore Contribution Agreement 1.1"
in the commit message.

Temporarily continue to allow the TianoCore Contribution
Agreement 1.0 agreement.

Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Andrew Fish <afish@apple.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
2017-08-03 11:01:46 -07:00
Hao Wu 6a69dd4937 BaseTools/PatchCheck.py: Add warning info for new binary files
The commit adds the detection of adding new binary files in a patch file
or in a commit.

The following warning messages will be appended at the end of the script
output:

WARNING - The following binary files will be added into the repository:
  <BinaryFile1>
  <BinaryFile2>
  ...

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-06-26 09:34:11 +08:00
Hao Wu aab57eff5a BaseTools/PatchCheck.py: Fix misreport for binary changes in patch
For a patch file that:
1. Contains a binary change
2. Contains any other changes after the binary change

PatchCheck.py will complains with the following error:
* Patch format error: diff found after end of patch
   Line: literal XXXX

This commit resolves this misreport.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Hao Wu <hao.a.wu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-06-26 09:26:56 +08:00
Ard Biesheuvel f4d3ba87bb BaseTools/Scripts: fix GccBase.lds line endings
Replace a <LF> line ending that was introduced inadvertently by a
recent commit with the correct <CR><LF>

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
2017-05-28 12:07:02 +00:00
Ard Biesheuvel 00b00cc57b BaseTools/Scripts: discard .gnu.hash section in GCC builds
Some builds of GCC/binutils will default to using the GNU flavor of
the symbol hash table, and will emit it into a section called .gnu.hash
rather than .hash. We have no use for its contents, and GenFw ignores
it anyway, so it shouldn't really matter what we do with it.

However, due to a workaround for AARCH64 we have in GenFw to deal with
older GCCs that corrupt section-based relocations when merging sections
during the final link, we need the ELF and PE/COFF views of the binary
to be identical. Since we don't place the .gnu.hash section explicitly,
it may end up at the beginning of the ELF binary, causing other sections
to be shifted in the ELF view but not in the PE/COFF view.

So let's add .gnu.hash to the GCC linker script. We don't care about its
contents so add it to the /DISCARD/ section.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Tested-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-05-24 06:09:39 -07:00
Ard Biesheuvel 3cf41b8728 BaseTools: GCC: move most AutoGen.obj contents back to .data section
The generated AutoGen.c files mostly contain read-only data, but due to
lacking annotations, all of it is emitted into the .data section by the
compiler.

Given that GUIDs are UEFI's gaffer tape, having writable GUIDs is a
security hazard, and this was the main rationale for putting AutoGen.obj
in the .text section. However, as it turns out, patchable PCDs are emitted
there as well, which can legally be modified at runtime.

So update the wildcard pattern to only match g...Guid sections, and move
everything else back to .data (Note that this relies on -fdata-sections,
without that option, everything is emitted into .data)

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Fixes: 233bd25b00
[lersek@redhat.com: add reference to previous commit being fixed up]
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
2017-02-24 11:27:56 +01:00
Jiewen Yao 88dab294d2 BaseTool/Script: Add SmiHandleProfile OS tool to get symbol.
This tool accepts the input XML file generated by SmiHandlerProfile
application and convert the RVA address to be a user readable
symbol.
It also converts the GUID to be a user readable string.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-02-22 15:28:22 +08:00
Michael Kinney fd0597aa9c BaseTools/Scripts: Add BinToPcd utility
https://bugzilla.tianocore.org/show_bug.cgi?id=228

Add a utility that converts a binary file into a VOID* PCD value
or a full DSC file VOID* PCD statement with support for all the
DSC supported PCD sections.

usage: BinToPcd [-h] [--version] -i INPUTFILE [-o OUTPUTFILE] [-p PCDNAME]
                [-t {VPD,HII}] [-m MAXSIZE] [-f OFFSET] [-n VARIABLENAME]
                [-g VARIABLEGUID] [-v] [-q] [--debug [0-9]]

Convert a binary file to a VOID* PCD value or DSC file VOID* PCD statement.
Copyright (c) 2016, Intel Corporation. All rights reserved.

optional arguments:
  -h, --help        show this help message and exit
  --version         show program's version number and exit
  -i INPUTFILE, --input INPUTFILE
                    Input binary filename
  -o OUTPUTFILE, --output OUTPUTFILE
                    Output filename for PCD value or PCD statement
  -p PCDNAME, --pcd PCDNAME
                    Name of the PCD in the form
                    <PcdTokenSpaceGuidCName>.<PcdCName>
  -t {VPD,HII}, --type {VPD,HII}
                    PCD statement type (HII or VPD). Default is standard.
  -m MAXSIZE, --max-size MAXSIZE
                    Maximum size of the PCD. Ignored with --type HII.
  -f OFFSET, --offset OFFSET
                    VPD offset if --type is VPD. UEFI Variable offset if
                    --type is HII.
  -n VARIABLENAME, --variable-name VARIABLENAME
                    UEFI variable name. Only used with --type HII.
  -g VARIABLEGUID, --variable-guid VARIABLEGUID
                    UEFI variable GUID C name. Only used with --type HII.
  -v, --verbose     Increase output messages
  -q, --quiet       Reduce output messages
  --debug [0-9]     Set debug level

This utility can be used in PCD value mode to convert a binary
file into a string that can then be copied into the PCD value field
of a VOID* PCD.  The following is an example of PCD value mode on
an 8 byte test.bin file.

  BinToPcd.py -i test.bin

  {0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}

The DSC file VOID* PCD statement mode can be used to generate a
complete PCD statement for the PCD section types that a DSC file
supports:

  [PcdsFixedAtBuild]
  [PcdsPatchableInModule]
  [PcdsDynamicDefault]
  [PcdsDynamicExDefault]
  [PcdsDynamicVpd]
  [PcdsDynamicExVpd]
  [PcdsDynamicHii]
  [PcdsDynamicExHii]

The PCD statement mode is useful when combined with a !include
statement in a DSC file.  BinToPcd.py can be used to convert a
binary file to a PCD statement in an output file, and that output
file can be included into a DSC file in the matching PCD section
to set the value of the PCD to the value from the binary file
without having to copy the value into the DSC file.  Updates can be
made to the included file without editing the DSC file.  Some
example use cases are the setting the public key PCDs such as:

  gEfiSecurityPkgTokenSpaceGuid.PcdRsa2048Sha256PublicKeyBuffer
  gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBuffer

The following example converts a public key binary file to a
[PcdsFixedAtBuild] compatible PCD statement:

  BinToPcd.py -i PublicKey.bin -o PublicKey.pcd
    --pcd gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBufferkenSpaceGuid

The PublicKey.pcd output file contains a single line:

  gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBuffer|{0x48, ...}

A DSC file can be updated to include the PublicKey.pcd file:

  [PcdsFixedAtBuild]
  !include PublicKey.pcd

Value examples
===============
  BinToPcd.py -i test.bin
    {0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}

Normal examples:
=================
  BinToPcd.py -i test.bin -p Guid.Token
    Guid.Token|{0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}

  BinToPcd.py -i test.bin -p Guid.Token -m 20
    Guid.Token|{0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}|VOID*|20

VPD examples:
=============
  BinToPcd.py -i test.bin -p Guid.Token -t VPD
    Guid.Name|*|8|{0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}

  BinToPcd.py -i test.bin -p Guid.Token -t VPD -f 20
    Guid.Name|20|8|{0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}

  BinToPcd.py -i test.bin -p Guid.Token -t VPD -m 10
    Guid.Name|*|10|{0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}

  BinToPcd.py -i test.bin -p Guid.Token -t VPD -f 20 -m 10
    Guid.Name|20|10|{0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x0d, 0x0a}

HII examples:
=============
  BinToPcd.py -i test.bin -p Guid.Token -t HII -g VarGuid -n VarName
    Guid.Name|L"VarName"|VarGuid|0|{0x48, 0x65, 0x6c, 0x6c}

  BinToPcd.py -i test.bin -p Guid.Token -t HII -g VarGuid -n VarName -f 8
    Guid.Name|L"VarName"|VarGuid|8|{0x48, 0x65, 0x6c, 0x6c}

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Michael Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2016-11-14 08:40:41 -08:00
Yonghong Zhu c3926cdbbd BaseTools/PatchCheck.py: Update to report error for EFI_D_*
In EDK2, DEBUG_* is recommended to be used instead of EFI_D_*. For new
code, they should use DEBUG_* macro.

Fixes:https://bugzilla.tianocore.org/show_bug.cgi?id=143
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-10-21 07:28:01 +08:00