Commit Graph

1231 Commits

Author SHA1 Message Date
Liming Gao fcdedafd97 Revert "BaseTools: Improve GetDependencyList function"
This reverts commit bc9e4194cf.
This change causes the dependent header files are missing in Makefile.
It makes the incremental build not work. So, revert this change.

Cc: Bob Feng<bob.c.feng@Intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@Intel.com>
2019-09-21 00:25:48 +08:00
Fan, ZhijuX 85ccbee2ab BaseTools:Fix the issue that build report failed
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2201

An error occurs using special VpdPcd that is not used in the Inf file

In dsc:
[PcdsDynamicExVpd.common.DEFAULT]
gBoardModuleTokenSpaceGuid.test1|*|{CODE({
  {0x0}  // terminator
})}

In dec:
[PcdsDynamicEx]
# Vpd GPIO table
  gBoardModuleTokenSpaceGuid.test1|{0}|GPIO_INIT_CONFIG[]|0x50000018 {
  <HeaderFiles>
    Library/GpioLib.h
  <Packages>
    MdePkg/MdePkg.dec
}
ValueError: invalid literal for int() with base 0: '*'

This Patch is going to fix issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-19 17:11:57 +08:00
Steven Shi b0c15fb128 BaseTools: Add more parameter checking for CopyFileOnChange()
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2193

The current CopyFileOnChange() method in Misc.py does not
accept the input SrcFile parameter as a dir, but the method
does not check the SrcFile is dir or not. This patch is to
add more input parameter type checking and error message output
for method CopyFileOnChange.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-19 08:48:31 +08:00
Feng, Bob C 22e75231ae BaseTools:Remove the unnecessary operation of renaming a file.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2183

This patch is going to remove rename call to reduce unnecessary
io operation so that saving build time.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Feng, Bob C 0075ab2cec BaseTools: remove unnecessary calls of os.exist
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2101

This patch is going to remove unnecessary calls
of os.exist()

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Feng, Bob C bc9e4194cf BaseTools: Improve GetDependencyList function
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2102

GetDependencyList get the header file via
re.findall in the whole header file.

This patch is to pre-process the header file and
to feed the shorter string to re.findall.

This patch is to improve GetDependencyList() efficiency

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Fan, ZhijuX e32f7bc96d BaseTools:change some incorrect parameter defaults
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1858

for Dict={},There are pitfalls in the way this default parameter is set
and Dict is not used in functions, other functions have these two cases,
I will change some incorrect parameter defaults

This patch is going to fix this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-17 10:18:51 +08:00
Fan, ZhijuX b67735a7e8 BaseTools:Replace PlatformInfo with PlatformAutoGen for Moudle
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2188

build -p MdeModulePkg\MdeModulePkg.dsc -a IA32 -m
MdeModulePkg\Universal\PCD\Pei\Pcd.inf

Error:
AttributeError: 'PlatformInfo' object has no attribute
'DynamicPcdList'

The DSC data object used to build a separate module today
is PlatformInfo rather than PlatformAutoGen
'PlatformAutoGen' object has attribute 'DynamicPcdList'

This patch is going to fixed this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-17 10:18:51 +08:00
Feng, Bob C 5a9db85880 BaseTools: Fixed a bug of IgnoreAutoGen
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2080
After checking that if the build can't
ignore Autogen due to there is no compelet autogen files,
the build tool need to do a completely Autogen.

This patch is to fix a bug that if AutoGen
can't be skiped, the SkipAutoGen flag
need to set to False

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-12 14:08:15 +08:00
Feng, Bob C 7809492c10 BaseTools: Enable --genfds-multi-thread to default build
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302

This patch enable --genfds-multi-thread to default build.
This patch keep --genfds-multi-thread build option for
compatibility and also add a new build option to disable
genfds-multi-thread as --no-genfds-multi-thread.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-11 09:43:59 +08:00
Feng, Bob C f23da86415 BaseTools: Fixed the build fail on Linux with --genfds-multi-thread
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302

If GenSec input file not exist, the related command will fail and make
will stop on Linux. GenSec input file is allow to be non-existent.

This patch is to let "make" continue if gensec input file not exist.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-11 09:43:59 +08:00
Feng, Bob C e3c8311f8a BaseTools: Fixed the bug of multi-thread genffs for override inf
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302
If there is a inf override, and multi-thread genffs is enabled, build
will fail.

This patch is going to fix this bug.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-11 09:43:58 +08:00
Feng, Bob C 000ab98574 BaseTools: Sort Pcd settings to make PcdTokenNumber be fixed
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2147

This patch is to sort the Pcd settings so that PcdTokenNumber
will not change if the platform's Pcd settings are the same.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-10 08:39:19 +08:00
Feng, Bob C 373298ca0d BaseTools: Fixed issue for IgnoreAutoGen
https://bugzilla.tianocore.org/show_bug.cgi?id=2080

This patch is to improve build -u option to re-use
GlobalVar_<platformguid>_<arch>.bin file which is
introduced by multiple-process-autogen feature.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-09-10 08:39:18 +08:00
Feng, Bob C 53e2eaba7a BaseTools: Fixed build clean regression issue
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2166

file_lock and cache_lock are used as global data,
so move file_lock and cache_lock initialization in Build
object __init__ function.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-10 08:39:17 +08:00
Bob Feng cbe7543ebb BaseTools: Fix a bug for Hii Pcd override
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2157

Hii Pcd links to a efi variable. The Variable
default value is evaluated by related Hii Pcds
setting. If multiple Hii Pcds links to one variable,
and the offset overlap, the later Hii Pcds setting
should be effective. There is a tool bug that is if
the Pcds are in different dsc file which are included
into the platform dsc file, build tool does not get
the Pcds relative position correctly. That means
build tool does not know which Pcd is the later one. As
the result, the variable default value will be incorrect.

This patch is to fix this bug.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-10 08:39:16 +08:00
Fan, ZhijuX d6f5a5050b BaseTools:ECC need to handle lower case 'static'
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1778

We are going to add keyword (lower case) 'static' for functions and global
variables which are not referenced outside their current C file.

However, the ECC tool only recognizes upper case 'STATIC' at this moment.
This will lead to issue reports for new codes that follow the above coding
style.

This patch is going to handle lower case 'static'

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-09 11:14:42 +08:00
Fan, ZhijuX 03ad3d889c BaseTools:Ecc handle another copyright format
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2057

Ecc can not handle the copyright format like

(C) Copyright 2015-2016 Hewlett Packard Enterprise Development LP<BR>

This will cause Ecc to report wrong information.

This patch is going to handle this format

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-09 11:13:52 +08:00
Shi, Steven 40db176d9d BaseTools: Support more file types in build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1958

Current build cache does not store and restore all
types file of Hii/vfr, version and dpx. This patch
adds more file types to support them in build cache.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-02 12:17:56 +08:00
Feng, Bob C 37eef91017 BaseTools: Fixed regression issue for building !x86 builds
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2134

This patch is to fix a build tool regression issue which
was introduced by commit e8449e1d8e.

In commit e8449e1d8e, build tool check the pcd before
filter out the irrelevant library instance.

The logic of evaluating the priority of the library class
resolutions was not changed.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Tested-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Acked-by: Leif Lindholm <leif.lindholm@linaro.org>
Tested-by: Leif Lindholm <leif.lindholm@linaro.org>
2019-08-29 11:10:29 +08:00
Feng, Bob C df43ea6c08 BaseTools: fix an incremental build issue.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2137

This patch is going to fix a regression issue of incremental
build which was introduced by the commit 94459080c.
The changing on INF file will not trigger module rebuild.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Tested-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-29 11:07:48 +08:00
Feng, Bob C abc0155b03 BaseTools: Incorrect error message for library instance not found
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2099
This is a regression issue introduced by commit e8449e.

This patch is to fix this issue.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-23 11:26:24 +08:00
Feng, Bob C 5f7bb39117 BaseTools: Fix incremental build genmake issue
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2090

This is a regression issue introduced by commit e8449e.
This patch is going to fix this issue.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-23 11:26:24 +08:00
Shi, Steven 6dd9aa4019 BaseTools: Support long file path in windows for misc functions
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2103

Current CopyFileOnChange() and SaveFileOnChange() in
BaseTools\Source\Python\Common\Misc.py don't use the dedicated
long file path API to handle the file path strings and cannot
support the long file path copy and save in windows. This patch
enhances them to support the long file path copy and save
correctly.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-23 11:26:23 +08:00
Fan, ZhijuX 0970a80583 BaseTools: Update incorrect variable name 'DataPile'
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2093

The PlatformAutoGen object has a DataPipe property but no DataPile property
So change the variable name 'DataPile' to 'DataPipe' in BuildReport.py

This patch is going to fix that issue.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-20 14:15:47 +08:00
Shi, Steven 94459080c1 BaseTools: Improve the file saving and copying reliability
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2079

The Basetool CopyFileOnChange() and SaveFileOnChange()
functions might raise the IOError occasionally when build
in Windows with multi-process and build cache enabled.
The CopyFileOnChange() and SaveFileOnChange() might be invoked
in multiple sub-processes simultaneously, and this patch adds
global locks to sync these functions invoking which can
harden their reliability.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:07 +08:00
Shi, Steven d01a998612 BaseTools: Add GenFds multi-thread support in build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1923

Fix the issue that the GenFds multi-thread will build fail
if enable the build cache together.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:07 +08:00
Shi, Steven c340c5bd1b BaseTools: Change the [Arch][Name] module key in Build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1951

Current build cache use the module's [self.Arch][self.Name]
info as the ModuleAutoGen object key in hash list and dictionary.
The [self.Arch][self.Name] is not safe as the module key because
there could be two modules with same module name and arch name in
one platform. E.g. A platform can override a module or library
instance in another different path, the overriding module can has
the same module name and arch name as the original one.
Directly use the ModuleAutoGen obj self as the key, because
the obj __hash__ and __repr__ attributes already contain the
full path and arch name.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:06 +08:00
Shi, Steven 56c786b0a9 BaseTools: Print first cache missing file for build cachle
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925

When a module build cache miss, add support to print the first
cache missing file path and name.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:06 +08:00
Shi, Steven 0e7e7a264c BaseTools: Improve the cache hit in the edk2 build cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1927

Current cache hash algorithm does not parse and generate
the makefile to get the accurate dependency files for a
module. It instead use the platform and package meta files
to get the module depenedency in a quick but over approximate
way. These meta files are monolithic and involve many redundant
dependency for the module, which cause the module build
cache miss easily.
This patch introduces one more cache checkpoint and a new
hash algorithm besides the current quick one. The new hash
algorithm leverages the module makefile to achieve more
accurate and precise dependency info for a module. When
the build cache miss with the first quick hash, the
Basetool will caculate new one after makefile is generated
and then check again.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-20 14:06:05 +08:00
Feng, Bob C 76e12fa334 BaseTools: Fixed issue of incorrect Module Unique Name
https://bugzilla.tianocore.org/show_bug.cgi?id=2088
If there are more than one override instance for a same module,
the Module Unique Name is generated incorrectly.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-19 09:06:14 +08:00
Jin, Eric 48d8d4d80b BaseTools/Capsule: Tool to generate Windows Firmware Update Driver
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1837

The tool is designed to generate Windows Firmware Update Drivers,
the input is one drivername.cap with related parameters, the output
Windows Driver package are composed by drivername.cap, drivername.inf
and drivername.cat to update the single payload in device.

usage:
GenerateWindowsDriver [-h] [--output-folder OUTPUTFOLDER]
                      [--product-fmp-guid PRODUCTFMPGUID]
                      [--capsuleversion-dotstring CAPSULEVERSION_DOTSTRING]
                      [--capsuleversion-hexstring CAPSULEVERSION_HEXSTRING]
                      [--product-fw-provider PRODUCTFWPROVIDER]
                      [--product-fw-mfg-name PRODUCTFWMFGNAME]
                      [--product-fw-desc PRODUCTFWDESC]
                      [--capsule-file-name CAPSULEFILENAME]
                      [--pfx-file PFXFILE] [--arch ARCH]
                      [--operating-system-string OPERATINGSYSTEMSTRING]

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Kinney Michael D <michael.d.kinney@intel.com>
Signed-off-by: Eric Jin <eric.jin@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-15 20:13:55 +08:00
Sean Brogan e2aacac580 BaseTools/Capsule: Add WindowsCapsuleSupportHelper
Add initial version WindowsCapsuleSupportHelper.py
original source comes from
https://github.com/microsoft/mu_basecore/blob/dev/201905/BaseTools/Plugin/WindowsCapsuleSupportHelper/WindowsCapsuleSupportHelper.py
commit ebd9aad90dee3a3094b575928670cf81b8c685b5
Convert EOL to CRLF

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Eric Jin <eric.jin@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-15 20:13:05 +08:00
Bob Feng 08a54c9e0a BaseTools: Move buildoptions.py to Common folder
buildoptions is used by build.py and TargetTxtClassObj.py
Move buildoptions.py to Common folder so that
build tool and ECC tool work fine.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-14 13:54:49 +08:00
Rodriguez, Christian 82407bd129 BaseTools: Fix checking for Sources section in INF file
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804

The check to see if [Sources] section lists all the header type
files of a module is missing the exclusion of source files that
fall under the scope of Package includes. This change adds the
exclusions.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-08-14 10:59:08 +08:00
Feng, Bob C 2b4c07bc22 BaseTools: Add "-" in a re to parse line in .map file
https://bugzilla.tianocore.org/show_bug.cgi?id=2058

If there are multiple module instances with different
guid used in .dsc file, the temp path of the module will
contain 8-4-4-4-12 format guid string. So "-" need to be
added into the regular expression for parsing it correctly.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-14 10:24:36 +08:00
Feng, Bob C a000d5d1ca BaseTools: Fix incremental build bug
https://bugzilla.tianocore.org/show_bug.cgi?id=2078

This is a build tool regression issue that
is introduced by multiple-process-autogen.

The workspace timestamp value is not pass
to module autogen object correctly, so build tool
does not detect the change of .dsc and ignore the autogen.

This patch is to fix this issue.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Tested-by: Michael D Kinney <michael.d.kinney@intel.com>
2019-08-14 10:23:25 +08:00
Feng, Bob C a0792697bc BaseTools: Fixed an issue that build raise exception
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2047

Build raise exception if there is libraryclass
missing in dsc file and --pcd build option is used.

Build should report the missing libraryclass but not
a call stack.

This patch is going to fix it.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-12 10:40:24 +08:00
Feng, Bob C 1a624dd7cf BaseTools: Enable block queue log agent.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

To support Ctrl+S and Ctrl+Q, we enable block queue
for log.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:55 +08:00
Feng, Bob C 4acae2b38e BaseTools: Add the support for python 2
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

python3 change the module name of Queue to queue.
python3 add a new log handler of QueueHandler.

This patch is to make Multiple process AutoGen
feature work for python2

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:55 +08:00
Feng, Bob C c60fb00f6c BaseTools: Move BuildOption parser out of build.py
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

Build tool supports user to specify the conf folder.
To make the build options be evaluated at the beginning
of launching build, extract the buildoption function
from build.py to a new .py file.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:54 +08:00
Feng, Bob C 636ed13a7f BaseTools: Add LogAgent to support multiple process Autogen
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

AutoGen processes race the logfile. To resolve this issue,
this patch create a LogAgent thread in main process to write
the log content to console or file, Other process will send
the log content to the LogAgent.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:54 +08:00
Feng, Bob C 3285fbda88 BaseTools: Add shared data for processes
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

Add shared data for autogen processes.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:53 +08:00
Feng, Bob C 673d09a2dd BaseTools: Enable Multiple Process AutoGen
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

Assign the Module AutoGen tasks into multiple
sub process.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:53 +08:00
Feng, Bob C e8449e1d8e BaseTools: Decouple AutoGen Objects
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

1. Separate the AutoGen.py into 3 small py files.
One is for AutoGen base class, one is for WorkspaceAutoGen class
and PlatformAutoGen class, and the one for ModuleAutoGen class.
2. Create a new class DataPipe to store the Platform scope settings.
Create a new class PlatformInfo to provide the same interface
as PlatformAutoGen. PlatformInfo class is initialized by
DataPipe instance.
Create a new class WorkspaceInfo to provide the same interface
as WorkspaceAutoGen. WorkspaceInfo class is initialized by
DataPipe instance.
3. Change ModuleAutoGen to depends on DataPipe, PlatformInfo and
WorkspaceInfo. Remove the dependency of ModuleAutoGen to PlatformAutoGen.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:52 +08:00
Feng, Bob C 197ca7febf BaseTools: Add functions to get platform scope build options
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

These functions are used for get platform scope
build options. They will be used in later patches.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:52 +08:00
Feng, Bob C 2927a6fd99 BaseTools: Split WorkspaceAutoGen._InitWorker into multiple functions
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

The WorkspaceAutoGen.__InitWorker function is too long, it's hard
to read and understand.
This patch is to separate the __InitWorker into multiple small ones.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:51 +08:00
Feng, Bob C db01c8e3d8 BaseTools: Singleton the object to handle build conf file
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

The build config files are target.txt, build rule, tooldef
During a build, the config is not changed, so the object to
handle them need to be singleton.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
2019-08-09 23:15:51 +08:00
Feng, Bob C 83d6207f99 BaseTools: Fixed a typo in Trim.py
This is a regression issue introduced
by commit 307e1650be

This patch is to fix this issue.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-02 00:18:00 +08:00
Matthew Carlson 101dc9e257 BaseTools:Added arch output to build report
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2016

Added arch output to build report so it's easy to tell
which arch is being generated. Useful when multiple versions
of a single module is being emitted for multiple archs.

This patch is going to Added arch output

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-01 15:36:49 +08:00
Bob Feng 8ddec24dea BaseTools:Updata the output encoding of the Popen function
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2015

Not all output works in utf-8, so change the encoding to
the default

This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-01 15:36:48 +08:00
Max Knutsen 307e1650be BaseTools:Add extra debugging message
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2014

Add extra debugging to improve error identification.
Error while processing file if the file is read incorrectly

This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-01 15:36:48 +08:00
Sean Brogan 76912197fa BaseTools:replace the chinese quotation mark with unicode "
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2013

chinese quotation mark are used in the file
This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-08-01 15:36:47 +08:00
Feng, Bob C 959791dd6c BaseTools: Sort file list in Makefile
This patch is going to sort the file list in generated
Makefile. This change make the autogen makefile easy to
compare.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
2019-07-25 09:00:59 +08:00
Feng, Bob C bb824f685d BaseTools: Fixed the mis-using strip() function issue.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2003

lstrip(parameter) do the match based on the char
in parameter but not only the whole parameter string.

In GenMake line 1082,
CmdSign.lstrip('/Fo') will strip the '/' or
'F' or 'o' on the left of CmdSign. This is not expected.

This patch is going to fix such issue.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-07-22 12:06:03 +08:00
Feng, Bob C 5f89bcc460 BaseTools: Create ".cache" folder when initialize Build object
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1986

Create "Conf/.cache" folder as early as possible
so that the later code do need to check
if it exits and then create it.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-07-22 09:21:14 +08:00
Cole Robinson eebc135ffb BaseTools: Fix python3.8 SyntaxWarning
Building with python3.8 shows a warning like:

SyntaxWarning: invalid escape sequence \(
  GuidName = re.compile("\(GUID=[-a-fA-F0-9]+")

It seems harmless, but it's easy enough to fix: mark the string as
raw with the 'r' prefix like is used elsewhere in the file

Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-07-15 09:09:25 +08:00
Jiewen Yao a79841a024 BaseTools: Add HOST_APPLICATION module type.
It can be used to indicate a module can be build to run
as OS application and run in OS environment.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Xiaoyu Lu <xiaoyux.lu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-07-12 21:06:57 +08:00
Bob Feng 2d100d1d73 BaseTools: Fixed the issue when ToolDefinitionFile is not generated
ToolDefinitionFile is generated by PlatformAutoGen.ToolDefinition()
Code assume ToolDefinition is always called before using
ToolDefinitionFile, but in some cases, it's not true.

This patch is to fix this issue.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-07-12 17:11:58 +08:00
Feng, Bob C 688ec2d006 BaseTools: Detect the change of env variable used in tooldef.txt
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1921

Add TOOLS_DEF.ARCH file to all workspace meta files list.
TOOLS_DEF.ARCH include the evaluated Tool definition information
which is filtered by current tool_chain.

With this change, when the environment variable which
is used in ToolDef.txt is changed, build tool will
rebuild the platform.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-07-10 09:29:43 +08:00
Feng, Bob C 1d06b46cf1 BaseTools: Fixed the issue of the CFlag for compile PcdValueInit.c
This issue happens when using Structured PCD.
Build tool use set to store the cflags for compile PcdValueInit.c,
that is the order of cflags is disorder.

This patch make -U, /U flags appear before -D, /D

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-07-10 09:26:18 +08:00
Antoine Cœur 2d53d54a5b BaseTools: Fix various typos
Fix various typos in BaseTools.

Signed-off-by: Cœur <coeur@gmx.fr>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-07-08 08:59:29 +08:00
Jin, Eric 104a1aa19b BaseTools/Capsule: Supports multiple payloads and drivers in capsule
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1834

1)Add arguments "--embedded-driver" to support embedded driver
in command line.
2)Add arguments "--update-image-index" to identify ImageIndex
within the device in command line.
3)Add arguments "-j JSONFILE" to support multiple payloads and
embedded drivers with JSON file.

The update is in a backwards compatible manner, so all command
line options to support single payload are still supported. But
all the options associated with multiple payloads should be
provided in a JSON file.

Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Eric Jin <eric.jin@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-28 11:39:06 +08:00
Bob Feng b8ac0b7f28 BaseTools: Move Build Cache related function out of CreateAsBuiltInf
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1932
There are two functions in current CreateAsBuiltInf, Copy Binary files
to build cache folder and create asbuild inf file.

This patch is to separate UpdateBuildCache and CreateAsBuiltInf into
two functions.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Reviewed-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2019-06-25 09:23:57 +08:00
Bob Feng dc174cd9e5 BaseTools: refine CreateAsBuiltInf function
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1932
ModuleAutogen.CreateAsBuiltInf IsOnlyCopy parameter
and related function can be moved out of CreateAsBuiltInf,
so that CreateAsBuiltInf is more clean.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2019-06-25 09:23:53 +08:00
Fan, ZhijuX 370f16c548 BaseTools:Add import in FvImageSection
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1872

Since MultipleWorkspace is not imported in FvImageSection,
an error of "global name 'MWS' is not defined" appeared.

This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-24 17:02:53 +08:00
Shi, Steven 2378ea5515 BaseTools:Introduce CopyFileOnChange() function to copy cache files
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1894

Basetool need a CopyFileOnChange() function to avoid cache
file writing race in multi-thread build. Some platforms
build fail with file IO writing race issue when the
build cache is enabled to store cache files in multi-threads.
This is because common same library cache files (e.g. some
libs in MdePkg) can be stored by many different driver modules'
build threads at same time. Current build cache need a function
to check whether the same cache file already exist, and only
copy source file if it is different from the destination file.

This patch introduces an atomic copy function to avoid duplicated
cache files copy.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-17 17:00:46 +08:00
Shi, Steven 17b082ce67 BaseTools: Cannot store library cache of different arch together
https://bugzilla.tianocore.org/show_bug.cgi?id=1895

Build cache cannot store cache for the same library modules
in different arch together. E.g. Both the below IA32 and X64
arch BaseLib caches should exist after build Ovmf3264, but now
only the one in X64 arch exist.
The reason is the current Basetool use a set() to same all
library AutoGen objects, but the different arch lib AutoGen
objects have same __hash_ value which comes from the lib
MetaFile(The path of module file):
    def __hash__(self):
        return hash(self.MetaFile)

So the different arch lib AutoGen objects are duplicated one
to the set() and only one can exist. This is why the Basetool
can only store one arch cache for library.

This patch adds the arch string into the PlatformAutoGen and
ModuleAutoGen __hash_ definitions and ensure the different
platform and module AutoGen objects have different __hash_ values.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-17 17:00:45 +08:00
Shi, Steven 04df18535c BaseTools:Build Cache output notification message
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1868

Build need output the cache miss or hit notification
message when consume the build cache. Current build does not
output any message which is not clear for user to know
whether the module built result is from cache or not.

This patch adds message about the cache miss or hit when
build consumes the cache.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Zhiju.Fan <zhijux.fan@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-17 17:00:45 +08:00
Shi, Steven ede54e3588 BaseTools:Build cache cannot store the cache files for library package
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1867

Current build cache cannot store the cache for library package.
build -p MdePkg\MdePkg.dsc -a IA32 -b DEBUG -t VS2015x86 --hash
--binary-destination=BinCache
After build, the expected result is the BinCache folder is generated
and the MdePkg build cache files (e.g. .hash and .lib) are stored in
the BinCache folder. But the BinCache folder is not generated at all.

This patch is going to fix that issue.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Zhiju.Fan <zhijux.fan@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-17 17:00:45 +08:00
Rodriguez, Christian 48b0bf6476 BaseTools: Refactor hash tracking after checking for Sources section
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804

After adding a check to see if [Sources] section lists all the header
type files of a module, track module and library hashes for --hash
feature. If above check is not in compilance for a library or module,
force hash invalidation on that library or module.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-10 19:48:51 +08:00
Rodriguez, Christian 1fa6699e6c BaseTools: Add a checking for Sources section in INF file
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804

Add a check to see if [Sources] section lists all the header type
files of a module. Performance impact should be minimal with this patch
since information is already being fetched for Makefile purposes. All
other information is already cached in memory. No extra IO time is needed.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-10 19:48:51 +08:00
Fan, ZhijuX 04797875d1 BaseTools:Make BaseTools support new rules to generate RAW FFS FILE
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1765

If RAW FFS File Rule has no section for its data.For RAW FFS File,
directly call GenFfs tool to generate FFS file.

Ffs Rule:
[Rule.Common.USER_DEFINED.MicroCode]
  FILE RAW = $(NAMED_GUID) {
        $(INF_OUTPUT)/$(MODULE_NAME).bin
  }
[Rule.Common.USER_DEFINED.LOGO]
  FILE RAW = $(NAMED_GUID) {
                       |.bmp
  }

As shown in the rule above,if SectionType and FileType not defined,
FFS files are generated directly, and no other type of file is
generated.

The patch is to make the BaseTools support these two rules

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-06-10 19:48:50 +08:00
Steven Shi b6cfa686e4 BaseTools:Update binary cache restore time to current time
https://bugzilla.tianocore.org/show_bug.cgi?id=1742

Current Binary Cache doesn't update the restored file
creation and modification times to the current time.
Preserve the new restored file creation time as old
cached time might has potential issue to block the make
to build updated files based on the time stamp.
Enhance to update the restored file creation time to
current time.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Zhiju Fan <zhijux.fan@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-31 19:46:18 +08:00
Steven Shi 3a0b998bf3 BaseTools:Extend the binary cache to support library cache
https://bugzilla.tianocore.org/show_bug.cgi?id=1797

Current binary cache doesn't support to save and restore
the library module. If a driver module cache miss happen,
all its dependency library modules need rebuild which
is very time-consuming. This patch is to extend the binary
cache to support library.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-31 19:46:16 +08:00
Fan, ZhijuX c0fd7f734e BaseTools:Change the path of the file that Binary Cache
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1806

Current Binary Cache use the PlatformInfo.Name to distinguish
the different platform cache paths in the database folder.

But the different platforms might use the same PLATFORM_NAME
in the DSC files. Different platform caches might overlay each
other inside Binary Cache database if they use same platform name

Use the PlatformInfo.Platform.OutputDirectory instead of the
PlatformInfo.Name can solve the problem.

This patch is going to fix that issue.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-27 08:59:04 +08:00
Bob Feng e812a812c1 BaseTools: Remove './SecMain' from 'run' target
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1561

This patch is related to commit 9ca1f1
It's to remove the second .SecMain call from 'run' target.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2019-05-24 15:28:42 +08:00
Feng, Bob C 9ca1f1446a BaseTools: Remove './SecMain' from 'run' target
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1561

When a target of 'run' is passed into build.py,
BaseTools unconditionally attempts to execute the
application called './SecMain' in the build output directory.

This behavior applies to the Nt32Pkg which is being
replaced with features in the EmulatorPkg.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2019-05-22 09:15:56 +08:00
Fan, ZhijuX c1387446ed BaseTools:Update mailing list address in BaseTools error messages
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1815

The edk2 source tree contains four instances of the outdated mailing
list address "edk2-devel@lists.01.org".
I created a new variable, MSG_EDKII_MAIL_ADDR, to receive the
new email address and replaced the old one with this variable

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-22 09:15:55 +08:00
Rodriguez, Christian a7ef158b07 BaseTools: Library hashing fix and optimization for --hash feature
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1788

In V3: Must generate hashes before attempting to copy from cache for
hash verifcation
In V2: Build failure caused by passing incorrect boolean parameter to
SaveFileOnChange(). Fixed for patch instances.

Library hashing is now supported by the --hash feature. The --hash
feature implementation assumed that the hashing could be done in
place once per module, but that isn't true for libraries due to the
fact that they are built as dependencies. So on a clean build, we now
generate the .hash after the library dependencies are complete.
Added early escape as optimization, if hash already exists in memory.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-21 13:18:06 +08:00
Michael D Kinney 66b845ae06 BaseTools: Fix private includes for FILE_GUID override
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1648

When a module in a DSC file uses a FILE_GUID override in the
module scoped <Defines> section, a copy of the modified INF file
is placed in the Conf/.cache directory.  The check for private
includes uses the INF path to determine if the module is allowed
to use the private includes.  Since the INF path in this case is
not in any package, this check always fails, and no private
include paths are possible.

The fix is to keep both the OriginalPath and the new Path in
the PathClass object, and always use the OriginalPath to see if
the module INF is in the package with private includes.

Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-21 13:14:14 +08:00
Fan, ZhijuX 6b74ccf0f8 BaseTools:Enables FFS to support the GUID value format
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1768

Per FDF spec, FFS Rule can specify the hard code FILE Guid value.
But,BaseTools reports the build failure.It should be fixed.

This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-10 17:07:07 +08:00
Fan, ZhijuX afe8c4110c BaseTools: Correct the value assignment for StructurePcd
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1752

This patch is to fix the code bug in StructurePcd overall
value assignment logic. If a Pcd Array size is fixed but the
size of actual value in Dsc or Dec is bigger than the Pcd
array size, the tool will report error about such setting and
stop build.

The patch is tested minplatform, Ovmf, structure pcd regression test
These tests are build pass. The patch also tested the following cases.
Our cases focused on PcdArraySize. 1.flexiable PcdArraySize.
 2.Fixed PcdArraySize, Pcd overall value exceeds the size of PcdArray.
 3.Fixed PcdArraySize, Pcd overall value Not exceeds the size of PcdArray.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-05-10 17:06:44 +08:00
Bob Feng c648e90587 BaseTools: Fix the issue that build report missing Module Size
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1786

commit b1e27d changed GENFW output folder from DEBUG to OUTPUT,
but that commit missed updating the buildreport.py for FwReportFileName.

This patch is going to fix this issue.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-05-09 15:03:30 +08:00
Fan, ZhijuX 26e2b295ec BaseTools:Remove unused BaseTools\Source\Python\UPT\Dll
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1735

BaseTools runs from python source. It doesn't depend on
dll any more. So, DLL can be removed.

This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-08 09:41:42 +08:00
Fan, ZhijuX 0cb3f77153 BaseTools:ECC report errors on account of analyze special characters
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1751

In case that a C function body contains the string of L'', L'\"',
L"\"", L''', L""", L"\"\"", L"\"^", L" \"", L"\" \"", ('L",\\\""')
ECC tool running under python3 interpreter will report error.
The antlr4 module misidentified this character

This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-08 09:41:42 +08:00
Fan, ZhijuX 8b4b2fb9a1 BaseTools:corrected error message for the DatumType error
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1762

For structure Pcd, Its DatumType may be self-defined,
If the PCD DatumType format is incorrect,
this PCD DatumType error message is inappropriate.

This patch corrects the error message.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-08 09:41:41 +08:00
Fan, ZhijuX 273190e189 BaseTools:Fixed an issue where the order of GuidS changed in guid.xref
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1749

Add content to dsc
[PcdsPatchableInModule.common]
  gEfiMdeModulePkgTokenSpaceGuid.test1|FALSE

The order of file Guid.xref will change after increment build.
The root cause is set() is used in Get all the PCDS
the order of the data may change if set() is used

This patch is going to fix that issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-05-01 10:57:05 +08:00
Bob Feng c9b3fe1586 BaseTools: Support customized compiling command
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1737
User may add other commands before CC command from build_rule.txt
for specific purpose. It worked fine before commit 05217d210e.

This patch is going to fix the bug in commit 05217d210e to
support customized CC command.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-04-28 09:11:27 +08:00
Christian Rodriguez 56008eb7f6 BaseTools: Hash false success.. minor change in hash invalidation
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1692

Change part of the hash error handling to invalidate hashes in the
cache destination not the cache source.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-22 16:50:26 +08:00
Fan, ZhijuX 35c2af00d8 BaseTools:fixed the incorrect autogen makefile which cause build failure.
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1729

On some build environment, build fails but on the other build machines,
build success. This is the regression issue introduced by commit
 05217d210e

As Dict is unordered, an error occurs when extract the index of the Dict
in the order of the keys after the creation of a new item.
Keys are indexed inconsistently before and after adding a new item.
The logic of the program is to store the key's corresponding index as
reference data in the MakeFile and use it as part of the macro.
The data model is: $(LIST_%d) % Dict.keys().index(Key)
So for now, use OrdereDict instead of Dict.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-22 16:50:25 +08:00
Rodriguez, Christian 452b5ad61d BaseTools: Enhance Bin Cache database to support save the cache
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1726

V2: Added the platform name to the path
Add more level sub-directories in the database to support save
the cache for multiple platforms with multiple tool-chains and
targets, just like edk2 Build output.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-22 16:50:24 +08:00
Yunhua Feng 0b836855fb BaseTools: Sometime write file not immediate to disk
BZ: https://bugzilla.tianocore.org/process_bug.cgi

On Windows OS, sometime the generated file is not immediate saved to disk.
When run nmake, prompt AutoGen.h not found, and stop build.
Below blog shows Write-Replace to fix it. This patch uses this way to write
temp file, then rename the temp file to the real file.
https://blog.gocept.com/2013/07/15/reliable-file-updates-with-python/

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2019-04-22 10:05:52 +08:00
Rodriguez, Christian 0eccea3fbe BaseTools: Hash false success with back to back builds
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1692

Add error handling to the --hash feature so that hash files
are invalidated when a build error occurs.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-16 13:14:15 +08:00
Fan, ZhijuX 87bfb9bcb7 BaseTools:Coding problems cause can not encode FMP Auth Header
GenerateCapsule: error: can not encode FMP Auth Header
The system cannot find the file specified.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-16 13:14:14 +08:00
Fan, ZhijuX 05217d210e BaseTools:Enable the /MP option of MSVC compiler
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1672
The /MP option of MSVC compiler can reduce the total time to compile the
source files on the command line.

This patch is going to enable this MSVC option in BaseTools.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-16 13:14:13 +08:00
Christian Rodriguez db4d47fd3a BaseTools: Fix corner-cases of --hash feature
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1680

Re-use libraries, since they have already been hashed.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-10 13:32:14 +08:00
Rodriguez, Christian f2b5e04aca BaseTools: Fix corner-cases of --hash feature
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1680

Respect artifact location within directory structure.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-10 13:32:13 +08:00
Rodriguez, Christian 1b8caf0d87 BaseTools: Fix corner-cases of --hash feature
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1680

Re-order hashing operations so we don't do redundant hashes.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-10 13:32:12 +08:00
Rodriguez, Christian 2914e8153d BaseTools: Fix corner-cases of --hash feature
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1680

Consider modules with .inc source files as Binary Modules
and do not Skip by hash.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-10 13:32:11 +08:00
Feng, Bob C b1e27d175a BaseTools: Fixed issue in MultiThread Genfds function
https://bugzilla.tianocore.org/show_bug.cgi?id=1450
In the Multiple thread Genfds feature, build tool generates
GenSec, GenFFS command in Makefile.

The Non-Hii Driver does not generate .offset file for uni string offset,
but the build tool has not knowledge about this in autogen phase. So
in this patch, I add a check in Makefile for GenSec command. If the GenSec
input file does not exist, the GenSec will not be called. And if GenSec
command is not called, its output file, which is also the input file of
GenFfs command, will also not exist.So for GenFfs command,
I add a new command parameter -oi which means
the input file is an optional input file which would not exist. so
that I can generate GenFfs command with "-oi" parameter in Makefile.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-04-10 13:32:10 +08:00
Michael D Kinney 2e351cbe8e BaseTools: Replace BSD License with BSD+Patent License
https://bugzilla.tianocore.org/show_bug.cgi?id=1373

Replace BSD 2-Clause License with BSD+Patent License.  This change is
based on the following emails:

  https://lists.01.org/pipermail/edk2-devel/2019-February/036260.html
  https://lists.01.org/pipermail/edk2-devel/2018-October/030385.html

RFCs with detailed process for the license change:

  V3: https://lists.01.org/pipermail/edk2-devel/2019-March/038116.html
  V2: https://lists.01.org/pipermail/edk2-devel/2019-March/037669.html
  V1: https://lists.01.org/pipermail/edk2-devel/2019-March/037500.html

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-09 09:10:20 -07:00
Fan, ZhijuX 1c27ec4236 BaseTools:Coding problems caused by special characters
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1670

During BaseTools compiling under Chinese or Japanese
language Windows, python exception occurring.
UnicodeDecodeError: 'ascii' codec can't decode byte 0xbd
in position 3528: ordinal not in range(128)

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-01 13:37:19 +08:00
Fan, ZhijuX 58742d7945 BaseTools:Trim will trig exception when input asl UTF8 format file
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1641

The command trim --asl-file -o test.i UTF8.asl will trig the exception.
Trim tool should report error message for unsupported UTF8 file instead
of the exception.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-04-01 13:37:17 +08:00
Feng, Bob C a004d17d01 BaseTool: Fixed an issue of Structure PCD
https://bugzilla.tianocore.org/show_bug.cgi?id=1665

Build fail when the structure pcd has
member of flexible size array

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-03-27 10:18:43 +08:00
Feng, Bob C 2f2c51acfb BaseTools: Remove the logic SourceOverridePath
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
SOURCE_OVERRIDE_PATH is for EDK component INF files.
The corresponding logic should be removed.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-03-24 16:12:38 +08:00
Feng, Bob C f293841f93 BaseTools: Remove EDKI related logic from Trim tool
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350

Remove EDKI related logic from Trim tool.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-03-24 16:12:37 +08:00
mjohn4 b0189eac00 BaseTools: Explicitly close files after readlines
Rework some file open().readlines to open, readlines, close.
This prevents excessive file handles being open at the same time,
which may be a problem with alternative python environments.

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael Johnson <michael.johnson@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-03-15 15:38:59 +08:00
Jiewen Yao d868846a4e BaseTool/Build: Add --disable-include-path-check.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1620

This option is added to disable the include path check
for outside of package.
The original purpose of thie check is to make sure EDK II
modules must not reference header files outside of the packages
they depend on or within the module's directory tree.

However, we do see the usage to build EDKII as executable running
in the operating system which requires include path to outside.
For example, EmulatorPkg. The current solution (soft link) is
weird hack - EmulatorPkg\Unix\Host\X11IncludeHack.

With this solution, this can be supported easily.

The patch is validated with and without --disable-include-path-check.
If user does not use --disable-include-path-check, the build will fail
with outside path in the include path.
If user uses --disable-include-path-check, the build will pass
with outside path in the include path.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-03-15 13:29:29 +08:00
Liming Gao 3a4698202c Revert "BaseTools:BaseTools supports to the driver combination."
This reverts commit 838bc257ba.
After further evaluation, there are the unclear behavior in for the
driver combination feature. To not impact Q1 stable tag, remove it first.
1. If the drivers to be combined have the different PCD or library instance
   setting, build should not combine them and report build break. But this
   commit doesn't consider this case.
2. When start the sub driver fail, continue to start other sub driver. This
   behavior is required to be clarifed in build spec.
3. Unload the sub driver when the combined driver start fail. This case need
   to call the sub driver unload function for the driver start fail only.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-28 10:01:53 +08:00
Fan, ZhijuX de72c9d1da BaseTools:The BOM character is processed when python reads a file
When python3 reads an XML file it will parse the file in error
if the file has a BOM

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-27 09:39:09 +08:00
Feng, Bob C dc082d5029 BaseTools: Fix a Eot issue.
FirmwareVolume.UnDispatchedFfsDict is mutated during iteration,
convert the FirmwareVolume.UnDispatchedFfsDict.keys() to a new list
 to resolve this problem.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-26 09:10:23 +08:00
Feng, Bob C dde29b9ca5 BaseTools: Eot failed when enable python3
The Eot will report error when python3 enabled.
We replaced sdict with collections.OrderedDict in python3
patch set, but the sdict implement "append" method which is not
implemented in collections.OrderedDict.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-26 09:10:23 +08:00
Feng, Bob C 2cfb9d0f1a BaseTools: Fix a bug about Structure PCD
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1535
If there is Hii Structure Pcd, build will fail, root cause is that
there is an incorrect variable access method used in code.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-22 15:47:15 +08:00
Fan, ZhijuX 29c2de1272 BaseTools:Build fail if define [DEPEX] in library inf
When define [DEPEX] in lib inf to build, it will fail and report"
gUefiOvmfPkgTokenSpaceGuid.test1 used in [Depex] section should be
used as FixedAtBuild type and VOID* datum type in the module."
But we define this PCD to FixedAtBuild type and VOID* datum type indeed.

DEC:
[PcdsFixedAtBuild]
gUefiOvmfPkgTokenSpaceGuid.test1 |
{GUID("4096267b-da0a-42eb-b5eb-fef31d207cb4")}|VOID*|0x3c

DSC:
add pcd under lib inf as below:

NULL|TestPkg/TestLib/TestLib.inf
 <PcdsFixedAtBuild>
    gUefiOvmfPkgTokenSpaceGuid.test1 | {GUID(gUefiOvmfPkgTokenSpaceGuid)}

Lib inf:(TestPkg/TestLib/TestLib.inf)
[Depex]
gUefiOvmfPkgTokenSpaceGuid.test1
[FixedPcd]
gUefiOvmfPkgTokenSpaceGuid.test1

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-22 15:47:14 +08:00
Feng, Bob C f747640bf3 BaseTool: Fixed incremental rebuild issue.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1540

This issue in introduced by commit
d943b0c339

To convert bytes to string, we need to use bytes.decode()
instead of using str(bytes).

If the source file is not a txt file, ignore that file.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-22 15:47:14 +08:00
Jiewen Yao 4247c24fba BaseTool/GenC: Fix build error when type is BASE or USER_DEFINED.
https://bugzilla.tianocore.org/show_bug.cgi?id=1544

1) The GenC tool does not include the constructor/destructor for
USER_DEFINED module. It should be included.
Only constructor/destructor with BASE type is included here.
The constructor/destructor with PEI/DXE/SMM type is still excluded
to keep original behavior.

2) The GenC tool includes the UnloadImage code for BASE module.
It should NOT be included.

3) The GenC tool uses EFI_STATUS and ASSERT_EFI_ERROR for BASE type.
It should use RETURN_STATUS and ASSERT_RETURN_ERROR.

4) The GenC tool miss DebugLib.h for BASE or USER_DEFINED module
AutoGen.c. Only Base.h is there. It should add Library/DebugLib.h.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-22 12:20:36 +08:00
Fan, ZhijuX 8a64c7ea00 BaseTools:PCD value error in structure pcd sku case.
Defined 2 PCDs(Test4 & Test401) and 2 SKUs(DEFAULT & _),
then set "SKUID_Defines" to ALL, for FixedAtBuild
gEfiStructuredPcdPkgTokenSpaceGuid. Test401 in this case,
its value should get from "Default" SKU, not from "_" SKU,
but we does not set value in SKU "_" in dsc, so Test401
should only display the value get from dec.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-20 13:16:58 +08:00
Fan, ZhijuX e8d13fef90 BaseTools:Fixed build failure when specifying multiple BUILDTARGET
With Python3, the dict.value() method returns an iterator.
If a dictionary is updated while an iterator on its keys is used,
a RuntimeError is generated.
Converting the iterator to a list() forces a copy of the mutable
keys in an immutable list which can be safely iterated.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-20 13:16:57 +08:00
Feng, Bob C 7da3ed89d4 BaseTools: replace Sdict with OrderedDict in UPT
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1504

V2: import OrderredDict as Sdict in Misc.py instead
updating all the py files.

Sdict class is removed on the commit
174a9d3cc8
but there are still some py files import it.

This patch is to use collections.OrderedDict to replace Sdict.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2019-02-20 13:16:56 +08:00
Fan, ZhijuX 838bc257ba BaseTools:BaseTools supports to the driver combination.
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1520

To save the image size without the compression, more than
one drivers can be combined into single one. When more than
one drivers are combined, their depex will be AND together.
Below is the example to combine BootManagerPolicyDxe into
DriverHealthManagerDxe.

Besides this patch, BaseTools also needs to check the module
type and make sure all module type are same. Otherwise,
BaseTools will report the error.
DRIVER INF has the parameter ENTRY_POINT
LIBRARY INF has the parameter LIBRARY_CLASS

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-20 13:16:56 +08:00
Feng, Bob C 68c67d3a2a BaseTools: Fixed a code bug for Pcd Array.
For example, PCD gUefiOvmfPkgTokenSpaceGuid.Test001 datatype is Array:
TEST1[2]
and the filed TEST1UINT64ARRAY in TEST1 is also an array:
UINT64 TEST1UINT64ARRAY[2];

Then the following filed assignment in DSC will cause build failure.
gUefiOvmfPkgTokenSpaceGuid.Test001[0].TEST1UINT64ARRAY|{'A','B'}

The root cause is build tool generate incorrect PcdValueInit.c File.

This patch is going to fix this issue.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-19 20:32:44 +08:00
Feng, Bob C c0ebd9d30f BaseTools: Fixed a bug in Vpd handling
If there are multiple sku used in a platform and
gEfiMdeModulePkgTokenSpaceGuid.PcdNvStoreDefaultValueBuffer PCD
is used, build will fail.

This is a regression issue introduced by the commit:
5695877ec8 which only update the
Vpd offset for default SKU but not other SKUs.

This patch is going to fix this issue.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-19 20:32:43 +08:00
Feng, Bob C 97f003236f BaseTools: Fix a ParseDevPathValue function issue.
In ParseDevPathValue, the binary data need to convert to string
for python3 and python2 compatiblity.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-19 20:32:43 +08:00
Fan, ZhijuX 7b6a8c9cf2 BaseTools:Fix a ECC issue
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1522

A property error occurred because the property of the
function was not defined. a property is now redefined.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-19 20:32:42 +08:00
Feng, Bob C 9108039558 BaseTools: Add MaxSizeUserSet to Pcd deepcopy function
MaxSizeUserSet is missing in Pcd deepcopy function.
This patch is to add it back to Pcd deepcopy function.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-19 20:32:42 +08:00
Feng, Bob C c417c1b33d BaseTools: Fixed an issue about StructurePcd
If use a structure pcd in fdf, build tool crash
This is a regression issue introduced by py3 patch set.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-18 09:11:48 +08:00
Feng, Bob C 5f094268e0 BaseTools: Fix the build report issue about Structure PCD
https://bugzilla.tianocore.org/show_bug.cgi?id=1472
build report use incorrect method to parse DynamicDefault/DynamicExDefault
and DynamicVpd/DynamicExVpd structure Pcd value.

This patch is to fix this issue.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-18 09:11:48 +08:00
Feng, Bob C c196d1d129 BaseTools: Fixed a build report issue.
Generate report fail when -Y EXECUTION_ORDER in build command.
This patch is going to fix this issue.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-18 09:11:47 +08:00
Feng, Bob C 8059cd2483 BaseTools: Correct the error message for UPT
This patch is going to correct the error message
for UPT.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Hess Chen <hess.chen@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-18 09:11:46 +08:00
Antoine Coeur fb0b35e05f BaseTools: Various typo
Various typo in BaseTools.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Coeur <coeur@gmx.fr>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-14 10:02:28 +08:00
Philippe Mathieu-Daud? bac50dfbcd BaseTools: Fix build failure when specifying multiple BUILDTARGET
With Python3, the dict.value() method returns an iterator.
If a dictionary is updated while an iterator on its keys is used,
a RuntimeError is generated.
Converting the iterator to a list() forces a copy of the mutable
keys in an immutable list which can be safely iterated.

Commit f8d11e5a4a converted various uses but missed one:
When specifying multiple BUILDTARGET, the first target builds
successfully, but then the PGen.BuildDatabase._CACHE_ dictionary is
updated, and accessing the next target triggers a RuntimeError.

Convert this iterator to an immutable list, to solve this build error:

    $ build -a IA32 -t GCC5 -b RELEASE -b NOOPT -p OvmfPkg/OvmfPkgIa32.dsc
    [...]
    Processing meta-data ...
    build.py...
     : error C0DE: Unknown fatal error when processing [OvmfPkg/OvmfPkgIa32.dsc]

    (Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!)

    (Python 3.5.3 on linux) Traceback (most recent call last):
      File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2387, in Main
        MyBuild.Launch()
      File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2141, in Launch
        self._MultiThreadBuildPlatform()
      File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1921, in _MultiThreadBuildPlatform
        self.Progress
      File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 304, in __init__
        self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs)
      File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 477, in _InitWorker
        for BuildData in PGen.BuildDatabase._CACHE_.values():
    RuntimeError: dictionary changed size during iteration

Note: The culprit commit (f8d11e5a4a) can not be found with bisection.
In 9c2d68c0a2 the build tools default to the python version provided
by the ${PYTHON} environment variable, however the Python3 transition is
not functional before d943b0c339. f8d11e5a4a falls between the
previous two.

Reported-by: Leif Lindholm <leif.lindholm@linaro.org>
Fixes: f8d11e5a4a
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
Tested-by: Leif Lindholm <leif.lindholm@linaro.org>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2019-02-06 13:08:16 -08:00
Laszlo Ersek 963517211c BaseTools/BuildReport: fix report for platforms/arches without struct PCDs
The goal of commit 97c8f5b9e7 ("BaseTools:StructurePCD value display
incorrect in "Not used" section.", 2019-02-02) was to display the full
contents of such structure PCDs in the build report that were set in the
platform DSC or the FDF, but not used in any module INFs. The listings
would appear in the

  PCDs not used by modules or in conditional directives

section of the build report.

Commit 97c8f5b9e7 assumed that any (platform, architecture) combination
would have a (possibly empty) set of structure PCD (and so the set of the
structure PCDs could be filtered for set-but-unused ones).

This is not the case: in "DscBuildData.py", in method
UpdateStructuredPcds(), if "S_pcd_set" remains an empty OrderedDict(),
then it is not added to "GlobalData.gStructurePcd" *at all*, for the
current (platform, architecture) combination.

As a result, when the PCD report tries to fetch the set of structure PCDs
for the current (platform, architecture), "GlobalData.gStructurePcd" does
not return an empty OrderedDict(); instead, it raises a KeyError. Fix it
by defaulting to an empty OrderedDict(), with the get() method.

Reported-by: Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1513
Fixes: 97c8f5b9e7
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2019-02-06 13:07:21 -08:00
Fan, ZhijuX 97c8f5b9e7 BaseTools:StructurePCD value display incorrect in "Not used" section.
StructurePCD value display incorrect in "Not used" section,
that the value defined in structure does not show.

StructurePCD will not display in "Not Used" section if
main structure not define in dsc and pcd not in inf

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-02 10:28:56 +08:00
Feng, Bob C 4a6b179f4f BaseTools: Fixed incorrect line number in PcdValueInit.c
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1435
This patch is going to fix the issue of incorrect line
number in PcdValueInit.c

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-02 10:28:56 +08:00
Feng, Bob C 9f30e40128 BaseTools: Enable CODE format in DEC file
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1292.
Enable CODE format in DEC File.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-02 10:28:55 +08:00
Hess Chen ce3bfbec71 BaseTools/ECC: Add a new type of exception
Add a new type of exception for function body checkpoint.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Hess Chen <hess.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-02 10:28:54 +08:00
Fan, ZhijuX 5895f7f6d4 BaseTools: PCD value incorrect in structure pcd sku case.
Defined 2 PCDs(Test4 & Test401) and 2 SKUs(DEFAULT & _),
then set "SKUID_Defines" to ALL, for FixedAtBuild
gEfiStructuredPcdPkgTokenSpaceGuid. Test401 in this case,
its value should get from "Default" SKU, not from "_" SKU,
but we does not set value in SKU "_" in dsc, so Test401
should only display the value get from dec.

Missing the map() function causes SKU specific items to
not be deleted

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-02 10:28:54 +08:00
Feng, Bob C 8189be6fd7 BaseTools: Eot tool Python3 adaption
v2:
The python files under CParser4 are generated by antlr4 and for
python3 usage. They have python3 specific syntax, for example
the data type declaration for the arguments of a function. That
is not compitable with python2. this patch is to remove these syntax.

Eot tool Python3 adaption.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-01 11:09:25 +08:00
Feng, Bob C c60377d7f9 BaseTools: ECC tool Python3 adaption
v2:
The python files under CParser4 are generated by antlr4 and for
python3 usage. They have python3 specific syntax, for example
the data type declaration for the arguments of a function. That
is not compitable with python2. this patch is to remove these syntax.

ECC tool Python3 adaption.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-01 11:09:25 +08:00
Feng, Bob C d943b0c339 BaseTools: Handle the bytes and str difference
Deal with bytes and str is different, remove the unicode(),
correct open file parameter.
Using utcfromtimestamp instead of fromtimestamp.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:24 +08:00
Feng, Bob C f8d11e5a4a BaseTools: the list and iterator translation
In python3,The keys of the dictionary not a list,It needs to be converted

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:24 +08:00
Feng, Bob C 7fa0e68afd BaseTools:ord() don't match in py2 and py3
In python2, the FvHeaderBuffer Type is a str
In python3, the FvHeaderBuffer Type is a bytes

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:24 +08:00
Feng, Bob C 4a3773e578 BaseTools:Fixed Rsa issue and a set define issue.
ValueError: non-hexadecimal number found in
 fromhex() arg at position 0

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:23 +08:00
Liming Gao 7aef7b7cbf BaseTools: Update PYTHON env to PYTHON_COMMAND
Update PYTHON env to PYTHON_COMMAND.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:23 +08:00
Zhijux Fan fd2d74007b BaseTools:Update build tool to print python version information
print PYTHON3_ENABLE and PYTHON_COMMAND in build tool

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:22 +08:00
Zhijux Fan 94c912950c BaseTools: Similar to octal data rectification
In python3, if Value is octal data, the int(Value, 0) report an error

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:22 +08:00
Zhiju Fan 30e65c4e19 BaseTools:There is extra blank line in datalog
There should be no blank line across every
line in datalog if open it with Notepad++.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:22 +08:00
Zhijux Fan b3e94a0617 BaseTools: change the Division Operator
PEP 238 -- Changing the Division Operator
x/y to return a reasonable approximation of the mathematical result
    of the division ("true division")
x//y to return the floor ("floor division")

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:21 +08:00
Feng, Bob C f935bdd270 BaseTools:File open failed for VPD MapFile
correct open MapFile support python2 and python3

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:21 +08:00