Commit Graph

473 Commits

Author SHA1 Message Date
Michael D Kinney 2e351cbe8e BaseTools: Replace BSD License with BSD+Patent License
https://bugzilla.tianocore.org/show_bug.cgi?id=1373

Replace BSD 2-Clause License with BSD+Patent License.  This change is
based on the following emails:

  https://lists.01.org/pipermail/edk2-devel/2019-February/036260.html
  https://lists.01.org/pipermail/edk2-devel/2018-October/030385.html

RFCs with detailed process for the license change:

  V3: https://lists.01.org/pipermail/edk2-devel/2019-March/038116.html
  V2: https://lists.01.org/pipermail/edk2-devel/2019-March/037669.html
  V1: https://lists.01.org/pipermail/edk2-devel/2019-March/037500.html

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-04-09 09:10:20 -07:00
Feng, Bob C 2f2c51acfb BaseTools: Remove the logic SourceOverridePath
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
SOURCE_OVERRIDE_PATH is for EDK component INF files.
The corresponding logic should be removed.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-03-24 16:12:38 +08:00
mjohn4 b0189eac00 BaseTools: Explicitly close files after readlines
Rework some file open().readlines to open, readlines, close.
This prevents excessive file handles being open at the same time,
which may be a problem with alternative python environments.

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Michael Johnson <michael.johnson@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-03-15 15:38:59 +08:00
Jiewen Yao d868846a4e BaseTool/Build: Add --disable-include-path-check.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1620

This option is added to disable the include path check
for outside of package.
The original purpose of thie check is to make sure EDK II
modules must not reference header files outside of the packages
they depend on or within the module's directory tree.

However, we do see the usage to build EDKII as executable running
in the operating system which requires include path to outside.
For example, EmulatorPkg. The current solution (soft link) is
weird hack - EmulatorPkg\Unix\Host\X11IncludeHack.

With this solution, this can be supported easily.

The patch is validated with and without --disable-include-path-check.
If user does not use --disable-include-path-check, the build will fail
with outside path in the include path.
If user uses --disable-include-path-check, the build will pass
with outside path in the include path.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-03-15 13:29:29 +08:00
Liming Gao 3a4698202c Revert "BaseTools:BaseTools supports to the driver combination."
This reverts commit 838bc257ba.
After further evaluation, there are the unclear behavior in for the
driver combination feature. To not impact Q1 stable tag, remove it first.
1. If the drivers to be combined have the different PCD or library instance
   setting, build should not combine them and report build break. But this
   commit doesn't consider this case.
2. When start the sub driver fail, continue to start other sub driver. This
   behavior is required to be clarifed in build spec.
3. Unload the sub driver when the combined driver start fail. This case need
   to call the sub driver unload function for the driver start fail only.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-28 10:01:53 +08:00
Fan, ZhijuX 29c2de1272 BaseTools:Build fail if define [DEPEX] in library inf
When define [DEPEX] in lib inf to build, it will fail and report"
gUefiOvmfPkgTokenSpaceGuid.test1 used in [Depex] section should be
used as FixedAtBuild type and VOID* datum type in the module."
But we define this PCD to FixedAtBuild type and VOID* datum type indeed.

DEC:
[PcdsFixedAtBuild]
gUefiOvmfPkgTokenSpaceGuid.test1 |
{GUID("4096267b-da0a-42eb-b5eb-fef31d207cb4")}|VOID*|0x3c

DSC:
add pcd under lib inf as below:

NULL|TestPkg/TestLib/TestLib.inf
 <PcdsFixedAtBuild>
    gUefiOvmfPkgTokenSpaceGuid.test1 | {GUID(gUefiOvmfPkgTokenSpaceGuid)}

Lib inf:(TestPkg/TestLib/TestLib.inf)
[Depex]
gUefiOvmfPkgTokenSpaceGuid.test1
[FixedPcd]
gUefiOvmfPkgTokenSpaceGuid.test1

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-22 15:47:14 +08:00
Feng, Bob C f747640bf3 BaseTool: Fixed incremental rebuild issue.
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1540

This issue in introduced by commit
d943b0c339

To convert bytes to string, we need to use bytes.decode()
instead of using str(bytes).

If the source file is not a txt file, ignore that file.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-22 15:47:14 +08:00
Jiewen Yao 4247c24fba BaseTool/GenC: Fix build error when type is BASE or USER_DEFINED.
https://bugzilla.tianocore.org/show_bug.cgi?id=1544

1) The GenC tool does not include the constructor/destructor for
USER_DEFINED module. It should be included.
Only constructor/destructor with BASE type is included here.
The constructor/destructor with PEI/DXE/SMM type is still excluded
to keep original behavior.

2) The GenC tool includes the UnloadImage code for BASE module.
It should NOT be included.

3) The GenC tool uses EFI_STATUS and ASSERT_EFI_ERROR for BASE type.
It should use RETURN_STATUS and ASSERT_RETURN_ERROR.

4) The GenC tool miss DebugLib.h for BASE or USER_DEFINED module
AutoGen.c. Only Base.h is there. It should add Library/DebugLib.h.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-22 12:20:36 +08:00
Fan, ZhijuX e8d13fef90 BaseTools:Fixed build failure when specifying multiple BUILDTARGET
With Python3, the dict.value() method returns an iterator.
If a dictionary is updated while an iterator on its keys is used,
a RuntimeError is generated.
Converting the iterator to a list() forces a copy of the mutable
keys in an immutable list which can be safely iterated.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-20 13:16:57 +08:00
Fan, ZhijuX 838bc257ba BaseTools:BaseTools supports to the driver combination.
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1520

To save the image size without the compression, more than
one drivers can be combined into single one. When more than
one drivers are combined, their depex will be AND together.
Below is the example to combine BootManagerPolicyDxe into
DriverHealthManagerDxe.

Besides this patch, BaseTools also needs to check the module
type and make sure all module type are same. Otherwise,
BaseTools will report the error.
DRIVER INF has the parameter ENTRY_POINT
LIBRARY INF has the parameter LIBRARY_CLASS

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-20 13:16:56 +08:00
Feng, Bob C c0ebd9d30f BaseTools: Fixed a bug in Vpd handling
If there are multiple sku used in a platform and
gEfiMdeModulePkgTokenSpaceGuid.PcdNvStoreDefaultValueBuffer PCD
is used, build will fail.

This is a regression issue introduced by the commit:
5695877ec8 which only update the
Vpd offset for default SKU but not other SKUs.

This patch is going to fix this issue.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-19 20:32:43 +08:00
Antoine Coeur fb0b35e05f BaseTools: Various typo
Various typo in BaseTools.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Coeur <coeur@gmx.fr>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-14 10:02:28 +08:00
Philippe Mathieu-Daud? bac50dfbcd BaseTools: Fix build failure when specifying multiple BUILDTARGET
With Python3, the dict.value() method returns an iterator.
If a dictionary is updated while an iterator on its keys is used,
a RuntimeError is generated.
Converting the iterator to a list() forces a copy of the mutable
keys in an immutable list which can be safely iterated.

Commit f8d11e5a4a converted various uses but missed one:
When specifying multiple BUILDTARGET, the first target builds
successfully, but then the PGen.BuildDatabase._CACHE_ dictionary is
updated, and accessing the next target triggers a RuntimeError.

Convert this iterator to an immutable list, to solve this build error:

    $ build -a IA32 -t GCC5 -b RELEASE -b NOOPT -p OvmfPkg/OvmfPkgIa32.dsc
    [...]
    Processing meta-data ...
    build.py...
     : error C0DE: Unknown fatal error when processing [OvmfPkg/OvmfPkgIa32.dsc]

    (Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!)

    (Python 3.5.3 on linux) Traceback (most recent call last):
      File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2387, in Main
        MyBuild.Launch()
      File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2141, in Launch
        self._MultiThreadBuildPlatform()
      File "BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1921, in _MultiThreadBuildPlatform
        self.Progress
      File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 304, in __init__
        self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, **kwargs)
      File "BaseTools/Source/Python/AutoGen/AutoGen.py", line 477, in _InitWorker
        for BuildData in PGen.BuildDatabase._CACHE_.values():
    RuntimeError: dictionary changed size during iteration

Note: The culprit commit (f8d11e5a4a) can not be found with bisection.
In 9c2d68c0a2 the build tools default to the python version provided
by the ${PYTHON} environment variable, however the Python3 transition is
not functional before d943b0c339. f8d11e5a4a falls between the
previous two.

Reported-by: Leif Lindholm <leif.lindholm@linaro.org>
Fixes: f8d11e5a4a
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
Tested-by: Leif Lindholm <leif.lindholm@linaro.org>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2019-02-06 13:08:16 -08:00
Feng, Bob C d943b0c339 BaseTools: Handle the bytes and str difference
Deal with bytes and str is different, remove the unicode(),
correct open file parameter.
Using utcfromtimestamp instead of fromtimestamp.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:24 +08:00
Feng, Bob C f8d11e5a4a BaseTools: the list and iterator translation
In python3,The keys of the dictionary not a list,It needs to be converted

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:24 +08:00
Zhijux Fan 94c912950c BaseTools: Similar to octal data rectification
In python3, if Value is octal data, the int(Value, 0) report an error

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:22 +08:00
Zhijux Fan b3e94a0617 BaseTools: change the Division Operator
PEP 238 -- Changing the Division Operator
x/y to return a reasonable approximation of the mathematical result
    of the division ("true division")
x//y to return the floor ("floor division")

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:21 +08:00
Feng, Bob C 3f7cb70c5a BaseTools: Make sure AllPcdList valid.
This patch is to make sure the AllPcdList is always evaluated.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-02-01 11:09:20 +08:00
Zhijux Fan 2c2bb053a3 Basetools: It went wrong when use os.linesep
in python2 and python3,use of line breaks

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:20 +08:00
Zhijux Fan 1590d12315 BaseTools:Similar to octal data rectification
In python3, if Value is octal data, the int(Value, 0) report an error

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:18 +08:00
Yunhua Feng af881abc65 BaseTools: replace long by int
replace long by int
Because the long() was not exist in Python3

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:18 +08:00
Yunhua Feng 1d2aa01cdf BaseTools: Remove unnecessary super function
Remove unnecessary super function

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:17 +08:00
Zhijux Fan 3a041437c9 BaseTools: Rename iteritems to items
replace the list iteritems by items in Python3.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:16 +08:00
Zhijux Fan e77e59c9bd BaseTool:Rename xrange() to range()
Because the xrange() was not exist in Python3

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-02-01 11:09:16 +08:00
Carsey, Jaben 1bcc2cd63d BaseTools/AutoGen: move functions
Move SplitOption and ConvertStringToByteArray from
Common.Misc to this file.
There were no other consumers of the functions.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-01-21 18:03:33 +08:00
Fan, ZhijuX dd1988cf01 BaseTools:build break if the Path contains SingleFile.Ext
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1325

if SingleFile.Path = "/foo/bar.Sap/yada/source.S" and
SingleFile.Ext = ".S". Then key would end up "/foo/bar"
instead of "/foo/bar.Sap/yada/source" as intended.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-01-21 18:03:30 +08:00
Felix Polyudov 842695d2ff BaseTools: Fix incorrect formatting of GenFds command dictionary
GenFdsCommand returned dictionary with elements that
are not compatible with GenFdsApi.
As a result the following options were not processed by GenFdsApi:
-v, -q, -d, --genfds-multi-thread, --ignore-sources
The issue is introduced by commit b3497bad12.

V2: Remove EFI_SOURCE references

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Felix Polyudov <felixp@ami.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-21 18:03:29 +08:00
Fan, ZhijuX 3e1ab49481 BaseTools:Build fail when PCD use in the [DEPEX] section of INF files
Update _FixedPcdVoidTypeDict to FixedVoidTypePcds
'_FixedPcdVoidTypeDict' no longer exists because edk2 version (b23414f6).
'ModuleAutoGen' object has no attribute '_FixedPcdVoidTypeDict'.
Build fail when PCD use in the [DEPEX] section of INF files

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-21 18:03:28 +08:00
Feng, Bob C 8229250132 BaseTools: Remove unused logic for EDKI
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1350
Remove EDK module type support from BaseTools python code.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-01-14 09:34:23 +08:00
Carsey, Jaben 938cf4c33a BaseTools: fix imports
1 - Some of these imports are cascaded from another file. Import them locally.
2 - Some of these imports are not used. Remove them.
3 - Some of these were missing the namespace used to import them.

These changes facilitate optimization of BaseTools:
https://bugzilla.tianocore.org/show_bug.cgi?id=42

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-01-10 22:01:11 +08:00
Zhiju.Fan 073a76e662 BaseTools: Add $(INC)-like support when compiling .nasm files
current edk2\BaseTools\Conf\build_rule.template, the compile of nasm
source files does not have the $(INC) support.

The '-I' option only includes the directory of the nasm source file
(${s_path}(+)). Hence, it will be impossible for nasm files to include
files outside of the nasm source file directory.

As a comparison, the compile of both .s and .asm have $(INC) support
in their compile commands.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=1085
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-12-19 08:42:14 +08:00
Feng, Bob C 5695877ec8 BaseTools: Fix PcdNvStoreDefaultValueBuffer Value.
https://bugzilla.tianocore.org/show_bug.cgi?id=1385
This patch is going to fix the regression issue that is
introduced by commit e6eae3b4c7
and commit 0b6c5954e1.

PcdNvStoreDefaultValueBuffer value is update to Vpd Info File,
but it is not update into a internal cache. This patch will
fix this incorrect value in that internal cache.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-12-15 15:45:07 +08:00
BobCF 4e37570739 BaseTools: Optimize string concatenation
https://bugzilla.tianocore.org/show_bug.cgi?id=1288

This patch is one of build tool performance improvement
series patches.

This patch is going to use join function instead of
string += string2 statement.

Current code use string += string2 in a loop to combine
a string. while creating a string list in a loop and using
"".join(stringlist) after the loop will be much faster.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: BobCF <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-12-07 10:30:21 +08:00
Zhao, ZhiqiangX b3497bad12 BaseTools: AutoGen and GenFds share the parser data.
V2:
Extract the common part of new API and the original main() function
into one function.

V1:
https://bugzilla.tianocore.org/show_bug.cgi?id=1288

Currently, AutoGen and GenFds run in different python interpreters. The
parser are duplicated. This patch is going to create new API for GenFds
and have the build to call that API instead of executing GenFds.py. As
such, the GenFds and build can share the parser data.

This patch is expected to save the time of GenFds about 2~3 seconds.
More details will be logged in BZ.

This is the summary measure data generated from python cProfile for
building Ovmf.

Currently:
8379147 function calls (8135450 primitive calls) in 12.580 seconds

After applying this patch:
3428712 function calls (3418881 primitive calls) in 8.944 seconds

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: ZhiqiangX Zhao <zhiqiangx.zhao@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Carsey Jaben <jaben.carsey@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-12-07 10:12:05 +08:00
Jaben Carsey bc39c5cbea BaseTools: create and use a standard shared variable for '*'
add a variable for the string '*' and then use it instead of lots of '*'

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by : Bob Feng <bob.c.feng@intel.com>
2018-12-06 17:11:24 +08:00
Jaben Carsey ea55e37c36 BaseTools: cleanup LongFilePathSupport usage
1) remove an identical function and import it from Common.LongFilePathSupport
2) remove an import that is not needed/used.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by : Bob Feng <bob.c.feng@intel.com>
2018-12-06 17:10:27 +08:00
Zhao, ZhiqiangX 93f9898582 BaseTools: Check the max size for string PCD.
According to PCD_DATABASE_INIT in
edk2\MdeModulePkg\Include\Guid\PcdDataBaseSignatureGuid.h,
the max size for string PCD should not exceed USHRT_MAX 65535(0xffff).

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: ZhiqiangX Zhao <zhiqiangx.zhao@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-11-02 22:39:08 +08:00
Feng, Bob C 5af5153ad9 BaseTools: Add checking to EFI variable attribute.
This patch is going to add a check that only if the
attribute of a EFI variable include 'NV', it will be
added into PcdNvStoreDefaultValueBuffer.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-11-02 22:15:53 +08:00
zhijufan 5061efe70d BaseTools: list .nasm include inc files as its dependency
.nasm source file may include some header files.
header file should be listed in Makefile as .nasm source file
dependency.
But now, BaseTools doesn't find them and list them in Makefile.
This is a missing, because original ASM file supports it.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-25 16:43:00 +08:00
Yonghong Zhu 5d54ab94db BaseTools: Fix the bug for Pcd used in command line's override
V2: remove the not used parameter i

Fix the bug for Pcd used in command line not override the Pcd used
in the [component] driver's sub-section.

Case:
DSC file:
[PcdsFixedAtBuild]
TokenSpaceGuid.PcdTest

[Components]
 TestPkg/TestDriver.inf {
  <PcdsFixedAtBuild>
  TokenSpaceGuid.PcdTest|"b"
  }

build command with --pcd TokenSpaceGuid.PcdTest="AAAABB"

Then we found the Pcd value in the AutoGen.c file is incorrect,
because of the incorrect logic that use the pcd in the [component]
section to re-override it.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-25 15:16:16 +08:00
Zhaozh1x da2d4f76b7 BaseTools: Not convert the void* pcd string in command line to array.
For void* type pcd in command line, if its value is string, code should
not convert the void* pcd string in command line to array, otherwise it
will make the pcd value in report not match its real raw value.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: ZhiqiangX Zhao <zhiqiangx.zhao@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-10-24 22:27:04 +08:00
Zhaozh1x 32be12235d BaseTool: Support different PCDs that refers to the same EFI variable.
V2:
Make the code of patch both compatible for Python2 and Python3.

V1:
If different PCDs refer to the same EFI variable, then do EFI variable
combination, according to the VariableOffset of different PCDS.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: ZhiqiangX Zhao <zhiqiangx.zhao@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2018-10-22 15:19:26 +08:00
Yonghong Zhu eb99b52f98 BaseTools: Fix the crash issue when Dynamic structure Pcd use in FDF
The case is use Dynamic structure Pcd in the FDF file.
Current code already save the  Guid, Name and Filed info for those Pcd,
but it directly use the dict key as [Name, Guid] and cause this crash
issue.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=1264
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-10-21 20:03:37 +08:00
Yunhua Feng 04aa423c8d BaseTools: increment build generate inf file lost .depex file
increment build generate inf file in output directory lost .depex file info.
(re-add this patch since it be reverted in Python3 migration patches,
but this check is not related with Python3)

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=1244
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-15 10:28:05 +08:00
Liming Gao 1ccc4d895d Revert BaseTools: PYTHON3 migration
This reverts commit 6693f359b3c213513c5096a06c6f67244a44dc52..
678f851312.

Python3 migration is the fundamental change. It requires every developer
to install Python3. Before this migration, the well communication and wide
verification must be done. But now, most people is not aware of this change,
and not try it. So, Python3 migration is reverted and be moved to edk2-staging
Python3 branch for the edk2 user evaluation.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
2018-10-15 08:29:14 +08:00
Yunhua Feng b87099f336 BaseTools: increment build generate inf file lost .depex file
increment build generate inf file in output directory lost .depex file info.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=1244
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 10:01:36 +08:00
Yunhua Feng 8be15c61e8 BaseTools: Fix some build and report file issue
1. increment build not skip make file when not change any file
2. report file generate abundant blank line
3. Build encounter Database is locked on some platform, using database
   auto commit
4. Fv BaseAddress must have if set

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:57:13 +08:00
Yunhua Feng 2e300969ae BaseTools: change the Division Operator in the expression
PEP 238 -- Changing the Division Operator
x/y to return a reasonable approximation of the mathematical result
    of the division ("true division")
x//y to return the floor ("floor division")

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:54:53 +08:00
Yunhua Feng 00fcce9153 BaseTools: do the list and iterator translation
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:54:52 +08:00
Yunhua Feng 86e6cf98a8 BaseTools: Handle the bytes and str difference
Deal with bytes and str is different, remove the unicode()
Using utcfromtimestamp instead of fromtimestamp.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:54:07 +08:00
Yunhua Feng a09f4c91f7 BaseTools: fix the open file's read and write bugs
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:50:45 +08:00
Yunhua Feng 4ce4f757d7 BaseTools: remove the super() function argument
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:50:43 +08:00
Yunhua Feng 5135cc4852 BaseTools: replace the list iteritems by items
replace the list iteritems by items in Python3.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:50:43 +08:00
Yunhua Feng 8371d87412 BaseTools: Rename long() to int()
Because the long() function was not exist in Python3.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:50:42 +08:00
Yunhua Feng 261eee25de BaseTools: Rename xrange() to range()
Because the xrange() was not exist in Python3

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:50:41 +08:00
Yunhua Feng 487062c048 BaseTools: Remove the "from __future__ import" items
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-10-13 09:50:40 +08:00
zhijufan c526dcd40f BaseTools: refactor the error for PCD value is negative or exceed max
refactor the error handling for the PCD value that is negative or it
exceed the max value.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-09-30 16:05:49 +08:00
bob.c.feng@intel.com 6a147d6dae BaseTool: Support different PCDs that refers to the same EFI variable.
If Structure PCD and Normal Pcd refer to the
same EFI variable, do EFI variable merge, otherwise, do
EFI variable combination.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-09-26 22:50:40 +08:00
Liming Gao f27e800a99 BaseTools: Fix the wrong reference _GetSkuIds() in AutoGen code
DscBuildData has been updated to define SkuIds instead of _GetSkuIds().
The consumer code should refer to SkuIds.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-09-26 22:15:42 +08:00
Carsey, Jaben 4def57d98c BaseTools: refactor to not overcreate ModuleAutoGen objects
currently created for 3 different purposes and saved once.
this makes it created once and saved and then referenced.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-09-20 22:18:08 +08:00
Carsey, Jaben 4c92c81d61 BaseTools: AutoGen - refactor class properties
use function decorators

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-09-20 22:18:06 +08:00
Carsey, Jaben e6c62ab768 BaseTools: AutoGen refactor WorkspaceAutoGen class
Update the WorkspaceAutoGen class to use caching decorators and remove
the no longer needed private variables.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-09-20 22:18:05 +08:00
Carsey, Jaben e6eae3b4c7 BaseTools: Refactor PlatformAutoGen
use decorators for property and automatic caching
remove circular dependency between some APIs

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <Bob.c.Feng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-09-20 22:18:05 +08:00
Zhao, ZhiqiangX 4c6fda33c8 BaseTools: Correct DXE_PCD_DATABASE_INIT.
Add the handle of PCD_DATABASE_INIT and PCD_DATABASE_UNINIT
for Boolean type pcd.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: ZhiqiangX Zhao <zhiqiangx.zhao@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-09-12 11:25:22 +08:00
Zhao, ZhiqiangX ced8685838 BaseTools: Check PcdNvStoreDefaultValueBuffer.
Build tool should report warning if a platform
defines [DefaultStores] but forgets to defined
PcdNvStoreDefaultValueBuffer as PcdsDynamicExVpd in dsc file.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: ZhiqiangX Zhao <zhiqiangx.zhao@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-09-12 11:25:22 +08:00
Feng, Bob C 71127ce883 BaseTools: Structure Pcd value override incorrect.
This patch is going to fix the issue that
The Pcd field value is override incorrectly when there
is no Pcd overall value assignment in Dsc file.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-09-12 11:25:20 +08:00
Feng, Bob C 4cf022f2f1 BaseTools: Fixed the PcdValue trailing zero issue.
1. Not append trailing zero for PcdValue
2. make sure the point to Variable Name in PCD
DataBase 2 bytes aligned.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-09-03 10:14:50 +08:00
Carsey, Jaben f5f4667dae BaseTools: Refactor to remove functionally equivalent functions
IsSupportedArch and IsBinaryModule return the same value under the same
curcimstances.  Remove newer one with fewer callers and send them to the
other function.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-30 21:05:18 +08:00
Carsey, Jaben 94c0455937 BaseTools: Create and use a shared value for 'MSFT' from DataType
I see lots of 'MSFT' throughout code and this can reduce them.

Cc: Bob Feng <Bob.c.Feng@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-30 08:42:09 +08:00
Carsey, Jaben 0fa0e56ee0 BaseTools: AutoGen.py remove unused import
AutoGen does not use anything defined in BuildClassObject

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-29 08:45:29 +08:00
Feng, YunhuaX 87010d3d02 BaseTools: remove cmp due to deprecated in python3
remove cmp due to deprecated in python3

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-23 15:01:27 +08:00
Feng, YunhuaX 6cf6fed024 BaseTools: Fix regression issue by b23414f654
V2: Renaming function DepexExpressionTokenList to  DepexExpressionDict
instead of changing the callers

Fix regression issue by b23414f654
AttributeError: 'ModuleAutoGen' object has no attribute
'DepexExpressionDict'

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-21 14:08:24 +08:00
Jaben Carsey 0258ba6256 BaseTools: AutoGen refactor to iterate less
Don't iterate over new dictionaries with one item

Create the data and then add to dictionary.

Note: if you diff ignoring whitespace changes you
can more easily see the relevant changes.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-20 10:53:06 +08:00
Jaben Carsey 830bf22fa5 BaseTools: AutoGen - tag a function as cachable
MakeFile generation is once per module, so mark it as such.
also move the time stamp creation function inside as it's
only called from one place.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-20 10:53:05 +08:00
Jaben Carsey b23414f654 BaseTools: AutoGen refactor ModuleAutoGen caching
1) Add a new file Common/caching.py
    a.	Allows for automated caching of repeated class functions, class
        properties, and non-class functions
    b.	When called the first time the value is cached and if called a
        second time, the cached result is called, not the function.
    c.	When used, this saves lots of memory since the actual function
        pointers are replaced with smaller data elements.
    d.  note that not all features are used yet.
2) Fix AutoGen/GenMake and AutoGen/GetC to not call into private member
    variables of ModuleAutoGen class
    a. use the existing accessor properties for the data
3) Change AutoGen classes to remove a exception for duplicate members in
    __new__ and use ?in? testing to speed up
4)	Work on ModuleAutoGen class
    a.	Change all properties that use caching to use @caching_property
        (see #1 above)
    b.	Change all properties that do not use caching to use standard python
        decorator "@property"
    c.	Change all cases where a dictionary/set/list/object was created
        and then immediately updated to use constructor parameters
    d.	Refactor each property function to remove the internal variable
        that is no longer needed (this helps find items like #2 above)
    e.  Refactor _ApplyBuildRule with optional parameter to work around
        circular dependency with BinaryFileList.

Note that 4e was almost certainly unintended as the functions were acting on
incomplete information since they were directly accessing private instance
variables at the same time another function on the stack was creating the same
private isntance data.

This overall changes behavior slightly, but makes the codebase smaller and
easier to read.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-20 10:53:05 +08:00
Yunhua Feng 7b85a1afa3 BaseTools: Optimizing code for function doesn't match
Optimizing code for function doesn't match name and comment

Fix https://bugzilla.tianocore.org/show_bug.cgi?id=924

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-16 14:46:53 +08:00
Feng, Bob C 0fc5e71c68 BaseTools: Remove a unused function.
the call statement of _CheckDuplicateInFV() was commented out
in 2014. There is no call statement of _CheckDuplicateInFV(),
so remove it.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-08-16 08:32:58 +08:00
Liming Gao 669b55e6d5 BaseTool: Fixed the bug of Boolean Hii Pcd packing.
When packing HiiPcd into PcdNvStoreDefaultValueBuffer,
The boolean type pcd value packing incorrect.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-08-16 08:32:57 +08:00
Yunhua Feng e5cbb98256 BaseTools: Debug message make confused
Debug message make confused

Fix https://bugzilla.tianocore.org/show_bug.cgi?id=995

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-08-08 08:49:36 +08:00
Jaben Carsey 0f78fd7349 BaseTools: AutoGen - change class variable to funciton variable
This variable is only used in one function, make it local there.
Also when iterating on the variable, use dict.items() to get value
instead of re-looking up the value multiple times.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-07-25 09:40:49 +08:00
Yunhua Feng 5b73e17fb1 BaseTools: Fix the different token with the same PCD
If the different token with the same PCD names are used in the driver,
build can pass. If the different token with the same PCD name are used
in the different library, then the driver build will fail. The reason
is that the driver autogen.c is not generated correctly for the second
case. BaseTools should check the duplicated PCD name is the driver and
its linked libraries.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-24 16:28:56 +08:00
Yonghong Zhu bfa7eeb61d BaseTools: Correct _PCD_PATCHABLE_TokenName_SIZE's value
current if user use PatchPcdSetPtr in library, it will report the
_PCD_PATCHABLE_TokenName_SIZE is not defined.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-24 16:27:49 +08:00
Supreeth Venkatesh 549ae85ce1 BaseTools/AutoGen: Update header file for MM modules.
This patch corrects the Module Type Header file for Management Mode(MM)
as specified in PI v1.6 Specification. Also, it updates parameter for
auto generated template functions from EFI_SMM_SYSTEM_TABLE2 to
EFI_MM_SYSTEM_TABLE.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Supreeth Venkatesh <supreeth.venkatesh@arm.com>
Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-23 11:11:47 +08:00
Yunhua Feng a10def9165 BaseTools: enable FixedAtBuild (VOID*) PCD use in the [DEPEX] section
V3: Add some invalid type and datum check

V2: limit the PCD used in the [Depex] section should be used in the module

The PCD item used in INF [Depex] section must be defined as FixedAtBuild
type and VOID* datum type, and the size of the PCD must be 16 bytes.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=443
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-07-23 08:56:37 +08:00
Yonghong Zhu d900d7c985 BaseTools: Remove the duplicate Pcd items
The case is the Pcd item both used in 1 module inf and 1 lib inf, and
in the DSC component section, it override the Pcd value.
In the module, the pcd value is the override value, but in the lib inf
the pcd value is the value that in the DSC PCD section's value, then it
cause the Pcd value is different in the module and lib. but actually we
only need use the Pcd value in the module to decide whether it use the
same value.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-18 11:28:23 +08:00
Gary Lin 0ff3b52e06 BaseTools: Use absolute import in AutoGen
Based on "futurize -f libfuturize.fixes.fix_absolute_import

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-07-16 11:22:14 +08:00
BobCF 543f5ac30f BaseTools: Enable structure pcd in FDF file
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-13 17:25:43 +08:00
bob.c.feng@intel.com 0a563f3fec BaseTool: Fixed the incorrect cache key.
This patch is to fix the incorrect cache key of
skip ModuleAutoGen cache.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-12 15:57:32 +08:00
Yonghong Zhu 9edba51f93 BaseTools: Fix the bug that incorrect size info in the Lib autogen
The case is a PCD used in one library only, and in DSC component
section the PCD value is override in one of module inf. Then it cause
the bug the PCD size in the Lib autogen use the PCD value in the DSC
PCD section, but not use the override value.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-10 16:01:09 +08:00
Feng, Bob C 18ef4e713f BaseTool: Add cache for the result of SkipAutogen.
Add a cache for the value of skip ModuleAutoGen
process flag. This cache can improve build performance.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-07-09 10:28:08 +08:00
Liming Gao f7496d7173 BaseTools: Clean up source files
1. Do not use tab characters
2. No trailing white space in one line
3. All files must end with CRLF

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-07-09 10:25:47 +08:00
Jaben Carsey b420d98502 BaseTools: AutoGen - move constructor out of loop
Create the 2 comparison objects once outside the loop.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-29 15:39:28 +08:00
Jaben Carsey 7c12d613ba BaseTools: AutoGen - clean up access
1) add a property so others can access needed data
2) change GenMake to use property
3) add local variable in GenMake to speed up access

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-29 15:38:51 +08:00
Jaben Carsey 3c920616bb BaseTools: AutoGen - Remove unused variables.
There are 2 variables that we populate, but never use.
remove them entirely.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:36:51 +08:00
Gary Lin 86379ac48b BaseTools: Replace StringIO.StringIO with io.BytesIO
Replace StringIO.StringIO with io.BytesIO to be compatible with python3.
This commit also removes "import StringIO" from those python scripts
that don't really use it.

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:28 +08:00
Gary Lin 0d1f5b2b5d BaseTools: Fix old python2 idioms
Based on "futurize -f lib2to3.fixes.fix_idioms"

* Change some type comparisons to isinstance() calls:
    type(x) == T -> isinstance(x, T)
    type(x) is T -> isinstance(x, T)
    type(x) != T -> not isinstance(x, T)
    type(x) is not T -> not isinstance(x, T)

* Change "while 1:" into "while True:".

* Change both

    v = list(EXPR)
    v.sort()
    foo(v)

and the more general

    v = EXPR
    v.sort()
    foo(v)

into

    v = sorted(EXPR)
    foo(v)

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:27 +08:00
Gary Lin ccaa7754a2 BaseTools: Adjust the spaces around commas and colons
Based on "futurize -f lib2to3.fixes.fix_ws_comma"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:25 +08:00
Gary Lin 27c4ceb41c BaseTools: Remove the deprecated hash_key()
Replace "has_key()" with "in" to be compatible with python3.
Based on "futurize -f lib2to3.fixes.fix_has_key"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:23 +08:00
Gary Lin 87d2afd07c BaseTools: Remove the old python "not-equal"
Replace "<>" with "!=" to be compatible with python3.

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:22 +08:00
Gary Lin 72443dd250 BaseTools: Refactor python print statements
Refactor print statements to be compatible with python 3.
Based on "futurize -f libfuturize.fixes.fix_print_with_import"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:33:21 +08:00
Gary Lin 5b0671c1e5 BaseTools: Refactor python except statements
Convert "except ... ," to "except ... as" to be compatible with python3.
Based on "futurize -f lib2to3.fixes.fix_except"

Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-27 16:31:30 +08:00
Yonghong Zhu 58cf30f71f BaseTools: Enhance BaseTools supports FeaturePcd usage in VFR file
Bugzilla 348 only fixed FixedAtBuild Pcd type, now this patch also add
support for FeaturePcd type.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=348
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-06-22 13:34:01 +08:00
Yonghong Zhu b1aff264bf BaseTools: remove including Base.h if the module type is not BASE
According the module type to include the header file.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=867
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-06-14 14:27:34 +08:00
Jaben Carsey caf744956d BaseTools: Cleanup unneeded code
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-13 09:02:56 +08:00
Jaben Carsey c14b58614f BaseTools: refactor to remove functions
refactoring almost identical functions to delete and use the other.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-06-13 09:02:26 +08:00
Marvin.Haeuser@outlook.com 5a57246eab BaseTools: Rename String to StringUtils.
For case-insensitive file systems, edk2 String.py collides with the
Python string.py, which results in build errors. This,for example,
applies to building via the Windows Subsystem for Linux from a
DriveFS file system. This patch renames String to StringUtils to
prevent conflicts for case-insensitive file systems.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Marvin Haeuser <Marvin.Haeuser@outlook.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2018-05-28 13:05:00 +08:00
Yunhua Feng a253d217ee BaseTools: Report more clear error message when PCD type mismatch
Error message is not clear when PCD type defined in driver's Library
is different with PCD type defined in DSC components or PCD type
defined in DSC PCD section.

Case as below:
DSC:
[PcdsFixedAtBuild]
PcdToken.PcdCName | "A"
[Components]
 TestPkg/TestDriver.inf {
  <PcdsPatchableInModule>
    PcdToken.PcdCName | "B"
 }
Library:
[Pcd]
 PcdToken.PcdCName

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-22 19:35:28 +08:00
Yunhua Feng 5a444dfd7c BaseTools: Library PCD type will inherit from the driver
If a PCD is not referenced in global PCD section of DSC file at all,
but is referenced in module scope, then the default PCD type for libs
should be the module scoped PCD type.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=901
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-22 19:34:33 +08:00
Yunhua Feng 74f59e9275 BaseTools: Fix bug PCD type in component is not same with Pcd section
Per DSC spec 3.11 [Components] Sections:
The PCD access methods (and storage methods) are selected on a platform
basis - it is not permitted to have a PCD listed in one of the Pcd
sections and use it differently in an individual module. For example,
if a PCD is listed in a [PcdsFixedAtBuild] section, it is not permitted
to list it in a <PcdsPatchableInModule> sub-section of an INF file.
but current code doesn't report error for this case.

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=951
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-22 19:33:35 +08:00
Lin, Derek 3f34e36d04 BaseTools: Fix --hash Package and Module hash value.
The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it
generate different hash value even nothing changes.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Derek Lin <derek.lin2@hpe.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-16 15:21:57 +08:00
Yonghong Zhu c731b54505 BaseTools: Remove the redundant code
the ArraySize and Array already be got in line 1093, so this code are
redundant.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
2018-05-10 08:38:15 +08:00
Yunhua Feng 6b285ca366 BaseTools: Fix generating array's size is incorrect in AutoGen.c
case example:
DSC:
 [PcdsFixedAtBuild]
  PcdToken.PcdName | "A"
 [Components]
 TestPkg/TestDriver.inf {
  PcdToken.PcdName | {0x41,0x42,0x43,0x44}
 }

Generating the size of array is incorrect in AutoGen.c
 GLOBAL_REMOVE_IF_UNREFERENCED const UINT8
 _gPcd_FixedAtBuild_PcdName[2] = {0x41,0x42,0x43,0x44};

Fixes: https://bugzilla.tianocore.org/show_bug.cgi?id=950
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-10 08:36:00 +08:00
Carsey, Jaben 8252e6bf2d BaseTools: dont make iterator into list if not needed
functions (like join) can use the iterator just as easily.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:08:09 +08:00
Carsey, Jaben cf3062901d BaseTools: refactor Depex optomization
No need to make a list from the set.  just pop the item off.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:07:56 +08:00
Carsey, Jaben c5c7e68a91 BaseTools: Remove lists form set construction
There is no need to make a list to make a set.  remove lists
that are only used in constructing sets.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:07:55 +08:00
Carsey, Jaben d0a0c52c22 BaseTools: standardize GUID and pack size
currently GUID packing and pack size determination is spread
throughout the code. This introduces a shared function and dict and
routes all code paths through them.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:07:34 +08:00
Carsey, Jaben 31ff1c443e BaseTools: AutoGen - add Opcode constants
add constants for dependency expression opcode strings
use these new opcode string constants

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:11 +08:00
Carsey, Jaben 6c2d8cb238 BaseTools: remove redundant content in InfSectionParser
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:11 +08:00
Carsey, Jaben 1360bcb84e BaseTools: remove unused member variable
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:11 +08:00
Carsey, Jaben cb7e6aa77a BaseTools: AutoGen - refactor more functions only called in __init__
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:10 +08:00
Carsey, Jaben 1549328f5f BaseTools: AutoGen - refactor function to remove extra variables
we dont need to keep data we already have in different formats...

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:10 +08:00
Carsey, Jaben d3054be59e BaseTools: Refactor to share GUID packing function
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:09 +08:00
Carsey, Jaben a2e721704b BaseTools: AutoGen - remove another function no one calls
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:09 +08:00
Carsey, Jaben 8b88b1634f BaseTools: AutoGen - move function to clean file namespace
the function is only used in one other function.
just move it there.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:08 +08:00
Carsey, Jaben 032a7c9fe2 BaseTools: AutoGen - remove function no one calls
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:08 +08:00
Carsey, Jaben 64bb8d4d51 BaseTools: refactor __init__ functions to not compute temporary variable
just assign correct value to member variable in __init__ or call
parent __init__

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:08 +08:00
Carsey, Jaben 61f5b77dd5 BaseTools: AutoGen - no need to recompute
looping over a list and recomputing the same value has no impact on final value

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:07 +08:00
Carsey, Jaben 92a4eebb37 BaseTools: AutoGen - refactor out a useless class
this class was never instantiated.  the static function was called.
save the function, remove the rest.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:07 +08:00
Carsey, Jaben 0caa769dbb BaseTools: AutoGen - refactor out a list
the lists were used in __init__ then converted to sets
instead just use the sets from the begining

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:06 +08:00
Carsey, Jaben def89ed025 BaseTools: AutoGen - refactor out functions only called in __init__
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:06 +08:00
Carsey, Jaben 5565a8c4d2 BaseTools: move PCD size calculation functions to PcdClassObject
move both GetPcdMaxSize and GetPcdSize to the PcdClassObject.
fix MAX_SIZE_TYPE to have int values

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:05 +08:00
Carsey, Jaben 3b743b3b15 BaseTools: AutoGen - UniClassObject refactor static methods
change methods which do not use self to @staticmethod
change their calls to use class name instead of instance

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:03 +08:00
Carsey, Jaben 124d672ee6 BaseTools: AutoGen - remove global line
this serves no purpose since we dont change the global or assign to it.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:02 +08:00
Carsey, Jaben aaa5a23c95 BaseTools: AutoGen - refactor class factory
since instances are not added to cache, the factory does nothing.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:01 +08:00
Carsey, Jaben c23ef28c20 BaseTools: AutoGen - share StripComments API
add the API root in one class file.
delete the static API out of both classes.
share it in the single location.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:00 +08:00
Carsey, Jaben c541989741 BaseTools: AutoGen - GenVar refactor static methods
change methods which do not use self to @staticmethod
change their calls to use class name instead of instance

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:03:00 +08:00
Carsey, Jaben bffcfca88f BaseTools: AutoGen - refactor dictionary access
dont use dict.get() inside loops of dictionary contents. its not needed.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:02:59 +08:00
Carsey, Jaben dea5ef9dc4 BaseTools: AutoGen - refactor assemble_variable
make this function @staticmethod since self parameter is not used.
change valuelist to valuedict since it is a dictionary.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:02:59 +08:00
Carsey, Jaben 18011927b1 BaseTools: this function has no purpose.
it looks like a old POC of the concepts then used to make the classes
in the file.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:02:59 +08:00
Carsey, Jaben b7b51025b9 BaseTools: check before accessing members in __eq__
minimize risk for exceptions.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:02:58 +08:00
Carsey, Jaben a7360838d7 BaseTools: AutoGen - update to remove duplicate constant value
PCD size by type is shared.  just use it.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-04 13:02:58 +08:00
Carsey, Jaben c93356ada9 BaseTools: remove redundant if comparison
inherently python will check string and list for None and having data

if <x> in [None, ''] and similar are superflous.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-02 15:02:20 +08:00
Carsey, Jaben 91fa33eeca BaseTools: Replace Binary File type strings with predefined constant
BINARY_FILE_TYPE_FW was 'FW'
BINARY_FILE_TYPE_GUID was 'GUID'
BINARY_FILE_TYPE_PREEFORM was 'PREEFORM'
BINARY_FILE_TYPE_UEFI_APP was 'UEFI_APP'
BINARY_FILE_TYPE_UNI_UI was 'UNI_UI'
BINARY_FILE_TYPE_UNI_VER was 'UNI_VER'
BINARY_FILE_TYPE_LIB was 'LIB'
BINARY_FILE_TYPE_PE32 was 'PE32'
BINARY_FILE_TYPE_PIC was 'PIC'
BINARY_FILE_TYPE_PEI_DEPEX was 'PEI_DEPEX'
BINARY_FILE_TYPE_DXE_DEPEX was 'DXE_DEPEX'
BINARY_FILE_TYPE_SMM_DEPEX was 'SMM_DEPEX'
BINARY_FILE_TYPE_TE was 'TE'
BINARY_FILE_TYPE_VER was 'VER'
BINARY_FILE_TYPE_UI was 'UI'
BINARY_FILE_TYPE_BIN was 'BIN'
BINARY_FILE_TYPE_FV was 'FV'

v2 - split apart FV and GUID types with different meanings.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-02 15:01:43 +08:00
Carsey, Jaben be409b6756 BaseTools: Replace PCD type strings with predefined constant
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-02 14:59:25 +08:00
Carsey, Jaben eece4292ac BaseTools: DataType - cleanup list constants
remove unused ones
convert lists used for membership testing to sets
use shared ones not local ones

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-02 14:59:20 +08:00
Carsey, Jaben 88c6c1b64c BaseTools: Define and use a set for common list
share a set for both PEI module types

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-02 14:56:33 +08:00
Carsey, Jaben 8bb63e377b BaseTools: replace string constants used for module types
replace raw strings in the code (note: except UPT) with constants.
SUP_MODULE_BASE was 'BASE'
SUP_MODULE_SEC was 'SEC'
SUP_MODULE_PEI_CORE was 'PEI_CORE'
SUP_MODULE_PEIM was 'PEIM'
SUP_MODULE_DXE_CORE was 'DXE_CORE'
SUP_MODULE_DXE_DRIVER was 'DXE_DRIVER'
SUP_MODULE_DXE_RUNTIME_DRIVER was 'DXE_RUNTIME_DRIVER'
SUP_MODULE_DXE_SAL_DRIVER was 'DXE_SAL_DRIVER'
SUP_MODULE_DXE_SMM_DRIVER was 'DXE_SMM_DRIVER'
SUP_MODULE_UEFI_DRIVER was 'UEFI_DRIVER'
SUP_MODULE_UEFI_APPLICATION was 'UEFI_APPLICATION'
SUP_MODULE_USER_DEFINED was 'USER_DEFINED'
SUP_MODULE_SMM_CORE was 'SMM_CORE'
SUP_MODULE_MM_STANDALONE was 'MM_STANDALONE'
SUP_MODULE_MM_CORE_STANDALONE was 'MM_CORE_STANDALONE'

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-05-02 14:56:31 +08:00
Yunhua Feng 4085777757 BaseTools: Fix one invalid change in 6be94743
Roll back one change in 6be94743, it was updated incorrect.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-04-26 14:49:56 +08:00
Carsey, Jaben 34c006cd47 BaseTools: AutoGen - use defaultdict to auto initialize
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-04-26 14:44:36 +08:00
Carsey, Jaben 992fbe3519 BaseTools: AutoGen - use dafultdict instead of dict
since we manually make each entry a set(), just use defaultdict(set)

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
2018-04-26 14:44:35 +08:00