Commit Graph

473 Commits

Author SHA1 Message Date
Michael D Kinney ef3840c1ff BaseTools: Fix DSC override of Guided tool
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3359

If the DSC file provides an override of a Guided tool path
and/or Guided tool GUID value, then make sure the one from the
DSC file is used if it is higher priority than the Guided tool
in the tools_def.txt file.  This makes the Guided tool used by
GenFds match the tool listed GuidedSectionTools.txt.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2021-05-10 23:28:58 +00:00
Michael D Kinney 8c8f49f0dc BaseTools: Fix New Target/ToolChain/Arch in DSC [BuildOptions] issues
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3359

* Save/Restore global state in GetToolChainAndFamilyFromDsc()
  This resolves an issue where the multi-arch build for
  UefiPayloadPkg would skip the autogen and build of IA32
  components.
* Expand tools wildcard.
  This resolves the issue where autogen makefile contents
  would have a BUIDLRULEFAMILY tools definitions with an
  '*' in the tool field that breaks the build from invalid
  makefile syntax.
* Build rule family higher priority than Family.
  This resolves the issue where flags were appended from
  both the BUILDRULEFAMILY and FAMILY when only
  BUILDRULEFAMILY should be appended when present.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Acked-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Andrew Fish <afish@apple.com>
2021-05-03 18:16:23 +00:00
Michael D Kinney ab957f036f BaseTools/Source/Python: New Target/ToolChain/Arch in DSC [BuildOptions]
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3359

Update BaseTools to support new build targets, new tool chains,
and new architectures declared in DSC file [BuildOptions] sections.

* Do not expand * when tools_def.txt is parsed.  Only expand when
  both tools_def.txt and DSC [BuilsOptions] sections have been parsed.
  This also requires more flexible matching of tool keys that contain *
  in tool key fields.

* Pre-scan the platform DSC file for FAMILY and TOOLCHAIN declarations
  DSC in [BuildOptions] sections before the FAMILY and TOOLCHAIN need
  to be known.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2021-04-29 07:29:17 +00:00
Michael D Kinney 8d2012035d BaseTools/GenMake: Sort generated makefile tool definitions
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3353

Sort the tool definition content of generated makefiles to help
verify that makefile contents have not changed after BaseTools
code changes.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2021-04-27 05:56:39 +00:00
Michael D Kinney f2f4c6be2d BaseTools/Source/Python: Verify TAB_TOD_DEFINES_FAMILY key present
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3344

Veify that TAB_TOD_DEFINES_FAMILY key is present in ToolDef
dictionary before access.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2021-04-23 18:55:51 +00:00
Michael D Kinney c055be5b82 BaseTools/PlatformAutoGen: MAKE_FLAGS and MAKE_PATH fixes
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3312

Update parsing of MAKE_FLAGS in DSC [BuildOptions] sections
to split the flags into a list to be compatible with
running the make command using Popen().  Parsing MAKE_FLAGS
from tools_def.txt already uses _SplitOption().  This change
uses the same _SplitOption() method for MAKE_FLAGS from a
DSC [BuildOptions] section.

Also update the parsing of MAKE_PATH to support MAKE_PATH
from tools_def.txt or the DSC [BuildOptions] section.  MAKE_PATH
in DSC [BuildOptions] section is higher priority than MAKE_PATH
in tools_def.txt.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2021-04-14 16:28:41 +00:00
Irene Park 21e1dc286e BaseTools: Fix the improper error logging
EdkLogger.ERROR() was replaced with EdkLogger.error() to deliver the
expected error message when an error occurs.

Signed-off-by: Irene Park <ipark@nvidia.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2021-01-06 02:57:38 +00:00
Yunhua Feng d15d0d3d8a BaseTools: Should always define PCD TOKEN value as Zero for static PCD
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3120
FixedAtBuild, PatchableInModule and FeatureFlag PCD don't use PCD TOKEN.
Their PCD TOKEN value can always be zero.
AutoGen.h will not be changed when static PCD is added or removed.
Dynamic PCD add or remove will still cause dynamic PCD token value be
changed.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Yunhua Feng <fengyunhua@byosoft.com.cn>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-12-23 02:33:36 +00:00
Mingyue Liang 6c4efc0509 BaseTools: Resolve index out of range errors.
This problem is generated by solving bz2972's
patch, and the commit ID is
0af7f8e6a9.

This is a problem when updating the DEPs file.
The code does not consider that there is only
one line of content in the file, so the filter
condition is added to prevent the index from
exceeding the range.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-11-19 02:22:57 +00:00
Mingyue Liang 8c610e6075 BaseTools: Limit command line length.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2528

Currently, CL command contains multiple C files will be compiled,
and that caused command line too long, which may trigger build error.

In order to solve this issue, the following rules is used in this scene:

If the number of C files is greater than one, a txt file will be used
to record these C files, and replaces the corresponding content in
command line with the file name.

Else (only one C file listed in the command line), the length of the
whole CL command line will determine whether use a file to record. If
the length exceeds the limited max length, use the recording file; else
C file name directly listed in the command line

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-11-10 05:49:38 +00:00
Bob Feng 09af9bd9be BaseTools: Enable Module Scope Structure Pcd
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2648

This patch is to enable the Module scoped Structure Pcd usage.
User can set structure pcd field value in module scope. For example,
under the [components] section of a dsc file, user can override some
field value for a specific module.

  Package/Module.inf{
      <PcdsFixedAtBuild>
      gUefiTokenSpaceGuid.StructurePcdModule.FieldName | 5
  }

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Tested-by: Liming Gao <gaoliming@byosoft.com.cn>
Acked-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-11-05 04:27:28 +00:00
fengyunhua b65afdde74 BaseTools: Add EDKII_DSC_PLATFORM_GUID MACRO
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2969

Add EDKII_DSC_PLATFORM_GUID MACRO to AutoGen.h and AutoGen.c

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yunhua Feng <fengyunhua@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Tested-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-10-13 02:35:08 +00:00
Bob Feng 3a7a676114 BaseTools: Remove CanSkip calling for incremental build
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2978

If a module add a new PCD, the pcd token number will be
reassigned. The new Pcd token number should be updated
to all module's autogen files. CanSkip can only detect a
single module's change but not others. CanSkip block the
pcd token number update in incremental build, so this
patch is going to remove this call.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Mingyue Liang 9641a7f975 BaseTools: Normalize case of pathname when evaluating Macros.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880

Currently, When doing the Incremental build, the directory
macros extended to absolute path in output Makefile, which
is inconsistent with the output of Clean build.

When we do macro replacement, we can't replace macro due to
inconsistent path case, which results in inconsistent display
of incremental build and clean build in makefile.Therefore,
the path is converted to achieve the correct macro replacement.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Mingyue Liang aec99d9bc3 BaseTools: Add included files to deps_target file.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2882

After changing the name of the include source file,
when doing incremental build, the previous source file
is not covered in the. DEPs file, and a build error occurs.

The root cause is that the build tools filter out some dependency
files, which are listed in inf source section, from the deps_target file.
Add those files back to deps_target file to resolve the above problem.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Bob Feng 859e09523d BaseTools: Sort the Pcd set when generating the VPD binary
If VPD PcdNvStoreDefaultValueBuffer is used, all DynamicHii and
DynamicExHii PCD value will be generated into that VPD.

In order to generate the same VPD binary file in every build,
sort the Pcd set when generating VPD.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
2020-09-08 08:48:24 +00:00
Cole Robinson 5d8648345c BaseTools: fix ucs-2 lookup on python 3.9
python3.9 changed/fixed codec.register behavior to always replace
hyphen with underscore for passed in codec names:

  https://bugs.python.org/issue37751

So the custom Ucs2Search needs to be adapted to handle 'ucs_2' in
addition to existing 'ucs-2' for back compat.

This fixes test failures on python3.9, example:

======================================================================
FAIL: testUtf16InUniFile (CheckUnicodeSourceFiles.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 375, in PreProcess
    FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path))
  File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 303, in OpenUniFile
    UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding)
  File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data
    Ucs2Info = codecs.lookup('ucs-2')
LookupError: unknown encoding: ucs-2

Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-08-14 02:21:18 +00:00
Bob Feng 5dc2699d10 BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel
AsBuiltInf can be created during AutoGen phase. Move CreateAsBuiltInf
into AutoGenWorker to make this step run in parallel.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-08-14 01:43:32 +00:00
Feng, Bob C bfe36cb4ef BaseTools: Improve the method of checking queue empty
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2807

The Queue.empty() method is not reliable in the multiple
process runtime environment. This patch uses a new method
to check if all modules are processed and workers need
to be stopped. That is to add a None item at the bottom
of the queue. Worker check if it gets that None item to
know if all the module is processed.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Lucy Yan <lucyyan@google.com>

Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-08-07 01:44:36 +00:00
Yuwei Chen d35773d5c0 BaseTools: Add Guid name support in GenFfs.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2762

The Fv Section in the FDF files use hard coding Guid values
which is inconvenient to manage. This patch adds Guid name
support in GenFfs to solve this problem.

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
2020-07-17 05:25:41 +00:00
Pierre Gondois 0622a7b1b2 BaseTools: Fix string concatenation
Using Python 3.7.2 on win32, when printing a FileBuildRule
instance, the following error occurs:
File "edk2\BaseTools\Source\Python\AutoGen\BuildEngine.py",
line 177, in __str__
  DestString = ", ".join(self.DestFileList)
  TypeError: sequence item 0: expected str instance, PathClass found

This patch converts each PathClass element of the list to a string
instance before concatenating them.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 6511277827 BaseTools: Generate multiple rules when multiple output files
This patch modifies the Makefile generation not to stop
adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a
module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files
are a final target. However, there is still [Rule2] to
apply to .Z files.

This patch also adds a dependency between the first
ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y
and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the
execution of the Makefile. Indeed, .Y and .Z files are
generated by the execution of a common set of commands,
and without this rule, there is no explicit dependency
relation between them.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Fan, ZhijuX 8c944c9383 BaseTools:GuidedSectionTools.txt is not generated correctly
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-31 04:09:43 +00:00
Fan, ZhijuX 2f524a745e BaseTools:Fix build tools print traceback info issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2610

We meet a case that the DEC file declaring the PCD isn't
included in the INF.it cause build tools report Traceback error.

Remove raise statements that generate Tracebacks that were only
intended for development/debug. With the raise statements removed
proper error messages are shown.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-24 02:27:27 +00:00
Bob Feng 2be4828af1 BaseTools: Remove invalid leading space before !INCLUDE in Makefile
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2563

This patch is to fix a incremental build regression bug
which happen when using nmake. That's introduced by 818283de3f.

If there is white space before !INCLUDE instruction, nmake will not
process it. Source code's dependent header files are listed in
${deps_file} file, if it's not included successfully, nmake will
not detect the change of those header file.

This patch has been verified in Windows with VS2015 and Linux with GCC5.
The header file add/modify/delete can trig the incremental build with this fix.
There is no impact on the clean build.

Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
2020-03-02 02:36:25 +00:00
Fan, ZhijuX 422bf2725b BaseTools:build failure in CLANGPDB tool chain
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2520

Incremental build failure in CLANGPDB tool chain on Windows host
The build failure is like below when do incremental build.
The root cause is in generated deps_target file. It has one line ":".

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-13 03:42:36 +00:00
Pierre Gondois c44e0a896c BaseTools: Remove caret in NASM_INC macro
NASM_INC contains the list of directory to include when using
the nasm assembler.

In nmake makefiles, a trailing backslash escapes the newline char
and replaces it with a space ' '. To have a literal trailing
backslash, it must be escaped with a caret '^'. This is not
necessary for GNU makefiles.

On windows platforms, for the NASM_INC macro, a caret escaping
a trailing a backslash was appended to the last included
folder regardless of the makefile type.
For instance, "/Include/" was replaced by "/Include/^\".

This is causing a build failure on windows platforms using
GNU makefiles since the caret '^' doesn't escape any chars in
GNU makefiles and is thus conserved.
"/Include^\" was replaced by "/Include\/" in nmake makefiles,
but remained "/Include/^\" in GNU makefiles.

This patch removes the caret '^' on the build using GNU makefiles.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Pierre Gondois 818283de3f BaseTools: Rationalise makefile generation
The GenMake.py script tests the platform environment
to determine the type of makefile that needs to be
generated. If a Windows build host is detected, the
makefile generated is of Nmake type. Otherwise a
GNUmake type is generated.

Furthermore, the <TARGET>_<TAGNAME>_<ARCH>_MAKE_PATH
option in tools_def.template defines the make tool
to use.
E.g.: for VS2017 this is configured to use Nmake, cf.
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
while for GCC5 it is setup to use GNU make.
*_GCC5_*_MAKE_PATH = DEF(GCC_HOST_PREFIX)make

This prevents using the GCC compiler toolchain on a
Windows build host.

To address this issue this patch introduces 2 factors
to determine the generated makefile output.
  1. Platform -> to determine shell commands used
                 in makefile.
  2. MakeTool -> to determine the type of makefile
                 that needs to be generated.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Bob Feng b34ed98694 BaseTools: Fixed a Incremental build issue
The .map file is not update to FFS_OUTPUT_DIR folder
in the incremental build.

Signed-off-by: Guo Dong <guo.dong@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-09 09:29:02 +00:00
Bob Feng 5cd3d4bc43 BaseTools: Fixed a incremental build bug
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2451

If removing a header file from source code and file
system, the incremental build will fail.

This patch is to fix this issue by setting each header file
as a target without any actions in makefile.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-01-19 07:46:18 +00:00
Fan, ZhijuX 4465cd124f BaseTools:Fix GenFds issue for BuildOption replace GenFdsOption
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2455

BuildOption is used by TargetTxtClassObj.py
GenFdsOption is used by GenFds.py
When the GenFds tool is used alone (e.g. python3 -m GenFds.GenFds -h)
With the OptionParser function, the first detected function
prints the help message

import TargetTxtClassObj to GenFds,
The BuildOption will be executed and replace GenFdsOption

We removed all objects associated with this problem that
were created directly during the import process
(e.g. BuildOption, BuildTarget = MyOptionParser(),
 TargetTxt = TargetTxtDict())

The Patch is going to fix this issue

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-01-13 02:08:46 +00:00
Bob Feng 78fb6b0e02 BaseTools: Fix build failure when multiple build targets given
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2371

This patch is to fix a regression issue that build fails
if multiple build targets given.

Two changes cause this regression issue.
One is AutoGen object __hash__ function only
hash file path and arch, missing ToolChain and build target.

The other is changing the multiple-thread-genfds function as default
build behavior. To generate the genffs command to Makefile, there
is a global data set is used, GenFdsGlobalVariable, which cause build
tool use the data of first build-target build in
the second build-target build.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-12-18 01:01:57 +00:00
Steven Shi a80032dc44 BaseTools: Remove redundant binary cache file
Redesign the binary cache and not need to save the
cache intermediate result and state in memory as a
ModuleBuildCacheIR class instance. So remove the
CacheIR.py which define the ModuleBuildCacheIR class.

Signed-off-by: Steven Shi <steven.shi@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-12-10 02:07:12 +00:00
Steven Shi fc8b8deac2 BaseTools: Leverage compiler output to optimize binary cache
Redesign the binary cache and bases on the compiler to
output the dependency header files info for every module.
The binary cache will directly consume the dependency header
files info and doesn't parse the C source code by iteself.
Also redesign the dependency files list format for module
and try to share the common lib hash result as more as
possible in local process. Remove the unnecessary share data
access across multiprocessing.

Signed-off-by: Steven Shi <steven.shi@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-12-10 02:07:12 +00:00
Steven Shi 3bfbc91507 BaseTools: enhance the CacheCopyFile method arg names
Enhance the CacheCopyFile method arg names to be more
clear and readable

Signed-off-by: Steven Shi <steven.shi@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-12-10 02:07:12 +00:00
Steven Shi 91f6c533f8 BaseTools: store more complete output files in binary cache
Binary cache use the OutputFile method to return the module
built output files needed to store in cache, but current
OutputFile implementation doesn't return complete output files.
Enhance the OutputFile method to return more complete output files.

Signed-off-by: Steven Shi <steven.shi@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-12-10 02:07:12 +00:00
Bob Feng 0c3e8e9947 BaseTools: Enhance Basetool for incremental build
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311

Include dependency file in Makefile to enhance
incremental build

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-12-10 01:31:55 +00:00
Philippe Mathieu-Daude 490a62beb7 BaseTools: Avoid "is" with a literal Python 3.8 warnings
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2304

The following statement produces a SyntaxWarning with Python 3.8:

  if str(FdRegion.RegionType) is 'FILE' and self.Platform.VpdToolGuid in \
    str(FdRegion.RegionDataList):
  BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py:168: SyntaxWarning: \
    "is" with a literal. Did you mean "=="?

Change the 'is' operator by the conventional '==' comparator.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Philippe Mathieu-Daude <philmd@redhat.com>
2019-12-06 03:07:37 +00:00
Fan, ZhijuX bf1ea933ec BaseTools:Add [packages] section in dsc file
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2270

Currently a PCD (e.g. FeaturePCD) cannot be used in a conditional
statement in a DSC/FDF file without a module in the build referencing
the PCD package DEC file.

An example implementation that to support this is to allow a [Packages]
section in the DSC file to list additional package dependencies for PCD
references in the package DSC/FDF files.

this patch is going to  add the ability to have the [packages] section
defined in the DSC file

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Acked-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-11-20 07:46:42 +00:00
Lin, Derek (HPS SW) b1c6e9f55e BaseTools: Fix an incremental build issue caused by macro in #include
When c/h file use macro after #include, for example,
In this case, GenMake is not able to create a healthy dependency for the c
file. GenMake used to add $(FORCE_REBUILD) dependency in the c file, this
guarantee the c file is always compiled in incremental build. But, this
function is broken since 05217d210e which
enable /MP for MSVC compiler, in order to compile multiple c files in one
command multi-processing. The fix here is adding '$(FORCE_REBUILD)' back to
retain the original function.

Line number 1728 and 978 are the code pieces which handle this logic.

Signed-off-by: Derek Lin <derek.lin2@hpe.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-10-18 08:30:38 +08:00
Leif Lindholm c1bcb2258c BaseTools: add missing newlines at end of files
Some scripts in Source/Python were missing newlines at end of files,
so add them.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
2019-10-04 11:18:22 +01:00
Kubacki, Michael A 61af5f2494 BaseTools: Fix GenMake multi-workspace failure
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2232

Commit 0075ab2cec introduced an issue that causes an exception
when multiple workspace packages paths are specified. For example,
if edk2-platforms is used, the root directory will contain an edk
and edk2-platforms directory representing the respective
repositories.

In GenMake, the path to the package DEC file for a module is
discovered by getting the relative path of the INF to the
workspace root directory. Each directory in the relative path
is incrementally joined to the WORKSPACE directory. The file
list in the joined path is searched for a DEC file.

As an example, if the build command is used on a package outside
the edk2 repository, the INF file path is relative to the
edk2-platforms directory not edk2. This causes directory paths
to be built that do not exist. Commit 0075ab2cec replaced the
os.path.exists() call with a try except block that always fails
when os.listdir() is invoked to enumerate the list of files in
the built directory path on packages outside edk2.

This commit restores the original conditional statement which
avoids calling os.listdir() with an invalid directory path.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Michael Kubacki <michael.a.kubacki@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by:  Bob Feng <bob.c.feng@intel.com>
2019-10-04 11:13:37 +08:00
Feng, Bob C 1f5e4d9128 BaseTools: Fix the lib order in static_library_files.lst
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2216

This patch is going to fix the lib order in static_library_files.lst.
This issue is introduced by commit 673d09a2dd.
Before 673d09a2dd, build tool apply build rule for the module's library
firstly and then apply build rule for module itself. Now, build tool
apply build rule for module self and then for its library. That behavior
impact the lib order in static_library_files.lst.

This patch is to call module's LibraryAutoGenList function
explicitly, where the applying build rule action for
library is triggered.

Cc: Liming Gao <liming.gao@intel.com>
Cc: dann frazier <dann.frazier@canonical.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Tested-by: dann frazier <dann.frazier@canonical.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-29 11:59:31 +08:00
Feng, Bob C 4747b92cab BaseTools: init DependencyHeaderFileSet for each MakeFile class
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2203

This patch is going to fix a regression issue that
build breaks when the custom makefile exist.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-24 10:29:09 +08:00
Liming Gao fcdedafd97 Revert "BaseTools: Improve GetDependencyList function"
This reverts commit bc9e4194cf.
This change causes the dependent header files are missing in Makefile.
It makes the incremental build not work. So, revert this change.

Cc: Bob Feng<bob.c.feng@Intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@Intel.com>
2019-09-21 00:25:48 +08:00
Feng, Bob C 0075ab2cec BaseTools: remove unnecessary calls of os.exist
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2101

This patch is going to remove unnecessary calls
of os.exist()

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Feng, Bob C bc9e4194cf BaseTools: Improve GetDependencyList function
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2102

GetDependencyList get the header file via
re.findall in the whole header file.

This patch is to pre-process the header file and
to feed the shorter string to re.findall.

This patch is to improve GetDependencyList() efficiency

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-17 10:19:05 +08:00
Fan, ZhijuX e32f7bc96d BaseTools:change some incorrect parameter defaults
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1858

for Dict={},There are pitfalls in the way this default parameter is set
and Dict is not used in functions, other functions have these two cases,
I will change some incorrect parameter defaults

This patch is going to fix this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2019-09-17 10:18:51 +08:00
Feng, Bob C 7809492c10 BaseTools: Enable --genfds-multi-thread to default build
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302

This patch enable --genfds-multi-thread to default build.
This patch keep --genfds-multi-thread build option for
compatibility and also add a new build option to disable
genfds-multi-thread as --no-genfds-multi-thread.

Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-11 09:43:59 +08:00
Feng, Bob C e3c8311f8a BaseTools: Fixed the bug of multi-thread genffs for override inf
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302
If there is a inf override, and multi-thread genffs is enabled, build
will fail.

This patch is going to fix this bug.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2019-09-11 09:43:58 +08:00