Commit Graph

1208 Commits

Author SHA1 Message Date
Mingyue Liang 6c4efc0509 BaseTools: Resolve index out of range errors.
This problem is generated by solving bz2972's
patch, and the commit ID is
0af7f8e6a9.

This is a problem when updating the DEPs file.
The code does not consider that there is only
one line of content in the file, so the filter
condition is added to prevent the index from
exceeding the range.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-11-19 02:22:57 +00:00
Mingyue Liang 8c610e6075 BaseTools: Limit command line length.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2528

Currently, CL command contains multiple C files will be compiled,
and that caused command line too long, which may trigger build error.

In order to solve this issue, the following rules is used in this scene:

If the number of C files is greater than one, a txt file will be used
to record these C files, and replaces the corresponding content in
command line with the file name.

Else (only one C file listed in the command line), the length of the
whole CL command line will determine whether use a file to record. If
the length exceeds the limited max length, use the recording file; else
C file name directly listed in the command line

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-11-10 05:49:38 +00:00
Yunhua Feng 8577d63cd8 BaseTools: Update report map file format
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2977

For a source-level BIOS debugger the .map files are quite useful with one
major shortcoming: the debugger cannot know, solely from the .map file,
the format (PE/COFF vs. TE) of the image included in the final BIOS ROM

Update report map file format

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Yunhua Feng <fengyunhua@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-11-10 04:22:27 +00:00
Mingyue Liang 0af7f8e6a9 BaseTools: Incremental build issue for included ASI file's deletion.
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2972

When using incremental build to delete an included xxx.asi file from
the ASL file, the xxx.asl.trim.deps file generated by previous build
process will not be deleted from the OUTPUT directory, which caused
the dependency file still include the xxx.asl.trim.deps file.

If the include file is deleted and DEPs is empty.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-11-10 00:24:06 +00:00
Bob Feng 09af9bd9be BaseTools: Enable Module Scope Structure Pcd
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2648

This patch is to enable the Module scoped Structure Pcd usage.
User can set structure pcd field value in module scope. For example,
under the [components] section of a dsc file, user can override some
field value for a specific module.

  Package/Module.inf{
      <PcdsFixedAtBuild>
      gUefiTokenSpaceGuid.StructurePcdModule.FieldName | 5
  }

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Tested-by: Liming Gao <gaoliming@byosoft.com.cn>
Acked-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-11-05 04:27:28 +00:00
Bob Feng 8d57088335 BaseTools: Remove the dependency on the build intermediate file
When generating compressed section, the build tool rely on the
build intermediate files, which were generated in last build, to
get the file list. This method will cause the incremental build to
generate incorrect build result. To reproduce this incremental build
error, you can do:
1. build Ovmf
2. change the module OvmfPkg\AcpiTables a source file Facp.aslc
name from Facp.aslc to Facpxxx.aslc.
3. change the Facp.aslc file name in [sources] section of AcpiTables.inf
4. incremental build Ovmf

you will see the in AcpiTables module Makefile, the corresponding
Facp.acpi file is not changed.

This patch is to make the build always get file list from the INF.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-11-04 12:00:16 +00:00
fengyunhua 0166dad496 BaseTools: Update the FV Space Information to display decimal and Hex
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3009

Update the FV Space Information to display decimal and Hex

FV Space Information
before format:
SECFV [13%Full] 212992 total, 28400 used, 184592 free
Updated format:
SECFV [13%Full] 212992 (0x34000) total, 28400 (0x6ef0) used,
184592 (0x2d110) free

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Yunhua Feng <fengyunhua@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-11-03 01:40:43 +00:00
Yunhua Feng eb520b93d2 BaseTools/Ecc: Fix ECC check MetaFile Copyright information issue
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3011

MetaFile Copyright support below format:

#  Copyright (C) 2020 Hewlett Packard Enterprise Development LP<BR>
#  (C) Copyright 2020 Hewlett Packard Enterprise Development LP<BR>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Yunhua Feng <fengyunhua@byosoft.com.cn>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-10-27 06:50:06 +00:00
fengyunhua b65afdde74 BaseTools: Add EDKII_DSC_PLATFORM_GUID MACRO
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2969

Add EDKII_DSC_PLATFORM_GUID MACRO to AutoGen.h and AutoGen.c

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Yunhua Feng <fengyunhua@byosoft.com.cn>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Tested-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-10-13 02:35:08 +00:00
gaoliming 151c270899 BaseTools: Move C tool flags before the common flags
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2938

C tool may add the additional INC include path. They should have
high priority than the common INC include path.
This fix is to resolve the structure PCD issue to refer to the same
header file defined in BaseTools and MdePkg. The one in MdePkg should
be used.

Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Tested-by: Garrett Kirkendall <garrett.kirkendall@amd.com>
2020-10-10 05:43:11 +00:00
Bob Feng d8be01079b BaseTools: Set section alignment as zero if its type is Auto
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2881

Currently, the build tool try to read the section alignment
from efi file if the section alignment type is Auto.
If there is no efi generated, the section alignment will
be set to zero. This behavior causes the Makefile to be different
between the full build and the incremental build.

Since the Genffs can auto get the section alignment from
efi file during Genffs procedure, the build tool can just set section
alignment as zero. This change can make the autogen makefile
consistent for the full build and the incremental build.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Bob Feng 3a7a676114 BaseTools: Remove CanSkip calling for incremental build
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2978

If a module add a new PCD, the pcd token number will be
reassigned. The new Pcd token number should be updated
to all module's autogen files. CanSkip can only detect a
single module's change but not others. CanSkip block the
pcd token number update in incremental build, so this
patch is going to remove this call.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Mingyue Liang 9641a7f975 BaseTools: Normalize case of pathname when evaluating Macros.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880

Currently, When doing the Incremental build, the directory
macros extended to absolute path in output Makefile, which
is inconsistent with the output of Clean build.

When we do macro replacement, we can't replace macro due to
inconsistent path case, which results in inconsistent display
of incremental build and clean build in makefile.Therefore,
the path is converted to achieve the correct macro replacement.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Bob Feng a8c77eba37 BaseTools: Clean the ffs folder before generating files in it
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2965

The content in Guid.xref depends on the files under the corresponding
ffs folder.(refer to the commit 5e9256cd7f)
To make Guid.xref update in the incremental build,
clean the files under that ffs folder before generating files in it.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Mingyue Liang aec99d9bc3 BaseTools: Add included files to deps_target file.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2882

After changing the name of the include source file,
when doing incremental build, the previous source file
is not covered in the. DEPs file, and a build error occurs.

The root cause is that the build tools filter out some dependency
files, which are listed in inf source section, from the deps_target file.
Add those files back to deps_target file to resolve the above problem.

Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2020-09-25 02:27:35 +00:00
Bob Feng 859e09523d BaseTools: Sort the Pcd set when generating the VPD binary
If VPD PcdNvStoreDefaultValueBuffer is used, all DynamicHii and
DynamicExHii PCD value will be generated into that VPD.

In order to generate the same VPD binary file in every build,
sort the Pcd set when generating VPD.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
2020-09-08 08:48:24 +00:00
Bob Feng 7513559926 BaseTools/Ecc: Fix an issue of path separator compatibility
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2904

The path separator is different in Windows and Linux, the
original code does not handle this difference. This patch
is to fix this issue.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Shenglei Zhang <shenglei.zhang@intel.com>
Message-Id: <20200901102315.38840-1-bob.c.feng@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
2020-09-01 17:55:56 +00:00
Cole Robinson 43bec9ea3d BaseTools: Work around array.array.tostring() removal in python 3.9
In python3, array.array.tostring() was a compat alias for tobytes().
tostring() was removed in python 3.9.

Convert this to use tolist() which should be valid for all python
versions.

This fixes this build error on python3.9:

(Python 3.9.0b5 on linux) Traceback (most recent call last):
  File "/root/edk2/edk2-edk2-stable202002/BaseTools/BinWrappers/PosixLike/../../Source/Python/Trim/Trim.py", line 593, in Main
    GenerateVfrBinSec(CommandOptions.ModuleName, CommandOptions.DebugDir, CommandOptions.OutputFile)
  File "/root/edk2/edk2-edk2-stable202002/BaseTools/BinWrappers/PosixLike/../../Source/Python/Trim/Trim.py", line 449, in GenerateVfrBinSec
    VfrUniOffsetList = GetVariableOffset(MapFileName, EfiFileName, VfrNameList)
  File "/root/edk2/edk2-edk2-stable202002/BaseTools/Source/Python/Common/Misc.py", line 88, in GetVariableOffset
    return _parseForGCC(lines, efifilepath, varnames)
  File "/root/edk2/edk2-edk2-stable202002/BaseTools/Source/Python/Common/Misc.py", line 151, in _parseForGCC
    efisecs = PeImageClass(efifilepath).SectionHeaderList
  File "/root/edk2/edk2-edk2-stable202002/BaseTools/Source/Python/Common/Misc.py", line 1638, in __init__
    if ByteArray.tostring() != b'PE\0\0':
AttributeError: 'array.array' object has no attribute 'tostring'

Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-08-14 02:21:18 +00:00
Cole Robinson 5d8648345c BaseTools: fix ucs-2 lookup on python 3.9
python3.9 changed/fixed codec.register behavior to always replace
hyphen with underscore for passed in codec names:

  https://bugs.python.org/issue37751

So the custom Ucs2Search needs to be adapted to handle 'ucs_2' in
addition to existing 'ucs-2' for back compat.

This fixes test failures on python3.9, example:

======================================================================
FAIL: testUtf16InUniFile (CheckUnicodeSourceFiles.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 375, in PreProcess
    FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path))
  File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 303, in OpenUniFile
    UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding)
  File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data
    Ucs2Info = codecs.lookup('ucs-2')
LookupError: unknown encoding: ucs-2

Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-08-14 02:21:18 +00:00
Bob Feng 5dc2699d10 BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallel
AsBuiltInf can be created during AutoGen phase. Move CreateAsBuiltInf
into AutoGenWorker to make this step run in parallel.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-08-14 01:43:32 +00:00
Feng, Bob C bfe36cb4ef BaseTools: Improve the method of checking queue empty
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2807

The Queue.empty() method is not reliable in the multiple
process runtime environment. This patch uses a new method
to check if all modules are processed and workers need
to be stopped. That is to add a None item at the bottom
of the queue. Worker check if it gets that None item to
know if all the module is processed.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Lucy Yan <lucyyan@google.com>

Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-08-07 01:44:36 +00:00
Feng, YunhuaX d0da48f112 BaseTools: Fix binary file not generate map file issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2624

When EFI file come from binary file, not generate .map file, so need ignore the map file.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-23 03:03:00 +00:00
Bob Feng b3ee616e67 BaseTools: Remove the file timestamp checking during parsing
During build, the meta files are not changed, so it's no need
to check file timestamp.

This patch is to remove useless logic.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-07-21 04:12:32 +00:00
Yuwei Chen d35773d5c0 BaseTools: Add Guid name support in GenFfs.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2762

The Fv Section in the FDF files use hard coding Guid values
which is inconvenient to manage. This patch adds Guid name
support in GenFfs to solve this problem.

Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
2020-07-17 05:25:41 +00:00
Michael D Kinney 5b86bbf891 BaseTools/Python: Allow HOST_APPLICATION to use NULL libraries
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2797

Update HOST_APPLICATION module type to use NULL library instances.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Sean Brogan <sean.brogan@microsoft.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-07-15 05:25:21 +00:00
Pierre Gondois 0622a7b1b2 BaseTools: Fix string concatenation
Using Python 3.7.2 on win32, when printing a FileBuildRule
instance, the following error occurs:
File "edk2\BaseTools\Source\Python\AutoGen\BuildEngine.py",
line 177, in __str__
  DestString = ", ".join(self.DestFileList)
  TypeError: sequence item 0: expected str instance, PathClass found

This patch converts each PathClass element of the list to a string
instance before concatenating them.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 0a4aa20e8d BaseTools: Compile AML bytecode arrays into .obj file
The AmlToHex script and Posix/WindowsLike wrappers convert
an AML file to a .hex file, containing a C array storing
AML bytecode. This ".hex" file can then be included in a
C file, allowing to access the AML bytecode from this C
file.

The EDK2 build system doesn't allow to a depict dependency
orders between files of different languages. For instance,
in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a
".c" file since this later ".hex" file may or may not
have been created yet.

This patch modifies the AmlToC script to generate a C file
instead of a ".hex" file.
It also adds the generation of an intermediate ".amli" file
when compiling an ASL file, and adds a rule to convert this
".amli" to a C file.

This allows to generate a C file containing the AML bytecode
from an ASL file. This C file will then be handled by the EDK2
build system to generate an object file.
Thus, no file inclusion will be required anymore. The C file
requiring the AML bytecode as a C array, and the ASL file,
will be compiled independently. The C array must be defined
as an external symbol. The linker is resolving the
reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;
 -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 88228db38e BaseTools: Rename AmlToHex script to AmlToC
The AmlToHex script and Posix/WindowsLike wrappers convert
an AML file to a .hex file, containing a C array storing
AML bytecode. This ".hex" file can then be included in a
C file, allowing to access the AML bytecode from this C
file.

The EDK2 build system doesn't allow to a depict dependency
orders between files of different languages. For instance,
in a module containing a ".c" file and a ".asl", the ".c"
file may or may not be built prior to the ".asl" file.
This prevents any inclusion of a generated ".hex" in a
".c" file since this later ".hex" file may or may not
have been created yet.

This patch renames the script as AmlToC. It is posted as
a separate patch to prevent git from seeing the renaming
as a deletion plus addition of a new file.
The ending line of the posix-like bin-wrapper script has
also been corrected.

This is a first step toward generating a C file containing
the AML bytecode from an ASL file. This C file will then
be handled by the EDK2 build system to generate an object
file.
Thus, no file inclusion will be required anymore. The C file
requiring the AML bytecode as a C array, and the ASL file,
will be compiled independently. The C array must be defined
as an external symbol. The linker is resolving the
reference to the C array symbol.

To summarize, the flow goes as:
 -1. ASL file is compiled to AML;
 -2. AML file is copied to a ".amli" intermediate file;
 -3. EDK2 build system applies the rule relevant to ".amli"
     files. This is, calling the "AmlToC" script, generating
     a C file from the ".amli" file;
 -4. EDK2 build system applies the rule relevant to C files.
     This is creating an object file.
 -5. EDK2 build system links the object file containing the
     AML bytecode with the object file requiring it.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Pierre Gondois 6511277827 BaseTools: Generate multiple rules when multiple output files
This patch modifies the Makefile generation not to stop
adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a
module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files
are a final target. However, there is still [Rule2] to
apply to .Z files.

This patch also adds a dependency between the first
ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y
and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the
execution of the Makefile. Indeed, .Y and .Z files are
generated by the execution of a common set of commands,
and without this rule, there is no explicit dependency
relation between them.

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
Suggested-by: Tomas Pilar <Tomas.Pilar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-07-02 10:09:17 +00:00
Bob Feng cfd73e0065 BaseTools: Warn user the file not found issue instead of break build.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2770

The Trim.py would break the build process when the file not found
issue occures, however sometimes we do not care about this issue.
This patch changes the error with warning in order to solve this
kind of break.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
2020-06-07 13:36:33 +00:00
Irene Park 28dd887d68 BaseTools/build.py: Exit with 1 when AutoGen error occurred
AutoGen manager/workers halt the progress when an error occurs but
doesn't propagate the error code to main and allows main exit with 0
and gets the build system unable to catch the occurrence of an error.
This change informs main with an error when a progress is halted and
helps main exit with 1.

Signed-off-by: Irene Park <ipark@nvidia.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
2020-06-07 12:51:25 +00:00
Feng, YunhuaX ceacd9e992 BaseTools: Fix parse PCD GUID expression issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2686

The build tool will give an incorrect GUID value if the GUID includes character ' or " ASCII value.
This patch is going to fix this issue.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-05-13 06:37:18 +00:00
Michael Kubacki 242ab73d7f BaseTools/Ecc: Replace deprecated function time.clock()
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2707

Ecc fails with Python 3.8 because it uses the deprecated time.clock()
function - https://docs.python.org/3.7/library/time.html#time.clock

This change updates EccMain.py to use time.perf_counter().

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-05-13 05:52:03 +00:00
Abner Chang f60d5ca97f BaseTools: BaseTools changes for RISC-V platform.
Python code changes for building EDK2 RISC-V platform.

Signed-off-by: Abner Chang <abner.chang@hpe.com>
Co-authored-by: Gilbert Chen <gilbert.chen@hpe.com>
Co-authored-by: Daniel Helmut Schaefer <daniel.schaefer@hpe.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Gilbert Chen <gilbert.chen@hpe.com>
2020-04-29 02:52:08 +00:00
Fan, Zhiju 3a3a3af4a2 BaseTools:Add the spare space FV image size checker
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654

If FV is placed in FD region, its FV image size is fixed.
When FV image size exceeds it, it will trig the build break.
To alert the developer to adjust FV image size earlier,
I request to add new checker for the the spare FV space.
When the spare FV space is less than the specified threshold,
build tool will report the error.

This checker is the optional.
It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=10000.
Macro is the value of the spare space threshold size.
It can be decimal or hex format. If it is enabled,
BaseTools will check every FV with the fixed size.
If FV doesn't meet with the size requirement,
Build tool will report error message to say there is no
enough spare space.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-04-23 01:29:34 +00:00
Fan, ZhijuX 8c944c9383 BaseTools:GuidedSectionTools.txt is not generated correctly
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2538

For LzmaCompress or BrotliCompress, the platform may use the different
options and add their batch file, such as LzmaCompressPlatform.
Then, specify it in platform.dsc [BuildOptions] to override the default
one in tools_def.txt.

*_*_*_LZMA_PATH = LzmaCompressPlatform

This override tool will be used. But, its name is not specified in the
generated GuidedSectionTools.txt.

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-31 04:09:43 +00:00
Fan, ZhijuX 2f524a745e BaseTools:Fix build tools print traceback info issue
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2610

We meet a case that the DEC file declaring the PCD isn't
included in the INF.it cause build tools report Traceback error.

Remove raise statements that generate Tracebacks that were only
intended for development/debug. With the raise statements removed
proper error messages are shown.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-24 02:27:27 +00:00
Fan, ZhijuX 0dee1d1358 BaseTools:fix issue for decode the stdout/stderr byte arrays
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2365

This patch is to fix a build tool regression issue which was introduced
by commit 8ddec24dea.

compiler output message includes localized string.
So build failed when code decode the stdout/stderr byte arrays.
The cause of the build failed is that Commit 8ddec24dea
removed "errors='ignore'".

The build tool does not need to deal with localized string,
so we need to add "errors='ignore'".

this function is only invoked for structure PCDs.
Build failed if structurePcd is used in platform dsc file.
The patch is going to fixed this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-24 02:27:27 +00:00
Fan, ZhijuX 484b1534ed BaseTools:copy the common PcdValueCommon.c to output directory
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2568

PcdValueInit shares the same Edk2\BaseTools\Source\C\PcdValueCommon.c.
To avoid the conflict, it should copy this file to its output directory,
If so, PcdValueCommon.c file will be private for PcdValueInit

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-03-12 02:02:55 +00:00
Bob Feng 2be4828af1 BaseTools: Remove invalid leading space before !INCLUDE in Makefile
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2563

This patch is to fix a incremental build regression bug
which happen when using nmake. That's introduced by 818283de3f.

If there is white space before !INCLUDE instruction, nmake will not
process it. Source code's dependent header files are listed in
${deps_file} file, if it's not included successfully, nmake will
not detect the change of those header file.

This patch has been verified in Windows with VS2015 and Linux with GCC5.
The header file add/modify/delete can trig the incremental build with this fix.
There is no impact on the clean build.

Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
2020-03-02 02:36:25 +00:00
Fan, ZhijuX 422bf2725b BaseTools:build failure in CLANGPDB tool chain
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2520

Incremental build failure in CLANGPDB tool chain on Windows host
The build failure is like below when do incremental build.
The root cause is in generated deps_target file. It has one line ":".

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-13 03:42:36 +00:00
Fan, ZhijuX ec97412b7c BaseTools: Fixed build failure when using python38
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2304

SyntaxWarning: "is" with a literal. Did you mean "=="?
Using "is" instead of "==" is an irregular syntax

Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Cc: Bob C Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 14:54:03 +00:00
Fan, ZhijuX 82af1cbf0d BaseTools:fix Ecc tool issue for check StructPcd
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2142

gAdvancedFeaturePkgTokenSpaceGuid.PcdSmbiosType0BiosInformation|
{0x0}|SMBIOS_TABLE_TYPE0|0x80010000 {
    <HeaderFiles>
      IndustryStandard/SmBios.h
    <Packages>
      MdePkg/MdePkg.dec
      AdvancedFeaturePkg/AdvancedFeaturePkg.dec
  }

If there's a PcdStructHF or PcdStructPKGs in StructPcd,
EccTool report error,IndexError: list index out of range

This patch is going to fix this issue

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 09:26:22 +00:00
Pierre Gondois c44e0a896c BaseTools: Remove caret in NASM_INC macro
NASM_INC contains the list of directory to include when using
the nasm assembler.

In nmake makefiles, a trailing backslash escapes the newline char
and replaces it with a space ' '. To have a literal trailing
backslash, it must be escaped with a caret '^'. This is not
necessary for GNU makefiles.

On windows platforms, for the NASM_INC macro, a caret escaping
a trailing a backslash was appended to the last included
folder regardless of the makefile type.
For instance, "/Include/" was replaced by "/Include/^\".

This is causing a build failure on windows platforms using
GNU makefiles since the caret '^' doesn't escape any chars in
GNU makefiles and is thus conserved.
"/Include^\" was replaced by "/Include\/" in nmake makefiles,
but remained "/Include/^\" in GNU makefiles.

This patch removes the caret '^' on the build using GNU makefiles.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Pierre Gondois 818283de3f BaseTools: Rationalise makefile generation
The GenMake.py script tests the platform environment
to determine the type of makefile that needs to be
generated. If a Windows build host is detected, the
makefile generated is of Nmake type. Otherwise a
GNUmake type is generated.

Furthermore, the <TARGET>_<TAGNAME>_<ARCH>_MAKE_PATH
option in tools_def.template defines the make tool
to use.
E.g.: for VS2017 this is configured to use Nmake, cf.
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
while for GCC5 it is setup to use GNU make.
*_GCC5_*_MAKE_PATH = DEF(GCC_HOST_PREFIX)make

This prevents using the GCC compiler toolchain on a
Windows build host.

To address this issue this patch introduces 2 factors
to determine the generated makefile output.
  1. Platform -> to determine shell commands used
                 in makefile.
  2. MakeTool -> to determine the type of makefile
                 that needs to be generated.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-12 02:34:44 +00:00
Bob Feng b34ed98694 BaseTools: Fixed a Incremental build issue
The .map file is not update to FFS_OUTPUT_DIR folder
in the incremental build.

Signed-off-by: Guo Dong <guo.dong@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-09 09:29:02 +00:00
Kinney, Michael D 45b0be3840 BaseTools/Build: Do not use Common.lib in Structured PCD app
https://bugzilla.tianocore.org/show_bug.cgi?id=2496

Reduce the build and env dependencies for the Structured PCD
application by removing the dependency on Common.lib that
is only built when BaseTools is built which does not
happen if pre-compiled BaseToools are used.  Change the
makefile for the Structure PCD application to build all
files from sources which adds PcdValueCommon.c to the
makefile.  Also remove PcdValueCommon.c from Common.lib.

With the change to the makefile for the Structured PCD
application, multiple C files are compiled.  Only
PcdValueInit.c contains the extra information expected
by the error/warning message parser.  Only parse the
DSC line number into an error message if there is an
error/warning in PcdValueInit.c.  Errors/warnings in
other files should be passed through.  This fixes a build
failure with no useful log information that was observed
when there was a compiler error in PcdValueCommon.c.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-07 07:04:16 +00:00
Pierre Gondois 1549651da6 BaseTools: Script for converting .aml to .hex
The "-tc" option of the iasl compiler allows to generate a
.hex file containing a C array storing AML bytecode.

An online discussion suggested that this "-tc" option
was specific to the iasl compiler and it shouldn't be relied
on. This conversation is available at:
https://edk2.groups.io/g/devel/topic/39786201#49659

A way to address this issue is to implement a compiler
independent script that takes an AML file as input, and
generates a .hex file.

This patch implements a Python script that converts an AML
file to a .hex file, containing a C array storing AML bytecode.
This scipt has been tested with the AML output from the
following compilers supported by the EDKII implementation:
  * Intel ASL compiler
  * Microsoft ASL compiler

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
2020-02-06 13:52:11 +00:00
Michael D Kinney b47fe2655d BaseTools/DscBuildData: Fix PCD autogen include file conflict
https://bugzilla.tianocore.org/show_bug.cgi?id=2494

When using structured PCDs, a C application is auto generated
to fill in the structured PCD value.  The C application uses
the standard include files <stdio.h>, <stdlib.h>, and <string.h>.
This C application also supports include paths from package DEC
files when a structured PCD declaration provides a <Packages>
list.  The complete list of include paths are -I options for
include paths from package DEC files and the compiler's standard
include paths.

-I include paths are higher priority than the standard include
paths.  If the -I included paths from package DEC files contain
<stdio.h>, <stdlib.h>, or <string.h> the wrong include files are
used to compile the C application for the structured PCD value.

Update GenerateByteArrayValue() to skip a package DEC include
paths that contain <stdio.h>, <stdlib.h>, or <string.h>.

Build failures were observed when adding a structured PCD to
CryptoPkg.  CryptoPkg contains <stdio.h>, <stdlib.h>, and
<string.h> in the path CryptoPkg/Library/Include to support
building Open SSL.  The Library/Include path is listed as a
private include path in CryptoPkg.dec.  Without this change, the
standard include files designed to support build OpenSLL are
used to build the structured PCD C application, and that build
fails.

Other packages that provide a standard C lib or a gasket for
a subset of the standard C lib will run into this same issue
if they also define and use a Structured PCD.  So this issue
is not limited to the CryptoPkg.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-02-04 20:46:22 +00:00
Bob Feng 5cd3d4bc43 BaseTools: Fixed a incremental build bug
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2451

If removing a header file from source code and file
system, the incremental build will fail.

This patch is to fix this issue by setting each header file
as a target without any actions in makefile.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>

Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2020-01-19 07:46:18 +00:00