buildoptions is used by build.py and TargetTxtClassObj.py
Move buildoptions.py to Common folder so that
build tool and ECC tool work fine.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804
The check to see if [Sources] section lists all the header type
files of a module is missing the exclusion of source files that
fall under the scope of Package includes. This change adds the
exclusions.
Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Tested-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2058
If there are multiple module instances with different
guid used in .dsc file, the temp path of the module will
contain 8-4-4-4-12 format guid string. So "-" need to be
added into the regular expression for parsing it correctly.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2078
This is a build tool regression issue that
is introduced by multiple-process-autogen.
The workspace timestamp value is not pass
to module autogen object correctly, so build tool
does not detect the change of .dsc and ignore the autogen.
This patch is to fix this issue.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Tested-by: Michael D Kinney <michael.d.kinney@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2047
Build raise exception if there is libraryclass
missing in dsc file and --pcd build option is used.
Build should report the missing libraryclass but not
a call stack.
This patch is going to fix it.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
python3 change the module name of Queue to queue.
python3 add a new log handler of QueueHandler.
This patch is to make Multiple process AutoGen
feature work for python2
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
Build tool supports user to specify the conf folder.
To make the build options be evaluated at the beginning
of launching build, extract the buildoption function
from build.py to a new .py file.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
AutoGen processes race the logfile. To resolve this issue,
this patch create a LogAgent thread in main process to write
the log content to console or file, Other process will send
the log content to the LogAgent.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
1. Separate the AutoGen.py into 3 small py files.
One is for AutoGen base class, one is for WorkspaceAutoGen class
and PlatformAutoGen class, and the one for ModuleAutoGen class.
2. Create a new class DataPipe to store the Platform scope settings.
Create a new class PlatformInfo to provide the same interface
as PlatformAutoGen. PlatformInfo class is initialized by
DataPipe instance.
Create a new class WorkspaceInfo to provide the same interface
as WorkspaceAutoGen. WorkspaceInfo class is initialized by
DataPipe instance.
3. Change ModuleAutoGen to depends on DataPipe, PlatformInfo and
WorkspaceInfo. Remove the dependency of ModuleAutoGen to PlatformAutoGen.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
These functions are used for get platform scope
build options. They will be used in later patches.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
The WorkspaceAutoGen.__InitWorker function is too long, it's hard
to read and understand.
This patch is to separate the __InitWorker into multiple small ones.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
The build config files are target.txt, build rule, tooldef
During a build, the config is not changed, so the object to
handle them need to be singleton.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Liming Gao <liming.gao@intel.com>
This is a regression issue introduced
by commit 307e1650be
This patch is to fix this issue.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2016
Added arch output to build report so it's easy to tell
which arch is being generated. Useful when multiple versions
of a single module is being emitted for multiple archs.
This patch is going to Added arch output
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2015
Not all output works in utf-8, so change the encoding to
the default
This patch is going to fix that issue.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2014
Add extra debugging to improve error identification.
Error while processing file if the file is read incorrectly
This patch is going to fix that issue.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2013
chinese quotation mark are used in the file
This patch is going to fix that issue.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
This patch is going to sort the file list in generated
Makefile. This change make the autogen makefile easy to
compare.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2003
lstrip(parameter) do the match based on the char
in parameter but not only the whole parameter string.
In GenMake line 1082,
CmdSign.lstrip('/Fo') will strip the '/' or
'F' or 'o' on the left of CmdSign. This is not expected.
This patch is going to fix such issue.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1986
Create "Conf/.cache" folder as early as possible
so that the later code do need to check
if it exits and then create it.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Building with python3.8 shows a warning like:
SyntaxWarning: invalid escape sequence \(
GuidName = re.compile("\(GUID=[-a-fA-F0-9]+")
It seems harmless, but it's easy enough to fix: mark the string as
raw with the 'r' prefix like is used elsewhere in the file
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
It can be used to indicate a module can be build to run
as OS application and run in OS environment.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Xiaoyu Lu <xiaoyux.lu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
ToolDefinitionFile is generated by PlatformAutoGen.ToolDefinition()
Code assume ToolDefinition is always called before using
ToolDefinitionFile, but in some cases, it's not true.
This patch is to fix this issue.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
This reverts commit dc7b0dc8d6.
There are the concerns on code design and code quality, and
request to rewrite FCE, BfmLib and FMMT for the review.
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Feng Bob C <bob.c.feng@intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Feng Bob C <bob.c.feng@intel.com>
This reverts commit 3c59d94637.
There are the concerns on code design and code quality, and
request to rewrite FCE, BfmLib and FMMT for the review.
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Feng Bob C <bob.c.feng@intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Feng Bob C <bob.c.feng@intel.com>
This reverts commit 080981d72d.
There are the concerns on code design and code quality, and
request to rewrite FCE, BfmLib and FMMT for the review.
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Feng Bob C <bob.c.feng@intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Feng Bob C <bob.c.feng@intel.com>
This reverts commit d031fc07eb.
There are the concerns on code design and code quality, and
request to rewrite FCE, BfmLib and FMMT for the review.
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Feng Bob C <bob.c.feng@intel.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Feng Bob C <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1921
Add TOOLS_DEF.ARCH file to all workspace meta files list.
TOOLS_DEF.ARCH include the evaluated Tool definition information
which is filtered by current tool_chain.
With this change, when the environment variable which
is used in ToolDef.txt is changed, build tool will
rebuild the platform.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
This issue happens when using Structured PCD.
Build tool use set to store the cflags for compile PcdValueInit.c,
that is the order of cflags is disorder.
This patch make -U, /U flags appear before -D, /D
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Commit 3c59d94637 ("BaseTools/FCE: Add a tool FCE") added a new tool
that is always built. Pre-existing tools locate the makefile includes
with relative paths, but FCE uses EDK_TOOLS_PATH, adding a new dependency
on having sourced edksetup.sh before building BaseTools.
Change FCE GNUmakefile to match other tools.
Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
FMMT is a tool to enable removal, addition and replacement of
FFS files in FD image binaries.
https://bugzilla.tianocore.org/show_bug.cgi?id=1847
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
FCE is a tool to retrieve and change HII configuration data in
Firmware Device(*.fd) files.
https://bugzilla.tianocore.org/show_bug.cgi?id=1848
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=1834
1)Add arguments "--embedded-driver" to support embedded driver
in command line.
2)Add arguments "--update-image-index" to identify ImageIndex
within the device in command line.
3)Add arguments "-j JSONFILE" to support multiple payloads and
embedded drivers with JSON file.
The update is in a backwards compatible manner, so all command
line options to support single payload are still supported. But
all the options associated with multiple payloads should be
provided in a JSON file.
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Eric Jin <eric.jin@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1932
There are two functions in current CreateAsBuiltInf, Copy Binary files
to build cache folder and create asbuild inf file.
This patch is to separate UpdateBuildCache and CreateAsBuiltInf into
two functions.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Reviewed-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1932
ModuleAutogen.CreateAsBuiltInf IsOnlyCopy parameter
and related function can be moved out of CreateAsBuiltInf,
so that CreateAsBuiltInf is more clean.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1872
Since MultipleWorkspace is not imported in FvImageSection,
an error of "global name 'MWS' is not defined" appeared.
This patch is going to fix that issue.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1894
Basetool need a CopyFileOnChange() function to avoid cache
file writing race in multi-thread build. Some platforms
build fail with file IO writing race issue when the
build cache is enabled to store cache files in multi-threads.
This is because common same library cache files (e.g. some
libs in MdePkg) can be stored by many different driver modules'
build threads at same time. Current build cache need a function
to check whether the same cache file already exist, and only
copy source file if it is different from the destination file.
This patch introduces an atomic copy function to avoid duplicated
cache files copy.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=1895
Build cache cannot store cache for the same library modules
in different arch together. E.g. Both the below IA32 and X64
arch BaseLib caches should exist after build Ovmf3264, but now
only the one in X64 arch exist.
The reason is the current Basetool use a set() to same all
library AutoGen objects, but the different arch lib AutoGen
objects have same __hash_ value which comes from the lib
MetaFile(The path of module file):
def __hash__(self):
return hash(self.MetaFile)
So the different arch lib AutoGen objects are duplicated one
to the set() and only one can exist. This is why the Basetool
can only store one arch cache for library.
This patch adds the arch string into the PlatformAutoGen and
ModuleAutoGen __hash_ definitions and ensure the different
platform and module AutoGen objects have different __hash_ values.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1868
Build need output the cache miss or hit notification
message when consume the build cache. Current build does not
output any message which is not clear for user to know
whether the module built result is from cache or not.
This patch adds message about the cache miss or hit when
build consumes the cache.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Zhiju.Fan <zhijux.fan@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1867
Current build cache cannot store the cache for library package.
build -p MdePkg\MdePkg.dsc -a IA32 -b DEBUG -t VS2015x86 --hash
--binary-destination=BinCache
After build, the expected result is the BinCache folder is generated
and the MdePkg build cache files (e.g. .hash and .lib) are stored in
the BinCache folder. But the BinCache folder is not generated at all.
This patch is going to fix that issue.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Zhiju.Fan <zhijux.fan@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804
After adding a check to see if [Sources] section lists all the header
type files of a module, track module and library hashes for --hash
feature. If above check is not in compilance for a library or module,
force hash invalidation on that library or module.
Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804
Add a check to see if [Sources] section lists all the header type
files of a module. Performance impact should be minimal with this patch
since information is already being fetched for Makefile purposes. All
other information is already cached in memory. No extra IO time is needed.
Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1765
If RAW FFS File Rule has no section for its data.For RAW FFS File,
directly call GenFfs tool to generate FFS file.
Ffs Rule:
[Rule.Common.USER_DEFINED.MicroCode]
FILE RAW = $(NAMED_GUID) {
$(INF_OUTPUT)/$(MODULE_NAME).bin
}
[Rule.Common.USER_DEFINED.LOGO]
FILE RAW = $(NAMED_GUID) {
|.bmp
}
As shown in the rule above,if SectionType and FileType not defined,
FFS files are generated directly, and no other type of file is
generated.
The patch is to make the BaseTools support these two rules
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=1742
Current Binary Cache doesn't update the restored file
creation and modification times to the current time.
Preserve the new restored file creation time as old
cached time might has potential issue to block the make
to build updated files based on the time stamp.
Enhance to update the restored file creation time to
current time.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Zhiju Fan <zhijux.fan@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=1797
Current binary cache doesn't support to save and restore
the library module. If a driver module cache miss happen,
all its dependency library modules need rebuild which
is very time-consuming. This patch is to extend the binary
cache to support library.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>