Commit Graph

28628 Commits

Author SHA1 Message Date
Ray Ni 3b769c5110 UefiCpuPkg/CpuMp: Fix hang when StackGuard is enabled in 16-core cpu
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3167

When StackGuard is enabled, the CpuMp driver allocates
known good stacks for all CPUs for DF# and PF# exceptions.
It uses AllocatePool to do so.

The size needed equals to 64KB
= StackSize (2K) * ExceptionNumber (2) * NumberOfProcessors (16)

However, AllocatePool max allocation size is less than 64K.
To fix the issue, AllocatePages() is used.

Signed-off-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
2021-01-22 03:23:53 +00:00
Zeng, Star 6c5801be6e UefiCpuPkg RegisterCpuFeaturesLib: NumberOfCpus may be uninitialized
NumberOfCpus local variable in GetAcpiCpuData will be uninitialized
when CpuS3DataDxe runs before DxeRegisterCpuFeaturesLib (linked by
CpuFeaturesDxe) because there is no code to initialize it at
(AcpiCpuData != NULL) execution path.

The issue is exposed after cefad282fb
and 38ee7bafa7.
There was negligence in that code review.
One further topic may be "Could EDK2 CI be enhanced to catch this kind
of uninitialized local variable case?". :)

This patch fixes this regression issue.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Star Zeng <star.zeng@intel.com>
Message-Id: <20210121093944.1621-1-star.zeng@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-21 14:30:06 +00:00
Bob Feng 45962a05da BaseTools: Add unittest for Split tool
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3165

This patch is to add the unit test for Split python tool

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2021-01-21 11:05:43 +00:00
Bob Feng 5b4a97bbc3 BaseTools: Convert Split tool to python
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3165

There are 2 reasons to convert Split tool from C to Python.
1. We are in the process of moving the Basetools Python code
to a separate repository. But there still are many C tools under
edk2/BaseTools. To make all Basetools be in the separate repo,
we can convert the C tools to Python tools.
2. The original Split tool is very slow. This python tool can reduce
90% time.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
2021-01-21 10:19:09 +00:00
Laszlo Ersek 339371ef78 OvmfPkg/CpuS3DataDxe: do not allocate useless register tables
CpuS3DataDxe allocates the "RegisterTable" and "PreSmmInitRegisterTable"
arrays in ACPI_CPU_DATA just so every processor in the system can have its
own empty register table, matched by APIC ID. This has never been useful
in practice.

Given commit e992cc3f48 ("UefiCpuPkg PiSmmCpuDxeSmm: Reduce SMRAM
consumption in CpuS3.c", 2021-01-11), simply leave both
"AcpiCpuData->RegisterTable" and "AcpiCpuData->PreSmmInitRegisterTable"
initialized to the zero address. This simplifies the driver, and saves
both normal RAM (boot services data type memory) and -- in PiSmmCpuDxeSmm
-- SMRAM.

(This simplification backs out a good chunk of commit 1158fc8e2c
("OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug", 2020-03-04).
But CpuS3DataDxe still differs between UefiCpuPkg and OvmfPkg, due to the
latter supporting CPU hotplug; thus, we can't remove OvmfPkg/CpuS3DataDxe
altogether.)

Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3159
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Message-Id: <20210119155440.2262-5-lersek@redhat.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
2021-01-20 18:20:14 +00:00
Laszlo Ersek 38ee7bafa7 UefiCpuPkg/CpuS3DataDxe: do not allocate useless register tables
CpuS3DataDxe allocates the "RegisterTable" and "PreSmmInitRegisterTable"
arrays in ACPI_CPU_DATA just so every processor in the system can have its
own empty register table, matched by APIC ID. This has never been useful
in practice.

Given commit e992cc3f48 ("UefiCpuPkg PiSmmCpuDxeSmm: Reduce SMRAM
consumption in CpuS3.c", 2021-01-11), simply leave both
"AcpiCpuData->RegisterTable" and "AcpiCpuData->PreSmmInitRegisterTable"
initialized to the zero address. This simplifies the driver, and saves
both normal RAM (boot services data type memory) and -- in PiSmmCpuDxeSmm
-- SMRAM.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3159
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Message-Id: <20210119155440.2262-4-lersek@redhat.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
2021-01-20 18:20:14 +00:00
Laszlo Ersek 1487c13ce0 UefiCpuPkg/AcpiCpuData: update comments on register table fields
After commit e992cc3f48 ("UefiCpuPkg PiSmmCpuDxeSmm: Reduce SMRAM
consumption in CpuS3.c", 2021-01-11), it is valid for a CPU S3 Data DXE
Driver to set "ACPI_CPU_DATA.PreSmmInitRegisterTable" and/or
"ACPI_CPU_DATA.RegisterTable" to 0, in case none of the CPUs needs a
register table of the corresponding kind, during S3 resume.

Document this fact in the "UefiCpuPkg/Include/AcpiCpuData.h" header file.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3159
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210119155440.2262-3-lersek@redhat.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
2021-01-20 18:20:14 +00:00
Ray Ni cefad282fb UefiCpuPkg/CpuFeature: Don't assume CpuS3DataDxe alloc RegisterTable
There are lots of fields in ACPI_CPU_DATA structure while only
followings are accessed by CpuFeature infra:
* NumberOfCpus
* PreSmmInitRegisterTable // pointer
* RegisterTable  // pointer
* CpuStatus
* ApLocation  // pointer

So it's possible that an implementation of CpuS3DataDxe doesn't
allocate memory for PreSmmInitRegisterTable/RegisterTable/ApLocation.

This patch handles the case when CpuS3DataDxe doesn't allocate
memory for PreSmmInitRegisterTable/RegisterTable.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3159
Signed-off-by: Ray Ni <ray.ni@intel.com>
[lersek@redhat.com: update CC list, add BZ reference, add my S-o-b]
[lersek@redhat.com: deal with RegisterTable and PreSmmInitRegisterTable
 being zero independently of each other; replacing the ASSERT()]
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210119155440.2262-2-lersek@redhat.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
2021-01-20 18:20:14 +00:00
Jiahui Cen via groups.io e843a21e23 ArmVirtPkg/ArmVirtQemu: Add support for HotPlug
It is necessary to add padding for hotplugable PCI Devices like
pcie-root-port.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Message-Id: <20210119011302.10908-12-cenjiahui@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io c4cbd86493 ArmVirtPkg/FdtPciHostBridgeLib: Add extra pci root buses support
In order to take advantages of extra pci root buses in ArmVirtPkg, it is
necessary to scan extra root buses when getting root briges. And now
PciHostBridgeUtilityLib already provides a set of utility functions that
support for extra pci root buses, like PciHostBridgeUtilityGetRootBridges()
/ PciHostBridgeUtilityFreeRootBridges(). So let's rebase
ArmVirtPkg/FdtPciHostBridgeLib to PciHostBridgeUtilityGetRootBridges() /
PciHostBridgeUtilityFreeRootBridges() to extend ArmVirtPkg with extra
pci root buses support.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Message-Id: <20210119011302.10908-11-cenjiahui@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io f4a257a355 OvmfPkg/PciHostBridgeUtilityLib: Extend GetRootBridges() with BusMin/BusMax
Extend parameter list of PciHostBridgeUtilityGetRootBridges() with BusMin/
BusMax, so that the utility function could be compatible with ArmVirtPkg
who uses mutable bus range [BusMin, BusMax] insteand of [0, PCI_MAX_BUS].

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Message-Id: <20210119011302.10908-10-cenjiahui@huawei.com>
[lersek@redhat.com: fix logging of UINTN values BusMin, BusMax]
[lersek@redhat.com: keep zeroing of (*Count) centralized]
[lersek@redhat.com: fix typos in ExtraRootBridges comment]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io 14d4b6be56 OvmfPkg/PciHostBridgeUtilityLib: Extend parameter list of GetRootBridges
Extend parameter list of PciHostBridgeUtilityGetRootBridges() with
DmaAbove4G, NoExtendedConfigSpace to support for ArmVirtPkg.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Message-Id: <20210119011302.10908-9-cenjiahui@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io 4edba29651 OvmfPkg/PciHostBridgeLib: Extract GetRootBridges() / FreeRootBridges()
Extract PciHostBridgeGetRootBridges() / PciHostBridgeFreeRootBridges() to
PciHostBridgeUtilityLib as common utility functions to share support for
scanning extra root bridges.

No change of functionality.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Message-Id: <20210119011302.10908-8-cenjiahui@huawei.com>
[lersek@redhat.com: keep zeroing of (*Count) centralized]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io aa445ce02b ArmVirtPkg/FdtPciHostBridgeLib: Refactor init/uninit of root bridge
Rebase ArmVirtPkg/FdtPciHostBridgeLib to the new
PciHostBridgeUtilityInitRootBridge()/PciHostBridgeUtilityUninitRootBridge()
utility functions.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Message-Id: <20210119011302.10908-7-cenjiahui@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io e1b259da42 OvmfPkg/PciHostBridgeUtilityLib: Extend parameters of InitRootBridge()
Extend parameter list of PciHostBridgeUtilityInitRootBridge() with
DmaAbove4G and NoExtendedConfigSpace to prepare for sharing with
ArmVirtPkg.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Julien Grall <julien@xen.org>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Message-Id: <20210119011302.10908-6-cenjiahui@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io 7ac1f28d4d OvmfPkg/PciHostBridgeLib: Extract InitRootBridge() / UninitRootBridge()
Extract InitRootBridge() / UninitRootBridge() to PciHostBridgeUtilityLib
as common utility functions. No change of functionality.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Julien Grall <julien@xen.org>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Message-Id: <20210119011302.10908-5-cenjiahui@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io 517055d298 OvmfPkg/PciHostBridgeLib: List missing PcdLib dependency
OvmfPkg/PciHostBridgeLib instance fails to list its PcdLib dependency,
both between the #include directives, and in the INF file. So let's list
the dependency.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Julien Grall <julien@xen.org>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Message-Id: <20210119011302.10908-4-cenjiahui@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io 166a32d09a ArmVirtPkg: Refactor with PciHostBridgeUtilityLib
Eliminate currently duplicated code in ArmVirtPkg with the common utility
class PciHostBridgeUtilityLib.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210119011302.10908-3-cenjiahui@huawei.com>
2021-01-20 16:14:20 +00:00
Jiahui Cen via groups.io 7a6172f88b OvmfPkg: Introduce PciHostBridgeUtilityLib class
Introduce a new PciHostBridgeUtilityLib class to share duplicate code
between OvmfPkg and ArmVirtPkg.

Extract function PciHostBridgeUtilityResourceConflict from
PciHostBridgeResourceConflict in OvmfPkg/PciHostBridgeLib.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Peter Grehan <grehan@freebsd.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Julien Grall <julien@xen.org>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Signed-off-by: Yubo Miao <miaoyubo@huawei.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210119011302.10908-2-cenjiahui@huawei.com>
2021-01-20 16:14:20 +00:00
GregX Yeh ca272b9513 DxeHttpIoLib: Http boot failure with no initializes timeout value.
https://bugzilla.tianocore.org/show_bug.cgi?id=3170
Using PcdHttpIoTimeout to set default timeout value to HttpIoLib.

Cc: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Signed-off-by: GregX Yeh <gregx.yeh@intel.com>
Reviewed-by: Siyuan Fu <siyuan.fu@intel.com>
Reviewed-by: Maciej Rabeda <maciej.rabeda@linux.intel.com>
2021-01-20 12:33:38 +00:00
Laszlo Ersek 6e55868631 ArmVirtPkg: disable list length checks in NOOPT and DEBUG builds
In NOOPT and DEBUG builds, if "PcdMaximumLinkedListLength" is nonzero,
then several LIST_ENTRY *node* APIs in BaseLib compare the *full* list
length against the PCD.

This turns the time complexity of node-level APIs from constant to linear,
and that of full-list manipulations from linear to quadratic.

(See some example OVMF numbers in the previous patch.)

Checking list lengths against an arbitrary maximum -- default value, and
current ArmVirtPkg setting: 1,000,000 -- seems useless even in NOOPT and
DEBUG builds, while the cost is significant; so set the PCD to 0.

Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Julien Grall <julien@xen.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3152
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Message-Id: <20210113085453.10168-11-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek d3f3c94201 OvmfPkg: disable list length checks in NOOPT and DEBUG builds
In NOOPT and DEBUG builds, if "PcdMaximumLinkedListLength" is nonzero,
then several LIST_ENTRY *node* APIs in BaseLib compare the *full* list
length against the PCD.

This turns the time complexity of node-level APIs from constant to linear,
and that of full-list manipulations from linear to quadratic.

As an example, consider the EFI_SHELL_FILE_INFO list, which is a data
structure that's widely used in the UEFI shell. I randomly extracted 5000
files from "/usr/include" on my laptop, spanning 1095 subdirectories out
of 1538, and then ran "DIR -R" in the UEFI shell on this tree. These are
the wall-clock times:

           PcdMaximumLinkedListLength  PcdMaximumLinkedListLength
           =1,000,000                  =0
           --------------------------  ---------------------------
FAT        4 min 31 s                        18 s
virtio-fs  5 min 13 s                  1 min 33 s

Checking list lengths against an arbitrary maximum (default: 1,000,000)
seems useless even in NOOPT and DEBUG builds, while the cost is
significant; so set the PCD to 0.

Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Julien Grall <julien@xen.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Peter Grehan <grehan@freebsd.org>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3152
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Message-Id: <20210113085453.10168-10-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek c8ce60eb29 ShellPkg/ShellProtocol: sort files by FullName in RemoveDupInFileList()
The current implementation of EfiShellRemoveDupInFileList():
- has quadratic time complexity, as a disadvantage, and
- needs no dynamic memory, as an advantage.

Because the UEFI Shell Spec requires
EFI_SHELL_PROTOCOL.RemoveDupInFileList() to succeed at all times, keep the
current method as a fallback (it cannot fail due to needing no dynamic
memory).

However, as a higher priority option, call the new ShellSortFileList()
function at first, separating out and releasing duplicates.
(ShellSortFileList() can fail due to EFI_OUT_OF_RESOURCES.)

Beyond improving the runtime of EfiShellRemoveDupInFileList(), this change
has the extremely desirable effect that the ShellOpenFileMetaArg()
function in the ShellPkg/Library/UefiShellLib instance will produce file
lists that are sorted by FullName.

Consequently, when used with wildcards, the ATTRIB, CP, FOR, LOAD,
LOADPCIROM, LS, MV, RM, TOUCH, TYPE commands will process files in
FullName order. (LS in recursive mode uses wildcards internally.)

Before:

> FS2:\> dir -r -sfo apps
> [...]
> FileInfo,"FS2:\apps\"
> FileInfo,"FS2:\apps\X64"
> FileInfo,"FS2:\apps\AARCH64"
> FileInfo,"FS2:\"
> FileInfo,"FS2:\apps\IA32"
> FileInfo,"FS2:\apps\X64\DumpDynPcd.efi"
> FileInfo,"FS2:\apps\X64\SmiHandlerProfileInfo.efi"
> FileInfo,"FS2:\apps\X64\"
> FileInfo,"FS2:\apps\X64\VariableInfo.efi"
> FileInfo,"FS2:\apps\X64\MemoryProfileInfo.efi"
> FileInfo,"FS2:\apps\X64\AcpiViewApp.efi"
> FileInfo,"FS2:\apps\X64\Cpuid.efi"
> FileInfo,"FS2:\apps\"
> FileInfo,"FS2:\apps\AARCH64\DumpDynPcd.efi"
> FileInfo,"FS2:\apps\AARCH64\"
> FileInfo,"FS2:\apps\AARCH64\VariableInfo.efi"
> FileInfo,"FS2:\apps\AARCH64\MemoryProfileInfo.efi"
> FileInfo,"FS2:\apps\AARCH64\AcpiViewApp.efi"
> FileInfo,"FS2:\apps\"
> FileInfo,"FS2:\apps\IA32\DumpDynPcd.efi"
> FileInfo,"FS2:\apps\IA32\SmiHandlerProfileInfo.efi"
> FileInfo,"FS2:\apps\IA32\"
> FileInfo,"FS2:\apps\IA32\VariableInfo.efi"
> FileInfo,"FS2:\apps\IA32\MemoryProfileInfo.efi"
> FileInfo,"FS2:\apps\IA32\AcpiViewApp.efi"
> FileInfo,"FS2:\apps\IA32\Cpuid.efi"
> FileInfo,"FS2:\apps\"

After:

> FS2:\> dir -r -sfo apps
> [...]
> FileInfo,"FS2:\"
> FileInfo,"FS2:\apps\"
> FileInfo,"FS2:\apps\AARCH64"
> FileInfo,"FS2:\apps\IA32"
> FileInfo,"FS2:\apps\X64"
> FileInfo,"FS2:\apps\"
> FileInfo,"FS2:\apps\AARCH64\"
> FileInfo,"FS2:\apps\AARCH64\AcpiViewApp.efi"
> FileInfo,"FS2:\apps\AARCH64\DumpDynPcd.efi"
> FileInfo,"FS2:\apps\AARCH64\MemoryProfileInfo.efi"
> FileInfo,"FS2:\apps\AARCH64\VariableInfo.efi"
> FileInfo,"FS2:\apps\"
> FileInfo,"FS2:\apps\IA32\"
> FileInfo,"FS2:\apps\IA32\AcpiViewApp.efi"
> FileInfo,"FS2:\apps\IA32\Cpuid.efi"
> FileInfo,"FS2:\apps\IA32\DumpDynPcd.efi"
> FileInfo,"FS2:\apps\IA32\MemoryProfileInfo.efi"
> FileInfo,"FS2:\apps\IA32\SmiHandlerProfileInfo.efi"
> FileInfo,"FS2:\apps\IA32\VariableInfo.efi"
> FileInfo,"FS2:\apps\"
> FileInfo,"FS2:\apps\X64\"
> FileInfo,"FS2:\apps\X64\AcpiViewApp.efi"
> FileInfo,"FS2:\apps\X64\Cpuid.efi"
> FileInfo,"FS2:\apps\X64\DumpDynPcd.efi"
> FileInfo,"FS2:\apps\X64\MemoryProfileInfo.efi"
> FileInfo,"FS2:\apps\X64\SmiHandlerProfileInfo.efi"
> FileInfo,"FS2:\apps\X64\VariableInfo.efi"

Regarding LS in non-SFO mode, the stability of ShellSortFileList() shows.
The ShellSortFileList() call added to LS in the previous patch re-sorts
the output of ShellOpenFileMetaArg(); and so this patch improves the
ordering between identical FileNames:

Before:

> FS2:\> dir -r apps
> Directory of: FS2:\apps\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53 <DIR>         4,096  AARCH64
> 12/22/2020  17:53 <DIR>         4,096  IA32
> 12/22/2020  17:53 <DIR>         4,096  X64
>           0 File(s)           0 bytes
>           5 Dir(s)
> Directory of: FS2:\apps\X64\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              11,456  VariableInfo.efi
>           6 File(s)     256,192 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\AARCH64\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:52              20,480  VariableInfo.efi
>           4 File(s)     233,472 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\IA32\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              10,880  VariableInfo.efi
>           6 File(s)     224,768 bytes
>           2 Dir(s)
>
> FS2:\> dir apps\*\*.efi
> Directory of: FS2:\apps\*\
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              11,456  VariableInfo.efi
> 12/22/2020  17:52              20,480  VariableInfo.efi
> 12/22/2020  17:52              10,880  VariableInfo.efi
>          16 File(s)     714,432 bytes
>           0 Dir(s)

After:

> FS2:\> dir -r apps
> Directory of: FS2:\apps\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53 <DIR>         4,096  AARCH64
> 12/22/2020  17:53 <DIR>         4,096  IA32
> 12/22/2020  17:53 <DIR>         4,096  X64
>           0 File(s)           0 bytes
>           5 Dir(s)
> Directory of: FS2:\apps\AARCH64\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:52              20,480  VariableInfo.efi
>           4 File(s)     233,472 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\IA32\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              10,880  VariableInfo.efi
>           6 File(s)     224,768 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\X64\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              11,456  VariableInfo.efi
>           6 File(s)     256,192 bytes
>           2 Dir(s)
>
> FS2:\> dir apps\*\*.efi
> Directory of: FS2:\apps\*\
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              20,480  VariableInfo.efi
> 12/22/2020  17:52              10,880  VariableInfo.efi
> 12/22/2020  17:52              11,456  VariableInfo.efi
>          16 File(s)     714,432 bytes
>           0 Dir(s)

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3151
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>
Message-Id: <20210113085453.10168-9-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek 70254306a8 ShellPkg/Ls: sort output by FileName in non-SFO mode
Sorting the LS output in non-SFO mode by FileName is best demonstrated
with two examples.

(1a) Before:

> FS2:\> dir -r apps
> Directory of: FS2:\apps\
> 01/01/1970  01:00 <DIR> r           0  .
> 12/22/2020  17:53 <DIR>         4,096  X64
> 12/22/2020  17:53 <DIR>         4,096  AARCH64
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53 <DIR>         4,096  IA32
>           0 File(s)           0 bytes
>           5 Dir(s)
> Directory of: FS2:\apps\X64\
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 01/01/1970  01:00 <DIR> r           0  .
> 12/22/2020  17:52              11,456  VariableInfo.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 01/01/1970  01:00 <DIR> r           0  ..
>           6 File(s)     256,192 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\AARCH64\
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 01/01/1970  01:00 <DIR> r           0  .
> 12/22/2020  17:52              20,480  VariableInfo.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 01/01/1970  01:00 <DIR> r           0  ..
>           4 File(s)     233,472 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\IA32\
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 01/01/1970  01:00 <DIR> r           0  .
> 12/22/2020  17:52              10,880  VariableInfo.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
> 01/01/1970  01:00 <DIR> r           0  ..
>           6 File(s)     224,768 bytes
>           2 Dir(s)

(1b) After:

> FS2:\> dir -r apps
> Directory of: FS2:\apps\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53 <DIR>         4,096  AARCH64
> 12/22/2020  17:53 <DIR>         4,096  IA32
> 12/22/2020  17:53 <DIR>         4,096  X64
>           0 File(s)           0 bytes
>           5 Dir(s)
> Directory of: FS2:\apps\X64\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              11,456  VariableInfo.efi
>           6 File(s)     256,192 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\AARCH64\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:52              20,480  VariableInfo.efi
>           4 File(s)     233,472 bytes
>           2 Dir(s)
> Directory of: FS2:\apps\IA32\
> 01/01/1970  01:00 <DIR> r           0  .
> 01/01/1970  01:00 <DIR> r           0  ..
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              10,880  VariableInfo.efi
>           6 File(s)     224,768 bytes
>           2 Dir(s)

(2a) Before:

> FS2:\> dir apps\*\*.efi
> Directory of: FS2:\apps\*\
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              11,456  VariableInfo.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 12/22/2020  17:52              20,480  VariableInfo.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              10,880  VariableInfo.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
>          16 File(s)     714,432 bytes
>           0 Dir(s)

(2b) After:

> FS2:\> dir apps\*\*.efi
> Directory of: FS2:\apps\*\
> 12/22/2020  17:53             126,656  AcpiViewApp.efi
> 12/22/2020  17:53             139,264  AcpiViewApp.efi
> 12/22/2020  17:53             105,536  AcpiViewApp.efi
> 12/22/2020  17:53              38,784  Cpuid.efi
> 12/22/2020  17:53              36,096  Cpuid.efi
> 12/22/2020  17:52              18,752  DumpDynPcd.efi
> 12/22/2020  17:52              32,768  DumpDynPcd.efi
> 12/22/2020  17:52              17,344  DumpDynPcd.efi
> 12/22/2020  17:52              26,304  MemoryProfileInfo.efi
> 12/22/2020  17:52              40,960  MemoryProfileInfo.efi
> 12/22/2020  17:52              24,192  MemoryProfileInfo.efi
> 12/22/2020  17:52              34,240  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              30,720  SmiHandlerProfileInfo.efi
> 12/22/2020  17:52              11,456  VariableInfo.efi
> 12/22/2020  17:52              20,480  VariableInfo.efi
> 12/22/2020  17:52              10,880  VariableInfo.efi
>          16 File(s)     714,432 bytes
>           0 Dir(s)

(In example (2), note that the sorting is stable; that is, whatever order
is established between identical FileNames by ShellOpenFileMetaArg(), it
is preserved by ShellSortFileList().)

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3151
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>
Message-Id: <20210113085453.10168-8-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek 101c55ac0d ShellPkg/ShellCommandLib: add ShellSortFileList()
Introduce the ShellSortFileList() function, for sorting an
EFI_SHELL_FILE_INFO list, by FileName or by FullName.

Duplicates can be kept in the same list, or separated out to a new list.
In either case, the relative order between duplicates does not change (the
sorting is stable).

For the sorting, use OrderedCollectionLib rather than SortLib:

- The PerformQuickSort() function from the latter has quadratic worst-case
  time complexity, plus it is implemented recursively (see
  "MdeModulePkg/Library/UefiSortLib/UefiSortLib.c"). It can also not
  return an error on memory allocation failure.

- In comparison, the Red-Black Tree instance of OrderedCollectionLib sorts
  in O(n*log(n)) worst-case time, contains no recursion with the default
  PcdValidateOrderedCollection=FALSE setting, and the OrderedCollectionLib
  class APIs return errors appropriately.

The OrderedCollectionLib APIs do not permit duplicates natively, but by
using lists as collection entries, stable sorting of duplicates can be
achieved.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3151
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>
Message-Id: <20210113085453.10168-7-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek ef03e72651 UefiPayloadPkg: add OrderedCollectionLib class resolution
A subsequent patch in the series will make the

  ShellPkg/Library/UefiShellCommandLib/UefiShellCommandLib.inf

instance dependent on the OrderedCollectionLib class. Because the shell
binary in this platform consumes the above UefiShellCommandLib instance,
resolve OrderedCollectionLib.

Cc: Benjamin You <benjamin.you@intel.com>
Cc: Guo Dong <guo.dong@intel.com>
Cc: Maurice Ma <maurice.ma@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3151
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210113085453.10168-6-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Guo Dong <guo.dong@intel.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek 130e929f98 EmulatorPkg: add OrderedCollectionLib class resolution
A subsequent patch in the series will make the

  ShellPkg/Library/UefiShellCommandLib/UefiShellCommandLib.inf

instance dependent on the OrderedCollectionLib class. Because the shell
binary in this platform consumes the above UefiShellCommandLib instance,
resolve OrderedCollectionLib.

Cc: Andrew Fish <afish@apple.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3151
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210113085453.10168-5-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Liming Gao <gaoliming@byosoft.com.cn>
2021-01-19 18:23:28 +00:00
Laszlo Ersek a0b352e18b ArmVirtPkg: raise PcdShellFileOperationSize to 128KB
Some UEFI shell commands read and write files in chunks. The chunk size is
given by "PcdShellFileOperationSize", whose default in
"ShellPkg/ShellPkg.dec" is 4KB (0x1000).

The virtio-fs daemon of QEMU advertizes a 128KB maximum buffer size by
default, for the FUSE_WRITE operation.

By raising PcdShellFileOperationSize 32-fold, the number of FUSE write
requests shrinks proportionately, when writing large files. And when a
Virtio Filesystem is not used, a 128KB chunk size is still not
particularly wasteful.

Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3125
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Message-Id: <20210113085453.10168-4-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek 2912341cd9 OvmfPkg: raise PcdShellFileOperationSize to 128KB
Some UEFI shell commands read and write files in chunks. The chunk size is
given by "PcdShellFileOperationSize", whose default in
"ShellPkg/ShellPkg.dec" is 4KB (0x1000).

The virtio-fs daemon of QEMU advertizes a 128KB maximum buffer size by
default, for the FUSE_WRITE operation.

By raising PcdShellFileOperationSize 32-fold, the number of FUSE write
requests shrinks proportionately, when writing large files. And when a
Virtio Filesystem is not used, a 128KB chunk size is still not
particularly wasteful.

Some ad-hoc measurements on my laptop, using OVMF:

- The time it takes to copy a ~270MB file from a Virtio Filesystem to the
  same Virtio Filesystem improves from ~9 seconds to ~1 second.

- The time it takes to compare two identical ~270MB files on the same
  Virtio Filesystem improves from ~11 seconds to ~3 seconds.

Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3125
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Message-Id: <20210113085453.10168-3-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Laszlo Ersek f72fd9616e ShellPkg/Comp: add file buffering
The COMP shell command compares two files byte for byte. In order to
retrieve the bytes to compare, it currently invokes
gEfiShellProtocol->ReadFile() on both files, using a single-byte buffer
every time. This is very inefficient; the underlying
EFI_FILE_PROTOCOL.Read() function may be costly.

Read both file operands in chunks of "PcdShellFileOperationSize" bytes.
Draw bytes for comparison from the internal read-ahead buffers.

Some ad-hoc measurements on my laptop, using OVMF, and the 4KB default of
"PcdShellFileOperationSize":

- When comparing two identical 1MB files that are served by EnhancedFatDxe
  on top of VirtioScsiDxe, this patch brings no noticeable improvement;
  the comparison completes in <1s both before and after.

- When comparing two identical 1MB files served by VirtioFsDxe, the
  comparison time improves from 2 minutes 25 seconds to <1s.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3123
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>
Message-Id: <20210113085453.10168-2-lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-19 18:23:28 +00:00
Igor Druzhinin e68c2a22ca OvmfPkg/XenPlatformPei: Use CPUID to get physical address width on Xen
We faced a problem with passing through a PCI device with 64GB BAR to UEFI
guest. The BAR is expectedly programmed into 64-bit PCI aperture at 64G
address which pushes physical address space to 37 bits. That is above
36-bit width that OVMF exposes currently to a guest without tweaking
PcdPciMmio64Size knob.

The reverse calculation using this knob was inhereted from QEMU-KVM
platform code where it serves the purpose of finding max accessible
physical address without necessary trusting emulated CPUID physbits value
(that could be different from host physbits). On Xen we expect to use
CPUID policy to level the data correctly to prevent situations with guest
physbits > host physbits e.g. across migrations.

The next aspect raising concern - resource consumption for DXE IPL page
tables and time required to map the whole address space in case of using
CPUID bits directly. That could be mitigated by enabling support for 1G
pages in DXE IPL configuration. 1G pages are available on most CPUs
produced in the last 10 years and those without don't have many phys bits.

Remove all the redundant code now (including PcdPciMmio64.. handling
that's not used on Xen anyway) and grab physbits directly from CPUID that
should be what baremetal UEFI systems do.

Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
Message-Id: <1610509335-23314-1-git-send-email-igor.druzhinin@citrix.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Julien Grall <julien@xen.org>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
[lersek@redhat.com: fix up authorship from groups.io-mangled From line]
[lersek@redhat.com: wrap commit message at 74 characters]
2021-01-19 17:00:08 +00:00
Lou, Yun 83facfd184 UefiCpuPkg/CpuCacheInfoLib: Add new CpuCacheInfoLib.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3105

This new library uses a platform agnostic algorithm to get CPU
cache information. It provides user with an API(GetCpuCacheInfo)
to get detailed CPU cache information by each package, each core
type included in this package, and each cache level & type.
This library can be used by code that produces SMBIOS_TABLE_TYPE7
SMBIOS table.

Signed-off-by: Jason Lou <yun.lou@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
2021-01-19 14:03:04 +00:00
Jason Lou 79f3404ad8 MdePkg/Cpuid.h: Add CPUID_HYBRID_INFORMATION Leaf(1Ah).
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3105

The UefiCpuPkg/CpuCacheInfoLib will reference new definition
about CPUID_HYBRID_INFORMATION Leaf(1Ah).

Signed-off-by: Jason Lou <yun.lou@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2021-01-19 14:03:04 +00:00
Ard Biesheuvel 4f214830ce ArmPlatformPkg/NorFlashDxe: use correct PCD accessors
Commit 8015f3f6d4 ("ArmPlatformPkg: Enable support for flash in
64-bit address space") updated the NorFlash DXE and StMM drivers to
take alternate PCDs into account when discovering the base of the
NOR flash regions.

This introduced a disparity between the declarations of the PCD references
in the .INF files, which permits the use of dynamic PCDs, and the code
itself, which now uses FixedPcdGet() accessors. On platforms that actually
use dynamic PCDs, this results in a build error.

So let's clean this up:
- for the DXE version, use the generic PcdGet() accessors, so dynamic PCDs
  are permitted
- for the standalone MM version, redeclare the PCDs as [FixedPcd] in the
  .INF description, and switch to the FixedPcdGet() accessors.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Vijayenthiran Subramaniam <vijayenthiran.subramaniam@arm.com>
2021-01-18 11:19:45 +00:00
Ard Biesheuvel 7dc277f9e4 Maintainers: update Ard's email address
I will no longer work for ARM as of next month, and will therefore
lose access to my @arm.com email account. I intend to remain active
in the Tianocore project nonetheless, so let's update my email accounts
to one that is not tied to my current or future employer.

Cc: <ardb+tianocore@kernel.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
Reviewed-by: Andrew Fish <afish@apple.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Acked-by: Sami Mujawar <sami.mujawar@arm.com>
2021-01-18 09:40:29 +00:00
Zarcd Zhong a7ef2a03b9 MdeModulePkg/PciBusDxe: Handle BAR sizing fail in high 32bit of MEM64.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3149

Address MEM64 BAR in type unknown if sizing fail in high 32bit.

Cc: Ray Ni <ray.ni@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Signed-off-by: Zarcd Zhong <zarcd.zhong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2021-01-18 01:38:29 +00:00
Abner Chang c88736f860 EmulatorPkg/library: RedfishPlatformCredentialLib
Platform specific implementation of acquiring credential
to access to Redfish service. This is the platform library
which incorporates with Redfish Credential DXE driver under
Redfish package.

Signed-off-by: Abner Chang <abner.chang@hpe.com>

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Nickle Wang <nickle.wang@hpe.com>
Cc: Peter O'Hanley <peter.ohanley@hpe.com>
Acked-by: Ray Ni <ray.ni@intel.com>
2021-01-16 03:35:31 +00:00
wenyi,xie via groups.io 014b9850f2 MdeModulePkg/FileExplorerLib: Add return value check
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3113
According to FAT specification, the length of file path
should not larger than 260. When the length exceed 260,
function FatLocateOFile will return EFI_INVALID_PARAMETER
and the parameter FileHandle will be NULL. Then on the
top-level function?an exception happens when the NULL
pointer is passed and be used.
So adding return value check after calling
LibGetFileHandleFromMenu, if return value is not success,
stop calling LibFindFiles.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Dandan Bi <dandan.bi@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Signed-off-by: Wenyi Xie <xiewenyi2@huawei.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Dandan Bi <dandan.bi@intel.com>
2021-01-15 01:08:45 +00:00
Abner Chang 40c4cd5421 NetworkPkg/DxeHttpLib: Migrate HTTP header manipulation APIs
Move HTTP header manipulation functions to DxeHttpLib from
HttpBootSupport.c. These general functions are used by both
Http BOOT and RedfishLib (patches will be sent later).

Signed-off-by: Abner Chang <abner.chang@hpe.com>

Cc: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Cc: Fan Wang <fan.wang@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Nickle Wang <nickle.wang@hpe.com>
Cc: Peter O'Hanley <peter.ohanley@hpe.com>
Reviewed-by: Maciej Rabeda <maciej.rabeda@linux.intel.com>
2021-01-14 14:54:12 +00:00
Bob Feng 536a3e6726 BaseTools: Fix the build report crash issue
In the following corner case, the build report
will crash. This patch is to fix this problem.

Case:
Multiple SKU are used and 2 more DynamicHii structure Pcds
are set in dsc file under different SKU. And 1 more of those
Pcds are not used in any INF file.

Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Yuwei Chen<yuwei.chen@intel.com>
2021-01-14 04:12:09 +00:00
Luo, Heng da45a36087 MdeModulePkg: Update UNI file for PCIe Resizable BAR Capability
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3138

PcdPcieResizableBarSupport was added in MdeModulePkg.dec, update
UNI file as the documentation of the PCD.

Cc: Ray Ni <ray.ni@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Signed-off-by: Heng Luo <heng.luo@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Hao A Wu <hao.a.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2021-01-14 01:34:16 +00:00
Luo, Heng ef23012e54 MdeModulePkg: Change default value of PcdPcieResizableBarSupport to FALSE
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3138

Change default value of PcdPcieResizableBarSupport to FALSE, avoid it
impacts existing platforms.

Cc: Ray Ni <ray.ni@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Signed-off-by: Heng Luo <heng.luo@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2021-01-14 01:34:16 +00:00
Guo Dong ebfe2d3eb5 UefiCpuPkg/CpuDxe: Fix boot error
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3084

When DXE drivers are dispatched above 4GB memory in 64bit
mode, the address setCodeSelectorLongJump in stack will
be override by parameter. Jump to Qword is not supported
by some processors. So use "o64 retf" instead.

Signed-off-by: Guo Dong <guo.dong@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Tested-by: James Bottomley <jejb@linux.ibm.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2021-01-12 06:35:30 +00:00
Guo Dong edd74ad3ad UefiCpuPkg/MpInitLib: Fix a hang in above 4GB case
This patch fixed the hang in UEFICpuPkg when it is dispatched above 4GB.
In UEFI BIOS case CpuInfoInHob is provided to DXE under 4GB from PEI.
When using UEFI payload and bootloaders, CpuInfoInHob will be allocated
above 4GB since it is not provided from bootloader. so we need update
the code to make sure this hob could be accessed correctly in this case.

Signed-off-by: Guo Dong <guo.dong@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2021-01-12 06:35:30 +00:00
Zeng, Star e992cc3f48 UefiCpuPkg PiSmmCpuDxeSmm: Reduce SMRAM consumption in CpuS3.c
This patch makes two refinements to reduce SMRAM consumption in CpuS3.c.
1. Only do CopyRegisterTable() when register table is not empty,
  IsRegisterTableEmpty() is created to check whether the register table
  is empty or not.

  Take empty PreSmmInitRegisterTable as example, about 24K SMRAM consumption
  could be reduced when mAcpiCpuData.NumberOfCpus=1024.
  sizeof (CPU_REGISTER_TABLE) = 24
  mAcpiCpuData.NumberOfCpus = 1024 = 1K
  mAcpiCpuData.NumberOfCpus * sizeof (CPU_REGISTER_TABLE) = 24K

2. Only copy table entries buffer instead of whole buffer.
  AllocatedSize in SourceRegisterTableList is the whole buffer size.
  Actually, only the table entries buffer needs to be copied, and the size
  is TableLength * sizeof (CPU_REGISTER_TABLE_ENTRY).

  Take AllocatedSize=0x1000=4096, TableLength=100 and NumberOfCpus=1024 as example,
  about 1696K SMRAM consumption could be reduced.
  sizeof (CPU_REGISTER_TABLE_ENTRY) = 24
  TableLength = 100
  TableLength * sizeof (CPU_REGISTER_TABLE_ENTRY) = 2400
  AllocatedSize = 0x1000 = 4096
  AllocatedSize - TableLength * sizeof (CPU_REGISTER_TABLE_ENTRY) = 4096 - 2400 = 1696
  NumberOfCpus = 1024 = 1K
  NumberOfCpus * (AllocatedSize - TableLength * sizeof (CPU_REGISTER_TABLE_ENTRY)) = 1696K

This patch also corrects the CopyRegisterTable() function description.

Signed-off-by: Star Zeng <star.zeng@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210111015419.28368-1-star.zeng@intel.com>
2021-01-11 09:13:48 +00:00
Rebecca Cran a8a0831589 ShellPkg: Remove bogus "Bits" text from some smbiosview strings.
Neither of the "CPU Socket Unpopulated" or "Do not reboot" messages
should contain the word "Bits".

Remove them and update the identifier for the "Do not reboot" message
to remove the word "BITS".

Signed-off-by: Rebecca Cran <rebecca@nuviainc.com>
Reviewed-by: Zhichao Gao <zhichao.gao@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
2021-01-11 02:09:10 +00:00
Nhi Pham 4ea3a6ded8 EmbeddedPkg/RealTimeClockRuntimeDxe: depend on gEfiVariableArchProtocolGuid
The EfiGetVariable() is used in the entry of this module. So, the
variable services are required to be ready before they are used. This
patch adds the arch protocol gEfiVariableArchProtocolGuid to dependency
expression to guarantee that this module will be started once the
variable protocol is available.

Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Nhi Pham <nhi@os.amperecomputing.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
2021-01-10 01:18:44 +00:00
Abner Chang acabf1b033 .pytool: Add required submodule for JsonLib
Open source project jansson is leveraged by edk2 JsonLib.
Add jansson to the required submodule in CiSettings.py.

Signed-off-by: Abner Chang <abner.chang@hpe.com>

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Nickle Wang <nickle.wang@hpe.com>
Cc: Peter O'Hanley <peter.ohanley@hpe.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2021-01-09 03:08:51 +00:00
Abner Chang fa4c4fb8cd RedfishPkg: Add EDK2 port of jansson library to build
Add EDK2 port jansson library (JsonLib) to RedfishPkg

Signed-off-by: Abner Chang <abner.chang@hpe.com>

Cc: Nickle Wang <nickle.wang@hpe.com>
Cc: Peter O'Hanley <peter.ohanley@hpe.com>
Reviewed-by: Nickle Wang <nickle.wang@hpe.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2021-01-09 03:08:51 +00:00
Abner Chang ea830b96fd RedfishPkg/library: EDK2 port of jansson library
edk2 JsonLib which is the edk2 port of open source
jansson library.
(https://github.com/akheron/jansson)
jansson library is the open source project to manipulate
JSON data structure.

Signed-off-by: Abner Chang <abner.chang@hpe.com>

Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Nickle Wang <nickle.wang@hpe.com>
Cc: Peter O'Hanley <peter.ohanley@hpe.com>
Reviewed-by: Nickle Wang <nickle.wang@hpe.com>
Acked-by: Leif Lindholm <leif@nuviainc.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
2021-01-09 03:08:51 +00:00