The QemuDetected() function wraps QemuFwCfgIsAvailable(); it always fails
on Xen. Because of that, we can eliminate the QemuDetected() call itself
from the Xen ACPI platform driver, and then the rest of "Qemu.c" becomes
useless -- the workhorse function of that source file is
QemuInstallAcpiTable(), which we no longer call.
Remove any dependencies that are no longer needed by the
XenAcpiPlatformDxe driver.
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Julien Grall <julien@xen.org>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-15-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The root of the QEMU ACPI linker/loader client in XenAcpiPlatformDxe is
the InstallQemuFwCfgTables() function. This function always fails on Xen,
due to its top-most QemuFwCfgFindFile() call.
Remove the InstallQemuFwCfgTables() function call from XenAcpiPlatformDxe,
along with all dependencies that now become unused.
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Julien Grall <julien@xen.org>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-14-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The "OvmfPkg/AcpiPlatformDxe/AcpiPlatformDxe.inf" module is no longer
referenced in any platform DSC file; remove it.
That orphans the "AcpiPlatform.c", "Qemu.c" and "Xen.c" files in the
"OvmfPkg/AcpiPlatformDxe/" directory; remove them.
That in turn removes the only definitions of the InstallAcpiTable(),
QemuDetected(), QemuInstallAcpiTable(), InstallXenTables() functions in
the "OvmfPkg/AcpiPlatformDxe/" directory, so remove their declarations
from "AcpiPlatform.h".
Remove "OvmfPkg/AcpiPlatformDxe/Xen.c" from the "OvmfPkg: Xen-related
modules" section of "Maintainers.txt", as well.
Cc: Andrew Fish <afish@apple.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-13-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
Create an almost verbatim copy of the
"OvmfPkg/AcpiPlatformDxe/AcpiPlatformDxe.inf" driver for the OvmfXen
platform. We're going to trim the driver in subsequent patches.
Ultimately, the XenAcpiPlatformDxe driver will duplicate a negligible
amount of code that is currently present in the QemuFwCfgAcpiPlatformDxe
driver.
List the new driver in "Maintainers.txt", in the "OvmfPkg: Xen-related
modules" section.
Switch the OvmfXen platform to the new driver at once.
This patch should be reviewed with "git show --find-copies-harder".
Cc: Andrew Fish <afish@apple.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Julien Grall <julien@xen.org>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-12-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
- #include only such public headers in "AcpiPlatform.h" that are required
by the function declarations and type definitions introduced in
"AcpiPlatform.h". Don't use "AcpiPlatform.h" as a convenience #include
file.
- In every file, list every necessary public #include individually, with
an example identifier that's actually consumed.
- Remove unnecessary lib classes, add unlisted lib classes.
- Remove unnecessary #include directives, add unlisted #include
directives.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-11-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Turn the "QemuLoader.h" header into a public (IndustryStandard) one. The
QEMU ACPI linker-loader interface is stable between QEMU and multiple
guest firmwares.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-10-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
"QemuLoader.h" needs the QEMU_FW_CFG_FNAME_SIZE macro. This macro used to
live in the QemuFwCfgLib class header, but we moved it to the more
foundational IndustryStandard include file called "QemuFwCfg.h" in commit
5583a8a4ff ("OvmfPkg/QemuFwCfgLib: move types/macros from lib class to
IndustryStandard", 2017-02-22).
Replace the lib class dependency with the more basic IndustryStandard
dependency in "QemuLoader.h".
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-9-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Place all public #includes first, all module-private #includes second.
Separate them with a single empty line. Keep each section sorted in
itself.
Sort all sections in both INF files.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-8-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
- Remove the leading underscores from the #include guard macro names;
clean up the names in general.
- Remove the useless "Include/" directory prefix from the public header
pathnames.
- Fix incorrect file-top comment.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-7-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Due to switching to the QemuFwCfgAcpiPlatformDxe driver earlier in this
series, require QEMU version 1.7.1 in the "OvmfPkg/README" file, and
require 1.7 or later machine types too.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-6-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
For consistency with the historical OvmfPkg* platforms, switch the
remotely attested, QEMU/KVM-only, AmdSev platform from the AcpiPlatformDxe
driver to the QemuFwCfgAcpiPlatformDxe driver.
No module remains dependent on XenPlatformLib, so remove the
XenPlatformLib class resolution too, from the DSC file.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-5-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Switch the historical OvmfPkg* platforms from the AcpiPlatformDxe driver
to the QemuFwCfgAcpiPlatformDxe driver. (The latter is used by the
ArmVirtQemu* platforms as well.)
The change effectively replaces the following call tree:
InstallAcpiTables [AcpiPlatform.c]
XenDetected [XenPlatformLib] *
InstallXenTables [Xen.c] *
GetXenAcpiRsdp [Xen.c] *
InstallQemuFwCfgTables [QemuFwCfgAcpi.c]
...
InstallOvmfFvTables [AcpiPlatform.c] *
QemuDetected [Qemu.c] *
LocateFvInstanceWithTables [AcpiPlatform.c] *
QemuInstallAcpiTable [Qemu.c] *
QemuInstallAcpiMadtTable [Qemu.c] *
CountBits16 [Qemu.c] *
QemuInstallAcpiSsdtTable [Qemu.c] *
GetSuspendStates [Qemu.c] *
PopulateFwData [Qemu.c] *
with the one below:
InstallAcpiTables [QemuFwCfgAcpiPlatform.c]
InstallQemuFwCfgTables [QemuFwCfgAcpi.c]
...
eliminating the sub-trees highlighted with "*".
There are two consequences:
(1) Xen compatibility is removed from the ACPI platform driver of the
historical OvmfPkg* platforms.
(2) The ACPI tables that are statically built into OVMF (via
"OvmfPkg/AcpiTables/AcpiTables.inf") are never installed. In
particular, OVMF's own runtime preparation of the MADT and SSDT is
eliminated.
Because of (2), remove the "OvmfPkg/AcpiTables/AcpiTables.inf" module as
well -- and then the ACPITABLE build rule too.
Note that (2) only removes effectively dead code; the QEMU ACPI
linker-loader has taken priority since QEMU 1.7.1 (2014). References:
- https://wiki.qemu.org/Planning/1.7
- https://wiki.qemu.org/Features/ACPITableGeneration
- edk2 commit 96bbdbc856 ("OvmfPkg: AcpiPlatformDxe: download ACPI
tables from QEMU", 2014-03-31)
- edk2 commit 387536e472 ("OvmfPkg: AcpiPlatformDxe: implement QEMU's
full ACPI table loader interface", 2014-09-22)
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-4-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
For symmetry with the historical OvmfPkg* platforms, remove the three Xen
drivers from the remotely attested, QEMU/KVM-only, AmdSev platform. Xen
(HVM and PVH) guests are supported by the dedicated OvmfXen platform.
No module remains dependent on XenHypercallLib, so remove the
XenHypercallLib class resolution too, from the DSC file.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-3-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Remove the three Xen drivers as the first step for removing Xen support
from the historical OvmfPkg* platforms. Xen (HVM and PVH) guests are
supported by the dedicated OvmfXen platform.
No module remains dependent on XenHypercallLib, so remove the
XenHypercallLib class resolutions too, from the DSC files.
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2122
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210526201446.12554-2-lersek@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
This error was found while compiling VirtioMmioDeviceLib for X64
with the GCC5 toolchain, where EFIAPI makes a difference.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20210602045935.762211-1-kraxel@redhat.com>
[lersek@redhat.com: prepend module name to subject, trim subject back to
allowed length]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275
The Flush parameter is used to provide a hint whether the specified range
is Mmio address. Now that we have a dedicated helper to clear the
memory encryption mask for the Mmio address range, its safe to remove the
Flush parameter from MemEncryptSev{Set,Clear}PageEncMask().
Since the address specified in the MemEncryptSev{Set,Clear}PageEncMask()
points to a system RAM, thus a cache flush is required during the
encryption mask update.
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
Message-Id: <20210519181949.6574-14-brijesh.singh@amd.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275
The MemEncryptSevClearMmioPageEncMask() helper can be used for clearing
the memory encryption mask for the Mmio region.
The MemEncryptSevClearMmioPageEncMask() is a simplified version of
MemEncryptSevClearPageEncMask() -- it does not flush the caches after
clearing the page encryption mask.
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
Message-Id: <20210519181949.6574-10-brijesh.singh@amd.com>
`CreateDirectoryIfCreating` is used only if `PermitCreation` is set.
`NewNodeIsDirectory` might not set in case of error, but that would lead
to leaving the function before invalid use.
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3228
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Signed-off-by: Sergei Dmitrouk <sergei@posteo.net>
Message-Id: <20210511225616.5942-3-sergei@posteo.net>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3345
The OVMF Tcg2Config PEIM adds the gOvmfTpmMmioAccessiblePpiGuid as a
Depex for IA32 and X64 builds so that the MMIO range is properly mapped
as unencrypted for an SEV-ES guest before the Tcg2Config PEIM is loaded.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Message-Id: <d6538e6c557173d260e272a0e5659683175e2e06.1619716333.git.thomas.lendacky@amd.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3345
During PEI, the MMIO range for the TPM is marked as encrypted when running
as an SEV guest. While this isn't an issue for an SEV guest because of
the way the nested page fault is handled, it does result in an SEV-ES
guest terminating because of a mitigation check in the #VC handler to
prevent MMIO to an encrypted address. For an SEV-ES guest, this range
must be marked as unencrypted.
Create a new x86 PEIM for TPM support that will map the TPM MMIO range as
unencrypted when SEV-ES is active. The gOvmfTpmMmioAccessiblePpiGuid PPI
will be unconditionally installed before exiting. The PEIM will exit with
the EFI_ABORTED status so that the PEIM does not stay resident. This new
PEIM will depend on the installation of the permanent PEI RAM, by
PlatformPei, so that in case page table splitting is required during the
clearing of the encryption bit, the new page table(s) will be allocated
from permanent PEI RAM.
Update all OVMF Ia32 and X64 build packages to include this new PEIM.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Message-Id: <42794cec1f9d5bc24cbfb9dcdbe5e281ef259ef5.1619716333.git.thomas.lendacky@amd.com>
[lersek@redhat.com: refresh subject line]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Define a new PPI GUID that is to be used as a signal of when it is safe
to access the TPM MMIO range. This is needed so that, when SEV is active,
the MMIO range can be mapped unencrypted before it is accessed.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Message-Id: <03e292339273721724c8b14605cfe9d7bbe45a71.1619716333.git.thomas.lendacky@amd.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3345
Enabling TPM support results in guest termination of an SEV-ES guest
because it uses MMIO opcodes that are not currently supported.
Add support for the new MMIO opcodes (0xA0 - 0xA3), MOV instructions which
use a memory offset directly encoded in the instruction. Also, add a DEBUG
statement to identify an unsupported MMIO opcode being used.
Fixes: c45f678a1e
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Min Xu <min.m.xu@intel.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Message-Id: <2fdde57707b52ae39c49341c9d97053aaff56e4a.1619716333.git.thomas.lendacky@amd.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3345
The MOVZX and MOVSX instructions use the ModRM byte in the instruction,
but the instruction decoding support was not decoding it. This resulted
in invalid decoding and failing of the MMIO operation. Also, when
performing the zero-extend or sign-extend operation, the memory operation
should be using the size, and not the size enumeration value.
Add the ModRM byte decoding for the MOVZX and MOVSX opcodes and use the
true data size to perform the extend operations. Additionally, add a
DEBUG statement identifying the MMIO address being flagged as encrypted
during the MMIO address validation.
Fixes: c45f678a1e
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Min Xu <min.m.xu@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Message-Id: <5949d54cb2c9ab69256f67ed5654b32654c0501c.1619716333.git.thomas.lendacky@amd.com>
Update gEfiMdePkgTokenSpaceGuid.PcdFSBClock so it can have the correct
value when SecPeiDxeTimerLibCpu start to use it for the APIC timer.
Currently, nothing appear to use the value in PcdFSBClock before
XenPlatformPei had a chance to set it even though TimerLib is included
in modules run before XenPlatformPei.
XenPlatformPei doesn't use any of the functions that would use that
value. No other modules in the PEI phase seems to use the TimerLib
before PcdFSBClock is set. There are currently two other modules in
the PEI phase that needs the TimerLib:
- S3Resume2Pei, but only because LocalApicLib needs it, but nothing is
using the value from PcdFSBClock.
- CpuMpPei, but I believe it only runs after XenPlatformPei
Before the PEI phase, there's the SEC phase, and SecMain needs
TimerLib because of LocalApicLib. And it initialise the APIC timers
for the debug agent. But I don't think any of the DebugLib that
OvmfXen could use are actually using the *Delay functions in TimerLib,
and so would not use the value from PcdFSBClock which would be
uninitialised.
A simple runtime test showed that TimerLib doesn't use PcdFSBClock
value before it is set.
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2490
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210412133003.146438-8-anthony.perard@citrix.com>
[lersek@redhat.com: cast Freq to UINT32 for PcdSet32S(), not for ASSERT()]
Calculate the frequency of the APIC timer that Xen provides.
Even though the frequency is currently hard-coded, it isn't part of
the public ABI that Xen provides and thus may change at any time. OVMF
needs to determine the frequency by an other mean.
Fortunately, Xen provides a way to determines the frequency of the
TSC, so we can use TSC to calibrate the frequency of the APIC timer.
That information is found in the shared_info page which we map and
unmap once done (XenBusDxe is going to map the page somewhere else).
The shared_info page is mapped at the highest physical address allowed
as it doesn't need to be in the RAM, thus there's a call to update the
page table.
The calculated frequency is only logged in this patch, it will be used
in a following patch.
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2490
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210412133003.146438-7-anthony.perard@citrix.com>
Some information available in a Xen guest can be mapped anywhere in
the physical address space and they don't need to be backed by RAM.
For example, the shared info page.
While it's easier to put those pages anywhere, it is better to avoid
mapping it where the RAM is. It might split a nice 1G guest page table
into 4k pages and thus reducing performance of the guest when it
accesses its memory. Also mapping a page like the shared info page and
then unmapping it or mapping it somewhere else would leave a hole in
the RAM that the guest would propably not be able to use anymore.
So the patch introduces a new function which can be used to 1:1
mapping of guest physical memory above 4G during the PEI phase so we
can map the Xen shared pages outside of memory that can be used by
guest, and as high as possible.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210412133003.146438-6-anthony.perard@citrix.com>
We are going to use the page table structure in yet another place,
collect the types and macro that can be used from another module
rather than making yet another copy.
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2490
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Message-Id: <20210412133003.146438-5-anthony.perard@citrix.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
We are going to use new fields from the Xen headers. Apply the EDK2
coding style so that the code that is going to use it doesn't look out
of place.
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2490
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210412133003.146438-4-anthony.perard@citrix.com>
To avoid nasm generating a warning, replace the macro by the value
expected to be stored in eax.
Ia32/XenPVHMain.asm:76: warning: dword data exceeds bounds
Reported-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210412133003.146438-2-anthony.perard@citrix.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3246
MdeLibs.dsc.inc was added for some basic/default library
instances provided by MdePkg and RegisterFilterLibNull Library
was also added into it as the first version of MdeLibs.dsc.inc.
So update platform dsc to consume MdeLibs.dsc.inc for
RegisterFilterLibNull which will be consumed by IoLib and BaseLib.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Signed-off-by: Dandan Bi <dandan.bi@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
GenFw will embed a NB10 section which contains the path to the input file,
which means the output files have build paths embedded in them. To reduce
information leakage and ensure reproducible builds, pass --zero in release
builds to remove this information.
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3256
Signed-off-by: Ross Burton <ross.burton@arm.com>
Message-Id: <20210324115819.605436-1-ross.burton@arm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
The CommandLine and InitrdData may be set to NULL if the provided
size is too large. Because the zero page is mapped, this would not
cause an immediate crash but can lead to memory corruption instead.
This patch just adds validation and returns error if either allocation
has failed.
Signed-off-by: Martin Radev <martin.b.radev@gmail.com>
Message-Id: <YFPJsaGzVWQxoEU4@martin-ThinkPad-T440p>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
[lersek@redhat.com: drop unnecessary empty line from code; remove personal
(hence likely unstable) repo reference from commit message]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Advertise OVMF support for CPU hot-unplug and negotiate it
if QEMU requests the feature.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Message-Id: <20210312062656.2477515-11-ankur.a.arora@oracle.com>
[lersek@redhat.com: preserve the empty line between the ICH9_LPC_SMI_F_*
group of macro definitions and the SCRATCH_BUFFER type definition]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Add logic in EjectCpu() to do the actual the CPU ejection.
On the BSP, ejection happens by first selecting the CPU via
its QemuSelector and then sending the QEMU "eject" command.
QEMU in-turn signals the remote VCPU thread which context-switches
the CPU out of the SMI handler.
Meanwhile the CPU being ejected, waits around in its holding
area until it is context-switched out. Note that it is possible
that a slow CPU gets ejected before it reaches the wait loop.
However, this would never happen before it has executed the
"AllCpusInSync" loop in SmiRendezvous().
It can mean that an ejected CPU does not execute code after
that point but given that the CPU state will be destroyed by
QEMU, the missed cleanup is no great loss.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Message-Id: <20210312062656.2477515-10-ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
[lersek@redhat.com: unneeded inner QemuSelector declaration in EjectCpu()
triggers VS warning #4456 (local variable shadowed); remove it]
Add EjectCpu(), which handles the CPU ejection, and provides a holding
area for said CPUs. It is called via SmmCpuFeaturesRendezvousExit(),
at the tail end of the SMI handling.
Also UnplugCpus() now stashes QEMU Selectors of CPUs which need to be
ejected in CPU_HOT_EJECT_DATA.QemuSelectorMap. This is used by
EjectCpu() to identify CPUs marked for ejection.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Message-Id: <20210312062656.2477515-9-ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Call the CPU hot-eject handler if one is installed. The condition for
installation is (PcdCpuMaxLogicalProcessorNumber > 1), and there's
a hot-unplug request.
The handler is called from SmmCpuFeaturesRendezvousExit(), which is
in-turn called at the tail-end of SmiRendezvous() after the BSP has
signalled an SMI exit via the "AllCpusInSync" loop.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Message-Id: <20210312062656.2477515-8-ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Init CPU_HOT_EJECT_DATA, which will be used to share CPU ejection
state between SmmCpuFeaturesLib (via PiSmmCpuDxeSmm) and CpuHotPlugSmm.
The init happens via SmmCpuFeaturesSmmRelocationComplete(), and so it
will run as part of the PiSmmCpuDxeSmm entry point function,
PiCpuSmmEntry(). Once inited, CPU_HOT_EJECT_DATA is exposed via
PcdCpuHotEjectDataAddress.
The CPU hot-eject handler (CPU_HOT_EJECT_DATA->Handler) is setup when
there is an ejection request via CpuHotplugSmm.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Message-Id: <20210312062656.2477515-7-ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Define CPU_HOT_EJECT_DATA and add PCD PcdCpuHotEjectDataAddress, which
will be used to share CPU ejection state between OvmfPkg/CpuHotPlugSmm
and PiSmmCpuDxeSmm.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Message-Id: <20210312062656.2477515-6-ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Introduce UnplugCpus() which maps each APIC ID being unplugged
onto the hardware ID of the processor and informs PiSmmCpuDxeSmm
of removal by calling EFI_SMM_CPU_SERVICE_PROTOCOL.RemoveProcessor().
With this change we handle the first phase of unplug where we collect
the CPUs that need to be unplugged and mark them for removal in SMM
data structures.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210312062656.2477515-5-ankur.a.arora@oracle.com>
Add QemuCpuhpWriteCpuStatus() which will be used to update the QEMU
CPU status register. On error, it hangs in a similar fashion as
other helper functions.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210312062656.2477515-4-ankur.a.arora@oracle.com>
Process fw_remove events in QemuCpuhpCollectApicIds(), and collect APIC IDs
and QEMU CPU Selectors for CPUs being hot-unplugged.
In addition, we now ignore CPUs which only have remove set. These
CPUs haven't been processed by OSPM yet.
This is based on the QEMU hot-unplug protocol documented here:
https://lore.kernel.org/qemu-devel/20201204170939.1815522-3-imammedo@redhat.com/
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Message-Id: <20210312062656.2477515-3-ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Refactor CpuHotplugMmi() to pull out the CPU hotplug logic into
ProcessHotAddedCpus(). This is in preparation for supporting CPU
hot-unplug.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Aaron Young <aaron.young@oracle.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132
Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210312062656.2477515-2-ankur.a.arora@oracle.com>
This change added NULL MmUnblockMemoryLib instance in dsc files of
OvmfPkg to pass CI build. When SMM_REQUIRE flag is set, the library
interface is consumed by VariableSmmRuntimeDxe to better support variable
runtime cache feature.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Signed-off-by: Kun Qin <kun.q@outlook.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <MWHPR06MB31028DFAB7AE46E32E5F9F86F3969@MWHPR06MB3102.namprd06.prod.outlook.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3183
Under SEV-ES, a write to the flash device is done using a direct VMGEXIT
to perform an MMIO write. The address provided to the MMIO write must be
the physical address of the MMIO write destitnation. During boot, OVMF
runs with an identity mapped pagetable structure so that VA == PA and the
VMGEXIT MMIO write destination is just the virtual address of the flash
area address being written.
However, when the UEFI SetVirtualAddressMap() API is invoked, an identity
mapped pagetable structure may not be in place and using the virtual
address for the flash area address is no longer valid. This results in
writes to the flash not being performed successfully. This can be seen
by attempting to change the boot order under Linux. The update will
appear to be performed, based on the output of the command. But rebooting
the guest will show that the new boot order has not been set.
To remedy this, save the value of the flash base physical address before
converting the address as part of SetVirtualAddressMap(). The physical
address can then be calculated by obtaining the offset of the MMIO target
virtual address relative to the flash base virtual address and adding that
to the original flash base physical address. The resulting value produces
a successful MMIO write during runtime services.
Fixes: 437eb3f7a8
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Message-Id: <84a5f9161541db5aa3b57c96b737afbcb4b6189d.1611410263.git.thomas.lendacky@amd.com>
[lersek@redhat.com: SetVitualAddressMap() -> SetVirtualAddressMap() typo
fix, in both the commit message and the code comment]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
CpuS3DataDxe allocates the "RegisterTable" and "PreSmmInitRegisterTable"
arrays in ACPI_CPU_DATA just so every processor in the system can have its
own empty register table, matched by APIC ID. This has never been useful
in practice.
Given commit e992cc3f48 ("UefiCpuPkg PiSmmCpuDxeSmm: Reduce SMRAM
consumption in CpuS3.c", 2021-01-11), simply leave both
"AcpiCpuData->RegisterTable" and "AcpiCpuData->PreSmmInitRegisterTable"
initialized to the zero address. This simplifies the driver, and saves
both normal RAM (boot services data type memory) and -- in PiSmmCpuDxeSmm
-- SMRAM.
(This simplification backs out a good chunk of commit 1158fc8e2c
("OvmfPkg/CpuS3DataDxe: enable S3 resume after CPU hotplug", 2020-03-04).
But CpuS3DataDxe still differs between UefiCpuPkg and OvmfPkg, due to the
latter supporting CPU hotplug; thus, we can't remove OvmfPkg/CpuS3DataDxe
altogether.)
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3159
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Message-Id: <20210119155440.2262-5-lersek@redhat.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
Extend parameter list of PciHostBridgeUtilityGetRootBridges() with BusMin/
BusMax, so that the utility function could be compatible with ArmVirtPkg
who uses mutable bus range [BusMin, BusMax] insteand of [0, PCI_MAX_BUS].
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3059
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Jiahui Cen <cenjiahui@huawei.com>
Message-Id: <20210119011302.10908-10-cenjiahui@huawei.com>
[lersek@redhat.com: fix logging of UINTN values BusMin, BusMax]
[lersek@redhat.com: keep zeroing of (*Count) centralized]
[lersek@redhat.com: fix typos in ExtraRootBridges comment]
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Laszlo Ersek
Gerd Hoffmann
Brijesh Singh
Sergei Dmitrouk
Lendacky, Thomas
Anthony PERARD
Ray Ni
Dandan Bi
Ross Burton
Martin Radev
Ankur Arora
Kun Qin
Jiahui Cen via groups.io