OvmfPkg/PlatformPei: fix MTRR for low-RAM sizes that have many bits clear

Assume that we boot OVMF in a QEMU guest with 1025 MB of RAM. The
following assertion will fire:

> ASSERT_EFI_ERROR (Status = Out of Resources)
> ASSERT OvmfPkg/PlatformPei/MemDetect.c(696): !EFI_ERROR (Status)

That's because the range [1025 MB, 4 GB) that we try to mark as
uncacheable with MTRRs has size 3071 MB:

   0x1_0000_0000
  -0x0_4010_0000
  --------------
   0x0_BFF0_0000

The integer that stands for the uncacheable area size has 11 (eleven) bits
set to 1. As a result, covering this size requires 11 variable MTRRs (each
MTRR must cover a naturally aligned, power-of-two sized area). But, if we
need more variable MTRRs than the CPU can muster (such as 8), then
MtrrSetMemoryAttribute() fails, and we refuse to continue booting (which
is justified, in itself).

Unfortunately, this is not difficult to trigger, and the error message is
well-hidden from end-users, in the OVMF debug log. The following
mitigation is inspired by SeaBIOS:

Truncate the uncacheable area size to a power-of-two, while keeping the
end fixed at 4 GB. Such an interval can be covered by just one variable
MTRR.

This may leave such an MMIO gap, between the end of low-RAM and the start
of the uncacheable area, that is marked as WB (through the MTRR default).
Raise the base of the 32-bit PCI MMIO aperture accordingly -- the gap will
not be used for anything.

On Q35, the minimal 32-bit PCI MMIO aperture (triggered by RAM size 2815
MB) shrinks from

  0xE000_0000 - 0xAFF0_0000 = 769 MB

to

  0xE000_0000 - 0xC000_0000 = 512 MB

On i440fx, the minimal 32-bit PCI MMIO aperture (triggered by RAM size
3583 MB) shrinks from

  0xFC00_0000 - 0xDFF0_0000 = 449 MB

to

  0xFC00_0000 - 0xE000_0000 = 448 MB

Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=1814
Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1666941
Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1701710
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

OvmfPkg/PlatformPei/MemDetect.c

@@ -42,6 +42,8 @@ STATIC UINT32 mS3AcpiReservedMemorySize;
 
 STATIC UINT16 mQ35TsegMbytes;
 
+UINT32 mQemuUc32Base;
+
 VOID
 Q35TsegMbytesInitialization (
   VOID
@@ -663,6 +665,8 @@ QemuInitializeRam (
   // cover it exactly.
   //
   if (IsMtrrSupported ()) {
+    UINT32 Uc32Size;
+
     MtrrGetAllMtrrs (&MtrrSettings);
 
     //
@@ -689,11 +693,24 @@ QemuInitializeRam (
 
     //
     // Set memory range from the "top of lower RAM" (RAM below 4GB) to 4GB as
-    // uncacheable
+    // uncacheable. Make sure one variable MTRR suffices by truncating the size
+    // to a whole power of two. This will round the base *up*, and a gap (not
+    // used for either RAM or MMIO) may stay in the middle, marked as
+    // cacheable-by-default.
     //
-    Status = MtrrSetMemoryAttribute (LowerMemorySize,
-               SIZE_4GB - LowerMemorySize, CacheUncacheable);
+    Uc32Size = GetPowerOfTwo32 ((UINT32)(SIZE_4GB - LowerMemorySize));
+    mQemuUc32Base = (UINT32)(SIZE_4GB - Uc32Size);
+    if (mQemuUc32Base != LowerMemorySize) {
+      DEBUG ((DEBUG_VERBOSE, "%a: rounded UC32 base from 0x%x up to 0x%x, for "
+        "an UC32 size of 0x%x\n", __FUNCTION__, (UINT32)LowerMemorySize,
+        mQemuUc32Base, Uc32Size));
+    }
+
+    Status = MtrrSetMemoryAttribute (mQemuUc32Base, Uc32Size,
+               CacheUncacheable);
     ASSERT_EFI_ERROR (Status);
+  } else {
+    mQemuUc32Base = (UINT32)LowerMemorySize;
   }
 }
 

OvmfPkg/PlatformPei/Platform.c

@@ -174,14 +174,12 @@ MemMapInitialization (
   AddIoMemoryRangeHob (0x0A0000, BASE_1MB);
 
   if (!mXen) {
-    UINT32  TopOfLowRam;
     UINT64  PciExBarBase;
     UINT32  PciBase;
     UINT32  PciSize;
 
-    TopOfLowRam = GetSystemMemorySizeBelow4gb ();
     PciExBarBase = 0;
-    PciBase = (TopOfLowRam < BASE_2GB) ? BASE_2GB : TopOfLowRam;
+    PciBase = (mQemuUc32Base < BASE_2GB) ? BASE_2GB : mQemuUc32Base;
     if (mHostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
       //
       // The 32-bit PCI host aperture is expected to fall between the top of

OvmfPkg/PlatformPei/Platform.h

@@ -114,4 +114,6 @@ extern UINT32 mMaxCpuCount;
 
 extern UINT16 mHostBridgeDevId;
 
+extern UINT32 mQemuUc32Base;
+
 #endif // _PLATFORM_PEI_H_INCLUDED_