Commit Graph

285 Commits

Author SHA1 Message Date
Jiaxin Wu 58d9463939 UefiCpuPkg/PiSmmCpuDxeSmm: Reduce one round BSP & AP sync
After BSP returned from SmmCoreEntry, there are several rounds BSP
and AP sync in BSP handler:

1 .ReleaseAllAPs();  /// Notify all APs to exit.
if (SmmCpuFeaturesNeedConfigureMtrrs()) {
  2. SmmCpuSyncWaitForAPs(); /// Wait for all APs to program MTRRs.
  3. ReleaseAllAPs(); /// Signal APs to restore MTRRs.
}

4. SmmCpuSyncWaitForAPs(); /// Wait for all APs to complete pending
                               tasks including MTRR.
5. ReleaseAllAPs(); /// Signal APs to Reset states.

6. SmmCpuSyncWaitForAPs(); /// Gather APs to exit SMM synchronously.

Before step 6 and after step 5, BSP performs below items:
A. InitializeDebugAgent() /// Stop source level debug.
B. SmmCpuUpdate() /// Perform pending operations for hot-plug.
C. Present = FALSE; /// Clear the Present flag of BSP.

For InitializeDebugAgent(), BSP needs to wait all APs complete
pending tasks and then notify all APs to stop source level debug.
So, above step 4 & step 5 are required for InitializeDebugAgent().

For SmmCpuUpdate(), it's to perform pending operations for
hot-plug CPUs take effect in next SMI. Existing APs in SMI do not
reply on the CPU switch & hot-add & hot-remove operations. So, no
need step 4 and step 5 for additional one round BSP & AP sync.
Step 6 can make sure all APs are ready to exit SMM, then hot-plug
operation can take effect in next SMI.

For BSP "Present" flag, AP does not reply on it. No need step 4
and step 5 for additional one round BSP & AP sync.

Based on above analysis, step 4 and step 5 are only required if
need configure MTRR and support SMM source level debug. So, we can
reduce one round BSP and AP sync if both are unsupported. With
this change, SMI performance can be improved.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@Intel.com>
2023-12-26 07:18:26 +00:00
Jiaxin Wu 41d1c4475b UefiCpuPkg/PiSmmCpuDxeSmm: Invert ReleaseAllAPs & InitializeDebugAgent
Existing BSP handler stops source level debug, then call ReleaseAllAPs
to tell all APs can reset the Present flag to FALSE:
  InitializeDebugAgent (); /// Stop source level debug
  ReleaseAllAPs ();        /// Tell APs can reset "Present" flag.

This patch is to invert ReleaseAllAPs & InitializeDebugAgent:
  ReleaseAllAPs ();        /// Tell APs can reset "Present" flag.
  InitializeDebugAgent (); /// Stop source level debug

After this change, there is no negative impact since SMM source level
debug feature doesn't depend on AP's "Present" flag, no impact to the
SMM source level debug capability.

Instead, the change will benefit the AP source level debug capability
to trace its "Present" flag change for SMI exit since the source
level debug feature will be stopped after each AP has the chance to
reset the state.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@Intel.com>
2023-12-26 07:18:26 +00:00
Jiaxin Wu 3a4ec6de01 UefiCpuPkg/PiSmmCpuDxeSmm: Align BSP and AP sync logic for SMI exit
Below piece of code is the BSP and AP sync logic for SMI exit.
1. AP after finish the scheduled procedure:
  if (SmmCpuFeaturesNeedConfigureMtrrs ()) {
    SmmCpuSyncReleaseBsp ();
    SmmCpuSyncWaitForBsp ();
    ...
  }
  SmmCpuSyncReleaseBsp ();
  SmmCpuSyncWaitForBsp ();
  SmmCpuSyncReleaseBsp ();

2. BSP after return from SmmCoreEntry:
  SmmCpuSyncWaitForAPs ();
  if (SmmCpuFeaturesNeedConfigureMtrrs ()) {
    ReleaseAllAPs ();
    ...
    SmmCpuSyncWaitForAPs ();
  }
  ReleaseAllAPs ();
  SmmCpuSyncWaitForAPs();

This patch is to make BSP same as AP sync logic:
  if (SmmCpuFeaturesNeedConfigureMtrrs ()) {
    SmmCpuSyncWaitForAPs ();
    ReleaseAllAPs ();
    ...
  }
  SmmCpuSyncWaitForAPs ();
  ReleaseAllAPs ();
  SmmCpuSyncWaitForAPs();

With the change, it will be easy to understand the sync flow as
below:
BSP: SmmCpuSyncWaitForAPs  <--  AP: SmmCpuSyncReleaseBsp
BSP: ReleaseAllAPs         -->  AP: SmmCpuSyncWaitForBsp

This patch doesn't have function impact.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@Intel.com>
2023-12-26 07:18:26 +00:00
Jiaxin Wu e1b62f3e28 UefiCpuPkg/PiSmmCpuDxeSmm: Check SMM Debug Agent support or not
This patch is to check SMM Debug Agent support or not before
InitializeDebugAgent.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@Intel.com>
2023-12-26 07:18:26 +00:00
Jiaxin Wu a83d953dc2 UefiCpuPkg/PiSmmCpuDxeSmm: Consume SmmCpuSyncLib
There is the SmmCpuSyncLib Library class define the SMM CPU sync
flow, which is aligned with existing SMM CPU driver sync behavior.
This patch is to consume SmmCpuSyncLib instance directly.

With this change, SMM CPU Sync flow/logic can be customized
with different implementation no matter for any purpose, e.g.
performance tuning, handle specific register, etc.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-20 02:30:56 +00:00
Jiaxin Wu cc698d0335 UefiCpuPkg/PiSmmCpuDxeSmm: Simplify RunningApCount decrement
To decrease the count of RunningApCount, InterlockedDecrement is
enough to achieve that.

This patch is to simplify RunningApCount decrement.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-20 02:30:56 +00:00
Jiaxin Wu e14a022246 UefiCpuPkg/PiSmmCpuDxeSmm: Optimize Semaphore Sync between BSP and AP
This patch is to define 3 new functions (WaitForBsp & ReleaseBsp &
ReleaseOneAp) used for the semaphore sync between BSP & AP. With the
change, BSP and AP Sync flow will be easy understand as below:
BSP: ReleaseAllAPs or ReleaseOneAp --> AP: WaitForBsp
BSP: WaitForAllAPs                 <-- AP: ReleaseBsp

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-20 02:30:56 +00:00
Dun Tan 725acd0b9c UefiCpuPkg: Avoid assuming only one smmbasehob
Modify the gSmmBaseHobGuid consumption code to
remove the asuumption that there is only one
gSmmBaseHobGuid. If the CPU number is big enough,
there will be more than one SmmBaseHob in the
HOB list.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-12-12 01:37:41 +00:00
Dun Tan be44fff723 UefiCpuPkg: Consume MpInfo2Hob in PiSmmCpuDxe
Consume MpInfo2Hob in PiSmmCpuDxe driver to get
NumberOfProcessors, MaxNumberOfCpus and
EFI_PROCESSOR_INFORMATION for all CPU from the
MpInformation2 HOB.
This can avoid calling MP service.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-12-12 01:37:41 +00:00
Wu, Jiaxin 7eb5040607 UefiCpuPkg/PiSmmCpuDxeSmm: Get processor extended information
This patch is to extend SmmAddProcessor function to get processor
extended information. It's to complete commit 1fadd18d.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Star Zeng <star.zeng@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Regression-tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20231115111553.6592-3-jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-08 13:25:11 +00:00
Sheng Wei 553dfb0f57 UefiCpuPkg: Backup and Restore MSR IA32_U_CET in SMI handler.
OS may enable CET-IBT feature by set MSR IA32_U_CET.bit2.
If IA32_U_CET.bit2 is set, CPU is in WAIT_FOR_ENDBRANCH state and
 the next assemble code is not ENDBR, it will trigger #CP exception
 when set CR4.CET bit.
SMI handler needs to backup MSR IA32_U_CET and clear MSR IA32_U_CET
 before set CR4.CET bit,
And SMI handler needs to restore MSR IA32_U_CET when exit SMI handler.

Signed-off-by: Sheng Wei <w.sheng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Tan Dun <dun.tan@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-07 09:43:43 +00:00
Sheng Wei fd1dd8568c UefiCpuPkg: Only change CR4.CET bit for enable and disable CET.
Signed-off-by: Sheng Wei <w.sheng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Tan Dun <dun.tan@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-07 09:43:43 +00:00
Sheng Wei 3018685da8 UefiCpuPkg: Use CET macro definitions in Cet.inc for SmiEntry.nasm files.
Signed-off-by: Sheng Wei <w.sheng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Tan Dun <dun.tan@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-07 09:43:43 +00:00
Sheng Wei 04d47a9bf0 UefiCpuPkg: Use macro CR4_CET_BIT to replace hard code value in Cet.nasm.
Signed-off-by: Sheng Wei <w.sheng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Tan Dun <dun.tan@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-07 09:43:43 +00:00
Sheng Wei b5f20eca8a UefiCpuPkg: Add macro definitions for CET feature for NASM files.
Signed-off-by: Sheng Wei <w.sheng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Wu Jiaxin <jiaxin.wu@intel.com>
Cc: Tan Dun <dun.tan@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-12-07 09:43:43 +00:00
Zhiguang Liu 68d506e0d1 UefiCpuPkg/PiSmmCpuDxeSmm: Use NonSmm BSP as default SMM BSP.
Currently, if BSP election is not enabled, will use Core0 as SMM BSP.
However, Core0 does not always have the highest performance.
So, we can used NonSmm BSP as default BSP.
This will take effect in normal SMM init flow and S3 boot flow.
In normal SMM flow, the code is executed before first SMI.
In S3 flow, the code is executed in Non-SMM BSP's first SMI, where the
gSmmCpuPrivate keeps the data from last boot.

Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
2023-11-29 07:23:37 +00:00
Taylor Beebe 960c7b25c2 UefiCpuPkg: Use Attribute From SMM MemoryAttributesTable if Nonzero
PiSmmCore fetches the EFI memory map and calls SplitTable() to
split each loaded image section into its own descriptor with
EFI_MEMORY_XP marking data sections and EFI_MEMORY_RO marking
code sections.

The SMM MAT logic is almost identical to the DXE MAT logic but goes
a step further and also updates the memory map descriptors which describe
image code and data sections to be of type EfiRuntimeServicesCode and
EfiRuntimeServicesData respectively. The consolidated MAT logic
(present in the new ImagePropertiesRecordLib) more closely follows
the DXE MAT logic which identifies image code sections by the presence
of the attribute EFI_MEMORY_RO in the descriptor and image data
sections by the presence of the attribute EFI_MEMORY_XP. Because of
the flow choice of the consolidated MAT logic, the SMM MAT logic should
just use the attributes from the table returned by SplitTable().

Additionally, the function EnforceMemoryMapAttribute() in the SMM MAT
logic will ensure that the CODE and DATA memory types have the desired
attributes so bisecting this patch series at this commit will still
function as before.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
2023-11-27 18:55:18 +00:00
Jiaxin Wu 589f2e49e5 UefiCpuPkg/PiSmmCpuDxeSmm: Fix CP Exception when CET enable
Root cause:
1. Before DisableReadonlyPageWriteProtect() is called, the return
address (#1) is pushed in shadow stack.
2. CET is disabled.
3. DisableReadonlyPageWriteProtect() returns to #1.
4. Page table is modified.
5. EnableReadonlyPageWriteProtect() is called, but the return
address (#2) is not pushed in shadow stack.
6. CET is enabled.
7. EnableReadonlyPageWriteProtect() returns to #2.
#CP exception happens because the actual return address (#2)
doesn't match the return address stored in shadow stack (#1).

Analysis:
Shadow stack will stop update after CET disable (DisableCet() in
DisableReadOnlyPageWriteProtect), but normal smi stack will be
continue updated with the function called and return
(DisableReadOnlyPageWriteProtect & EnableReadOnlyPageWriteProtect),
thus leading stack mismatch after CET re-enabled (EnableCet() in
EnableReadOnlyPageWriteProtect).

According SDM Vol 3, 6.15-Control Protection Exception:
Normal smi stack and shadow stack must be matched when CET enable,
otherwise CP Exception will happen, which is caused by a near RET
instruction.

CET is disabled in DisableCet(), while can be enabled in
EnableCet(). This way won't cause the problem because they are
implemented in a way that return address of DisableCet() is
poped out from shadow stack (Incsspq performs a pop to increases
the shadow stack) and EnableCet() doesn't use "RET" but "JMP" to
return to caller. So calling EnableCet() and DisableCet() doesn't
have the same issue as calling DisableReadonlyPageWriteProtect()
and EnableReadonlyPageWriteProtect().

With above root cause & analysis, define below 2 macros instead of
functions for WP & CET operation:
WRITE_UNPROTECT_RO_PAGES (Wp, Cet)
WRITE_PROTECT_RO_PAGES (Wp, Cet)
Because DisableCet() & EnableCet() must be in the same function
to avoid shadow stack and normal SMI stack mismatch.

Note: WRITE_UNPROTECT_RO_PAGES () must be called pair with
WRITE_PROTECT_RO_PAGES () in same function.

Change-Id: I4e126697efcd8dbfb4887da034d8691bfca969e3
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
2023-11-10 01:21:12 +00:00
Dun Tan 2ce5ae43c2 UefiCpuPkg/PiSmmCpuDxe: use MpService2Ppi to wakeup AP in s3
Use MpService2Ppi to wakeup AP in s3 boot flow during initializing
CPU. If mSmmS3ResumeState->MpService2Ppi is not 0, then BSP will
use MpService2Ppi->StartupAllCPUs to do CPU initialization for both
BSP and AP instead of only sending InitSipiSipi for AP.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
2023-09-08 06:19:34 +00:00
Dun Tan cf82c09bac UefiCpuPkg/PiSmmCpuDxe: code refinement for CpuS3.c
This commit is code logic refinement for s3 boot flow
in CpuS3.c. It doesn't change any code functionality.
This commit implementes InitializeAp and InitializeBsp
as a single function since they are doing almost the
same thing. Then both BSP and AP will execute the same
function InitializeCpuProcedure to do CPU initialization.
This can make the code logic easier to understand.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
2023-09-08 06:19:34 +00:00
Zhang, Hongbin1 1fadd18d0c UefiCpuPkg: Get processor extended information for SmmCpuServiceProtocol
Some features like RAS need to use processor extended information
under smm, So add code to support it

Signed-off-by: Hongbin1 Zhang <hongbin1.zhang@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Cc: Star Zeng <star.zeng@intel.com>
Reviewed-by: Jiaxin Wu <jiaxin.wu@intel.com>
2023-07-05 07:38:26 +00:00
Abdul Lateef Attar fbb6f18e1c UefiCpuPkg: Removes SmmCpuFeaturesReadSaveStateRegister
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=4182

Removes SmmCpuFeaturesReadSaveStateRegister and
SmmCpuFeaturesWirteSaveStateRegister function from
SmmCpuFeaturesLib library.

MmSaveStateLib library replaces the functionality of the above
functions.
Platform old/new need to use MmSaveStateLib library to read/write save
state registers.
Current implementation supports Intel and AMD.

Cc: Paul Grimes <paul.grimes@amd.com>
Cc: Abner Chang <abner.chang@amd.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Signed-off-by: Abdul Lateef Attar <abdattar@amd.com>
Reviewed-by: Abner Chang <abner.chang@amd.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-07-03 03:01:10 +00:00
Dun Tan f5c987fcac UefiCpuPkg/PiSmmCpuDxeSmm: Remove unnecessary function
Remove unnecessary function SetNotPresentPage(). We can directly
use ConvertMemoryPageAttributes to set a range to non-present.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan 4ceefd6dd1 UefiCpuPkg: Refinement to smm runtime InitPaging() code
This commit is code refinement to current smm runtime InitPaging()
page table update code. In InitPaging(), if PcdCpuSmmProfileEnable
is TRUE, use ConvertMemoryPageAttributes() API to map the range in
mProtectionMemRange to the attrbute recorded in the attribute field
of mProtectionMemRange, map the range outside mProtectionMemRange
as non-present. If PcdCpuSmmProfileEnable is FALSE, only need to
set the ranges not in mSmmCpuSmramRanges as NX.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan b4394cca2d UefiCpuPkg: Sort mProtectionMemRange when ReadyToLock
Sort mProtectionMemRange in InitProtectedMemRange() when
ReadyToLock.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan 7132df94ac UefiCpuPkg: Sort mSmmCpuSmramRanges in FindSmramInfo
Sort mSmmCpuSmramRanges after get the SMRAM info in
FindSmramInfo() function.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan b4dde1ae6a UefiCpuPkg: Use GenSmmPageTable() to create Smm S3 page table
Use GenSmmPageTable() to create both IA32 and X64 Smm S3
page table.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan 701b5797b2 UefiCpuPkg: Add GenSmmPageTable() to create smm page table
This commit is code refinement to current smm pagetable generation
code. Add a new GenSmmPageTable() API to create smm page table
based on the PageTableMap() API in CpuPageTableLib. Caller only
needs to specify the paging mode and the PhysicalAddressBits to map.
This function can be used to create both IA32 pae paging and X64
5level, 4level paging.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan d706d9c64a UefiCpuPkg: Extern mSmmShadowStackSize in PiSmmCpuDxeSmm.h
Extern mSmmShadowStackSize in PiSmmCpuDxeSmm.h and remove
extern for mSmmShadowStackSize in c files to simplify code.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan ef64ae06f8 UefiCpuPkg/PiSmmCpuDxeSmm: Clear CR0.WP before modify page table
Clear CR0.WP before modify smm page table. Currently, there is
an assumption that smm pagetable is always RW before ReadyToLock.
However, when AMD SEV is enabled, FvbServicesSmm driver calls
MemEncryptSevClearMmioPageEncMask to clear AddressEncMask bit
in smm page table for this range:
[PcdOvmfFdBaseAddress,PcdOvmfFdBaseAddress+PcdOvmfFirmwareFdSize]
If page slpit happens in this process, new memory for smm page
table is allocated. Then the newly allocated page table memory
is marked as RO in smm page table in this FvbServicesSmm driver,
which may lead to PF if smm code doesn't clear CR0.WP before
modify smm page table when ReadyToLock.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan 7b6e7d0098 UefiCpuPkg/PiSmmCpuDxeSmm: Add 2 function to disable/enable CR0.WP
Add two functions to disable/enable CR0.WP. These two unctions
will also be used in later commits. This commit doesn't change any
functionality.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan f51967280b UefiCpuPkg/PiSmmCpuDxeSmm: Avoid setting non-present range to RO/NX
In PiSmmCpuDxeSmm code, SetMemMapAttributes() marks memory ranges
in SmmMemoryAttributesTable to RO/NX. There may exist non-present
range in these memory ranges. Set other attributes for a non-present
range is not permitted in CpuPageTableMapLib. So add code to handle
this case. Only map the present ranges in SmmMemoryAttributesTable
to RO or NX.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan 6585ced558 UefiCpuPkg: Add DEBUG_CODE for special case when clear RP
In ConvertMemoryPageAttributes() function, when clear RP for a
specific range [BaseAddress, BaseAddress + Length], it means to
set the present bit to 1 and assign default value for other
attributes in page table. The default attributes for the input
specific range are NX disabled and ReadOnly. If there is existing
present range in [BaseAddress, BaseAddress + Length] and the
attributes are not NX disabled or not ReadOnly, then output the
DEBUG message to indicate that the NX and ReadOnly attributes of
the existing present range are modified in the function.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Dun Tan 2d212083d0 UefiCpuPkg: Use CpuPageTableLib to convert SMM paging attribute.
Simplify the ConvertMemoryPageAttributes API to convert paging
attribute by CpuPageTableLib. In the new API, it calls
PageTableMap() to update the page attributes of a memory range.
With the PageTableMap() API in CpuPageTableLib, we can remove
the complicated page table manipulating code.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
2023-06-30 11:07:40 +05:30
Ray Ni a87f5fdb3d UefiCpuPkg/SmmCpu: Add PcdSmmApPerfLogEnable control AP perf-logging
When a platform has lots of CPU cores/threads, perf-logging on every
AP produces lots of records. When this multiplies with number of SMIs
during post, the records are even more.

So, this patch adds a new PCD PcdSmmApPerfLogEnable (default TRUE)
to allow platform to turn off perf-logging on APs.

Signed-off-by: Ray Ni <ray.ni@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
2023-06-21 04:37:17 +00:00
Ray Ni ad6c8540cc UefiCpuPkg/CpuSmm: Add perf-logging for MP procedures
MP procedures are those procedures that run in every CPU thread.
The EDKII perf infra is not MP safe so it doesn't support to be called
from those MP procedures.

The patch adds SMM MP perf-logging support in SmmMpPerf.c.
The following procedures are perf-logged:
* SmmInitHandler
* SmmCpuFeaturesRendezvousEntry
* PlatformValidSmi
* SmmCpuFeaturesRendezvousExit

Signed-off-by: Ray Ni <ray.ni@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
2023-06-21 04:37:17 +00:00
Ray Ni 0da3df78ff UefiCpuPkg/CpuSmm: Add perf-logging for time-consuming BSP procedures
The patch adds perf-logging for the following potential
time-consuming BSP procedures:
* PiCpuSmmEntry
  - SmmRelocateBases
  - ExecuteFirstSmiInit

* BSPHandler
  - SmmWaitForApArrival
  - PerformRemainingTasks
    * InitPaging
    * SetMemMapAttributes
    * SetUefiMemMapAttributes
    * SetPageTableAttributes
    * ConfigSmmCodeAccessCheck
    * SmmCpuFeaturesCompleteSmmReadyToLock

Signed-off-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
2023-06-21 04:37:17 +00:00
Zhihao Li 9f12d6b6ec UefiCpuPkg/PiSmmCpuDxeSmm:add Ap Rendezvous check in PerformRemainingTasks.
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4424

In Relaxed-AP Sync Mode, BSP will not wait for all Aps arrive. However,
PerformRemainingTasks() needs to wait all Aps arrive before calling
SetMemMapAttributes and ConfigSmmCodeAccessCheck() when mSmmReadyToLock
is true. In SetMemMapAttributes(), SmmSetMemoryAttributesEx() will call
FlushTlbForAll() that need to start up the aps. So it need to let all
aps arrive. Same as SetMemMapAttributes(), ConfigSmmCodeAccessCheck()
also will start up the aps.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>

Signed-off-by: Zhihao Li <zhihao.li@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-05-31 09:23:01 +00:00
Zhihao Li 5bb7bfbe81 UefiCpuPkg/CpuService.c:check cpu sync mode in SmmCpuRendezvous()
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4431

In Ap relaxed mode, some SMI handlers should call SmmWaitForApArrival() to let all ap arrive in SmmCpuRendezvous(). But in traditional mode, these SMI handlers don't need to call SmmWaitForApArrival() again. So it need to be check cpu sync mode before calling SmmWaitForApArrival().

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>

Signed-off-by: Zhihao Li <zhihao.li@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-05-31 09:23:01 +00:00
Rebecca Cran 089013a697 UefiCpuPkg: Update code to be more C11 compliant by using __func__
__FUNCTION__ is a pre-standard extension that gcc and Visual C++ among
others support, while __func__ was standardized in C99.

Since it's more standard, replace __FUNCTION__ with __func__ throughout
UefiCpuPkg.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Sunil V L <sunilvl@ventanamicro.com>
2023-04-10 14:19:57 +00:00
Gerd Hoffmann 54051768cd UefiCpuPkg/PiSmmCpuDxeSmm: fix format string
BufferPages is UINTN, so we need "%Lu" when printing it to avoid
it being truncated.  Also cast to UINT64 to make sure it works
for 32bit builds too.

Fixes: 4f441d024b ("UefiCpuPkg/PiSmmCpuDxeSmm: fix error handling")
Reported-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-04-01 03:15:51 +00:00
Gerd Hoffmann 4f441d024b UefiCpuPkg/PiSmmCpuDxeSmm: fix error handling
ASSERT() is not proper handling of allocation failures, it gets compiled
out on RELEASE builds.  Print a message and enter dead loop instead.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-03-22 04:04:57 +00:00
Gerd Hoffmann 2e71876081 UefiCpuPkg/PiSmmCpuDxeSmm: drop support for obsolete processors
It's highly unlikely the code ever runs on processors which are
almost 30 years old.  Drop the code handling them.

Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=4345
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-03-22 04:04:57 +00:00
Wu, Jiaxin a1386bb252 UefiCpuPkg/PiSmmCpuDxeSmm: Fix S3 failure in SmmRestoreCpu
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4368

This issue is caused by the commit:
ec07fd0e35

GetFirstGuidHob() should not be used after exit boot service.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-03-14 01:29:52 +00:00
Yu Pu c20408163f UefiCpuPkg: Remove UefiCpuLib from module INFs.
Because UefiCpuPkg/UefiCpuLib is merged to MdePkg/CpuLib, remove the
dependency of UefiCpuLib.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Yu Pu <yu.pu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-03-10 08:23:56 +00:00
Wu, Jiaxin ec07fd0e35 UefiCpuPkg/PiSmmCpuDxeSmm: Consume SMM Base Hob for SmBase info
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4337

Existing SMBASE Relocation is in the PiSmmCpuDxeSmm driver, which
will relocate the SMBASE of each processor by setting the SMBASE
field in the saved state map (at offset 7EF8h) to a new value.
The RSM instruction reloads the internal SMBASE register with the
value in SMBASE field when each time it exits SMM. All subsequent
SMI requests will use the new SMBASE to find the starting address
for the SMI handler (at SMBASE + 8000h).

Due to the default SMBASE for all x86 processors is 0x30000, the
APs' 1st SMI for rebase has to be executed one by one to avoid
the processors over-writing each other's SMM Save State Area (see
existing SmmRelocateBases() function), which means the next AP has
to wait for the previous AP to finish its 1st SMI, then it can call
into its 1st SMI for rebase via Smi Ipi command, thus leading the
existing SMBASE Relocation has to be running in series. Besides, it
needs very complex code to handle the AP exit semaphore
(mRebased[Index]), which will hook return address of SMM Save State
so that semaphore code can be executed immediately after AP exits
SMM for SMBASE relocation (see existing SemaphoreHook() function).

With SMM Base Hob support, PiSmmCpuDxeSmm does not need the RSM
instruction to do the SMBASE Relocation. SMBASE Register for each
processors have already been programmed and all SMBASE address have
recorded in SMM Base Hob. So the same default SMBASE Address
(0x30000) will not be used, thus the processors over-writing each
other's SMM Save State Area will not happen in PiSmmCpuDxeSmm driver.
This way makes the first SMI init can be executed in parallel and
save boot time on multi-core system. Besides, Semaphore Hook code
logic is also not required, which will greatly simplify the SMBASE
Relocation flow.

Mainly changes as below:
* Assume the biggest possibility of tile size is 8k.
* Combine 2 SMIs (gcSmmInitTemplate & gcSmiHandlerTemplate) into one
(gcSmiHandlerTemplate), the new SMI handler needs to run to 2 paths:
one to SmmCpuFeaturesInitializeProcessor(), the other to SMM Core
Entry Point.
* Issue SMI IPI (All Excluding Self SMM IPI + BSP SMM IPI) for first
SMI init before normal SMI sources happen.
* Call SmmCpuFeaturesInitializeProcessor() in parallel.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
2023-03-06 06:07:48 +00:00
Wu, Jiaxin 85c6c14c4c UefiCpuPkg/PiSmmCpuDxeSmm: Replace mIsBsp by mBspApicId check
This patch is to replace mIsBsp by mBspApicId check.
mIsBsp becomes the local variable (IsBsp), then it can be
checked dynamically in the function. Instead, we define the
mBspApicId, which is to record the BSP ApicId used for
compare in SmmInitHandler. With this change, SmmInitHandler
can be run in parallel during SMM init.

Note:
This patch is the per-prepared work by refining the
SmmInitHandler, then, we can do the next step to
combine 2 SMIs (gcSmmInitTemplate & gcSmiHandlerTemplate)
into one (gcSmiHandlerTemplate), the new SMI handler
will call the SmmInitHandler in parallel to do the init.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
2023-03-06 06:07:48 +00:00
Wu, Jiaxin cb4820b6c1 UefiCpuPkg/PiSmmCpuDxeSmm: Fix invalid InitializeMpSyncData call
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4338

No need call InitializeMpSyncData during normal boot SMI init,
because mSmmMpSyncData is NULL at that time. mSmmMpSyncData is
allocated in InitializeMpServiceData, which is invoked after
normal boot SMI init (SmmRelocateBases).

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zeng Star <star.zeng@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Signed-off-by: Jiaxin Wu <jiaxin.wu@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Ray Ni <ray.ni@Intel.com>
2023-03-06 06:07:48 +00:00
Zhiguang Liu 13b97736c8 UefiCpuPkg: Fix SMM code hangs when InitPaging
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4246

In function InitPaging, NumberOfPml5Entries is calculated by below code
NumberOfPml5Entries = (UINTN)LShiftU64 (1, SizeOfMemorySpace - 48);
If the SizeOfMemorySpace is larger than 48, NumberOfPml5Entries will be
larger than 1. However, this doesn't make sense if the hardware doesn't
support 5 level page table.

Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Reviewed-by: Star Zeng <star.zeng@intel.com>
Reviewed-by: Wu, Jiaxin <jiaxin.wu@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Eric Dong <eric.dong@intel.com>
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
2023-02-03 08:24:55 +00:00
Tan, Dun b670700ddf UefiCpuPkg/PiSmmCpuDxeSmm:Fix PF issue caused by smm page table code
When setting new page table pool to RO, only disable/enable WP when
Cr0.WP has been set to 1 to fix potential PF caused by b822be1a20
(UefiCpuPkg/PiSmmCpuDxeSmm: Introduce page table pool mechanism).
With previous code, if someone want to modify the page table and
Cr0.WP has been cleared before modify page table, Cr0.WP may be set
to 1 again since new pool may be generated during this process
Then PF fault may happens.

Signed-off-by: Dun Tan <dun.tan@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
2023-01-03 06:41:11 +00:00