tyler.durden
/
audk
mirror of https://github.com/acidanthera/audk.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

UefiCpuPkg/RegisterCpuFeaturesLib: Add logic to support semaphore type. 

V4 changes include:
1. Serial debug message for different threads when program the register table.

V3 changes include:
1. Use global variable instead of internal function to return string for register type
   and dependence type.
2. Add comments for some complicated logic.

V2 changes include:
1. Add more description for the code part which need easy to understand.
2. Refine some code base on feedback for V1 changes.

V1 changes include:
In a system which has multiple cores, current set register value task costs huge times.
After investigation, current set MSR task costs most of the times. Current logic uses
SpinLock to let set MSR task as an single thread task for all cores. Because MSR has
scope attribute which may cause GP fault if multiple APs set MSR at the same time,
current logic use an easiest solution (use SpinLock) to avoid this issue, but it will
cost huge times.

In order to fix this performance issue, new solution will set MSRs base on their scope
attribute. After this, the SpinLock will not needed. Without SpinLock, new issue raised
which is caused by MSR dependence. For example, MSR A depends on MSR B which means MSR A
must been set after MSR B has been set. Also MSR B is package scope level and MSR A is
thread scope level. If system has multiple threads, Thread 1 needs to set the thread level
MSRs and thread 2 needs to set thread and package level MSRs. Set MSRs task for thread 1
and thread 2 like below:

            Thread 1                 Thread 2
MSR B          N                        Y
MSR A          Y                        Y

If driver don't control execute MSR order, for thread 1, it will execute MSR A first, but
at this time, MSR B not been executed yet by thread 2. system may trig exception at this
time.

In order to fix the above issue, driver introduces semaphore logic to control the MSR
execute sequence. For the above case, a semaphore will be add between MSR A and B for
all threads. Semaphore has scope info for it. The possible scope value is core or package.
For each thread, when it meets a semaphore during it set registers, it will 1) release
semaphore (+1) for each threads in this core or package(based on the scope info for this
semaphore) 2) acquire semaphore (-1) for all the threads in this core or package(based
on the scope info for this semaphore). With these two steps, driver can control MSR
sequence. Sample code logic like below:

  //
  // First increase semaphore count by 1 for processors in this package.
  //
  for (ProcessorIndex = 0; ProcessorIndex < PackageThreadsCount ; ProcessorIndex ++) {
    LibReleaseSemaphore ((UINT32 *) &SemaphorePtr[PackageOffset + ProcessorIndex]);
  }
  //
  // Second, check whether the count has reach the check number.
  //
  for (ProcessorIndex = 0; ProcessorIndex < ValidApCount; ProcessorIndex ++) {
    LibWaitForSemaphore (&SemaphorePtr[ApOffset]);
  }

Platform Requirement:
1. This change requires register MSR setting base on MSR scope info. If still register MSR
   for all threads, exception may raised.

Known limitation:
1. Current CpuFeatures driver supports DXE instance and PEI instance. But semaphore logic
   requires Aps execute in async mode which is not supported by PEI driver. So CpuFeature
   PEI instance not works after this change. We plan to support async mode for PEI in phase
   2 for this task.

Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>
This commit is contained in:
Eric Dong 2018-10-17 09:31:03 +08:00
parent 87e9395109
commit b3c71b472d
6 changed files with 940 additions and 134 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

395
UefiCpuPkg/Library/RegisterCpuFeaturesLib/CpuFeaturesInitialize.c
View File

@ -14,6 +14,9 @@
#include "RegisterCpuFeatures.h" #include "RegisterCpuFeatures.h"
CHAR16 *mDependTypeStr[] = {L"None", L"Thread", L"Core", L"Package", L"Invalid" };
CHAR16 *mRegisterTypeStr[] = {L"MSR", L"CR", L"MMIO", L"CACHE", L"SEMAP", L"INVALID" };
/** /**
Worker function to save PcdCpuFeaturesCapability. Worker function to save PcdCpuFeaturesCapability.
@ -145,6 +148,19 @@ CpuInitDataInitialize (
CPU_FEATURES_INIT_ORDER *InitOrder; CPU_FEATURES_INIT_ORDER *InitOrder;
CPU_FEATURES_DATA *CpuFeaturesData; CPU_FEATURES_DATA *CpuFeaturesData;
LIST_ENTRY *Entry; LIST_ENTRY *Entry;
UINT32 Core;
UINT32 Package;
UINT32 Thread;
EFI_CPU_PHYSICAL_LOCATION *Location;
BOOLEAN *CoresVisited;
UINTN Index;
ACPI_CPU_DATA *AcpiCpuData;
CPU_STATUS_INFORMATION *CpuStatus;
UINT32 *ValidCoreCountPerPackage;
Core = 0;
Package = 0;
Thread = 0;
CpuFeaturesData = GetCpuFeaturesData (); CpuFeaturesData = GetCpuFeaturesData ();
CpuFeaturesData->InitOrder = AllocateZeroPool (sizeof (CPU_FEATURES_INIT_ORDER) * NumberOfCpus); CpuFeaturesData->InitOrder = AllocateZeroPool (sizeof (CPU_FEATURES_INIT_ORDER) * NumberOfCpus);
@ -163,6 +179,17 @@ CpuInitDataInitialize (
Entry = Entry->ForwardLink; Entry = Entry->ForwardLink;
} }
CpuFeaturesData->NumberOfCpus = (UINT32) NumberOfCpus;
AcpiCpuData = GetAcpiCpuData ();
ASSERT (AcpiCpuData != NULL);
CpuFeaturesData->AcpiCpuData= AcpiCpuData;
CpuStatus = &AcpiCpuData->CpuStatus;
Location = AllocateZeroPool (sizeof (EFI_CPU_PHYSICAL_LOCATION) * NumberOfCpus);
ASSERT (Location != NULL);
AcpiCpuData->ApLocation = (EFI_PHYSICAL_ADDRESS)(UINTN)Location;
for (ProcessorNumber = 0; ProcessorNumber < NumberOfCpus; ProcessorNumber++) { for (ProcessorNumber = 0; ProcessorNumber < NumberOfCpus; ProcessorNumber++) {
InitOrder = &CpuFeaturesData->InitOrder[ProcessorNumber]; InitOrder = &CpuFeaturesData->InitOrder[ProcessorNumber];
InitOrder->FeaturesSupportedMask = AllocateZeroPool (CpuFeaturesData->BitMaskSize); InitOrder->FeaturesSupportedMask = AllocateZeroPool (CpuFeaturesData->BitMaskSize);
@ -175,7 +202,76 @@ CpuInitDataInitialize (
&ProcessorInfoBuffer, &ProcessorInfoBuffer,
sizeof (EFI_PROCESSOR_INFORMATION) sizeof (EFI_PROCESSOR_INFORMATION)
); );
CopyMem (
&Location[ProcessorNumber],
&ProcessorInfoBuffer.Location,
sizeof (EFI_CPU_PHYSICAL_LOCATION)
);
//
// Collect CPU package count info.
//
if (Package < ProcessorInfoBuffer.Location.Package) {
Package = ProcessorInfoBuffer.Location.Package;
} }
//
// Collect CPU max core count info.
//
if (Core < ProcessorInfoBuffer.Location.Core) {
Core = ProcessorInfoBuffer.Location.Core;
}
//
// Collect CPU max thread count info.
//
if (Thread < ProcessorInfoBuffer.Location.Thread) {
Thread = ProcessorInfoBuffer.Location.Thread;
}
}
CpuStatus->PackageCount = Package + 1;
CpuStatus->MaxCoreCount = Core + 1;
CpuStatus->MaxThreadCount = Thread + 1;
DEBUG ((DEBUG_INFO, "Processor Info: Package: %d, MaxCore : %d, MaxThread: %d\n",
CpuStatus->PackageCount,
CpuStatus->MaxCoreCount,
CpuStatus->MaxThreadCount));
//
// Collect valid core count in each package because not all cores are valid.
//
ValidCoreCountPerPackage= AllocateZeroPool (sizeof (UINT32) * CpuStatus->PackageCount);
ASSERT (ValidCoreCountPerPackage != 0);
CpuStatus->ValidCoreCountPerPackage = (EFI_PHYSICAL_ADDRESS)(UINTN)ValidCoreCountPerPackage;
CoresVisited = AllocatePool (sizeof (BOOLEAN) * CpuStatus->MaxCoreCount);
ASSERT (CoresVisited != NULL);
for (Index = 0; Index < CpuStatus->PackageCount; Index ++ ) {
ZeroMem (CoresVisited, sizeof (BOOLEAN) * CpuStatus->MaxCoreCount);
//
// Collect valid cores in Current package.
//
for (ProcessorNumber = 0; ProcessorNumber < NumberOfCpus; ProcessorNumber++) {
Location = &CpuFeaturesData->InitOrder[ProcessorNumber].CpuInfo.ProcessorInfo.Location;
if (Location->Package == Index && !CoresVisited[Location->Core] ) {
//
// The ValidCores position for Location->Core is valid.
// The possible values in ValidCores[Index] are 0 or 1.
// FALSE means no valid threads in this Core.
// TRUE means have valid threads in this core, no matter the thead count is 1 or more.
//
CoresVisited[Location->Core] = TRUE;
ValidCoreCountPerPackage[Index]++;
}
}
}
FreePool (CoresVisited);
for (Index = 0; Index <= Package; Index++) {
DEBUG ((DEBUG_INFO, "Package: %d, Valid Core : %d\n", Index, ValidCoreCountPerPackage[Index]));
}
CpuFeaturesData->CpuFlags.SemaphoreCount = AllocateZeroPool (sizeof (UINT32) * CpuStatus->PackageCount * CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
ASSERT (CpuFeaturesData->CpuFlags.SemaphoreCount != NULL);
// //
// Get support and configuration PCDs // Get support and configuration PCDs
// //
@ -310,7 +406,7 @@ CollectProcessorData (
LIST_ENTRY *Entry; LIST_ENTRY *Entry;
CPU_FEATURES_DATA *CpuFeaturesData; CPU_FEATURES_DATA *CpuFeaturesData;
CpuFeaturesData = GetCpuFeaturesData (); CpuFeaturesData = (CPU_FEATURES_DATA *)Buffer;
ProcessorNumber = GetProcessorIndex (); ProcessorNumber = GetProcessorIndex ();
CpuInfo = &CpuFeaturesData->InitOrder[ProcessorNumber].CpuInfo; CpuInfo = &CpuFeaturesData->InitOrder[ProcessorNumber].CpuInfo;
// //
@ -416,6 +512,15 @@ DumpRegisterTableOnProcessor (
RegisterTableEntry->Value RegisterTableEntry->Value
)); ));
break; break;
case Semaphore:
DEBUG ((
DebugPrintErrorLevel,
"Processor: %d: Semaphore: Scope Value: %s\r\n",
ProcessorNumber,
mDependTypeStr[MIN (RegisterTableEntry->Value, InvalidDepType)]
));
break;
default: default:
break; break;
} }
@ -441,6 +546,11 @@ AnalysisProcessorFeatures (
REGISTER_CPU_FEATURE_INFORMATION *CpuInfo; REGISTER_CPU_FEATURE_INFORMATION *CpuInfo;
LIST_ENTRY *Entry; LIST_ENTRY *Entry;
CPU_FEATURES_DATA *CpuFeaturesData; CPU_FEATURES_DATA *CpuFeaturesData;
LIST_ENTRY *NextEntry;
CPU_FEATURES_ENTRY *NextCpuFeatureInOrder;
BOOLEAN Success;
CPU_FEATURE_DEPENDENCE_TYPE BeforeDep;
CPU_FEATURE_DEPENDENCE_TYPE AfterDep;
CpuFeaturesData = GetCpuFeaturesData (); CpuFeaturesData = GetCpuFeaturesData ();
CpuFeaturesData->CapabilityPcd = AllocatePool (CpuFeaturesData->BitMaskSize); CpuFeaturesData->CapabilityPcd = AllocatePool (CpuFeaturesData->BitMaskSize);
@ -517,8 +627,15 @@ AnalysisProcessorFeatures (
// //
CpuInfo = &CpuFeaturesData->InitOrder[ProcessorNumber].CpuInfo; CpuInfo = &CpuFeaturesData->InitOrder[ProcessorNumber].CpuInfo;
Entry = GetFirstNode (&CpuInitOrder->OrderList); Entry = GetFirstNode (&CpuInitOrder->OrderList);
NextEntry = Entry->ForwardLink;
while (!IsNull (&CpuInitOrder->OrderList, Entry)) { while (!IsNull (&CpuInitOrder->OrderList, Entry)) {
CpuFeatureInOrder = CPU_FEATURE_ENTRY_FROM_LINK (Entry); CpuFeatureInOrder = CPU_FEATURE_ENTRY_FROM_LINK (Entry);
if (!IsNull (&CpuInitOrder->OrderList, NextEntry)) {
NextCpuFeatureInOrder = CPU_FEATURE_ENTRY_FROM_LINK (NextEntry);
} else {
NextCpuFeatureInOrder = NULL;
}
Success = FALSE;
if (IsBitMaskMatch (CpuFeatureInOrder->FeatureMask, CpuFeaturesData->SettingPcd)) { if (IsBitMaskMatch (CpuFeatureInOrder->FeatureMask, CpuFeaturesData->SettingPcd)) {
Status = CpuFeatureInOrder->InitializeFunc (ProcessorNumber, CpuInfo, CpuFeatureInOrder->ConfigData, TRUE); Status = CpuFeatureInOrder->InitializeFunc (ProcessorNumber, CpuInfo, CpuFeatureInOrder->ConfigData, TRUE);
if (EFI_ERROR (Status)) { if (EFI_ERROR (Status)) {
@ -532,6 +649,8 @@ AnalysisProcessorFeatures (
DEBUG ((DEBUG_WARN, "Warning :: Failed to enable Feature: Mask = ")); DEBUG ((DEBUG_WARN, "Warning :: Failed to enable Feature: Mask = "));
DumpCpuFeatureMask (CpuFeatureInOrder->FeatureMask); DumpCpuFeatureMask (CpuFeatureInOrder->FeatureMask);
} }
} else {
Success = TRUE;
} }
} else { } else {
Status = CpuFeatureInOrder->InitializeFunc (ProcessorNumber, CpuInfo, CpuFeatureInOrder->ConfigData, FALSE); Status = CpuFeatureInOrder->InitializeFunc (ProcessorNumber, CpuInfo, CpuFeatureInOrder->ConfigData, FALSE);
@ -542,9 +661,36 @@ AnalysisProcessorFeatures (
DEBUG ((DEBUG_WARN, "Warning :: Failed to disable Feature: Mask = ")); DEBUG ((DEBUG_WARN, "Warning :: Failed to disable Feature: Mask = "));
DumpCpuFeatureMask (CpuFeatureInOrder->FeatureMask); DumpCpuFeatureMask (CpuFeatureInOrder->FeatureMask);
} }
} else {
Success = TRUE;
} }
} }
if (Success) {
//
// If feature has dependence with the next feature (ONLY care core/package dependency).
// and feature initialize succeed, add sync semaphere here.
//
BeforeDep = DetectFeatureScope (CpuFeatureInOrder, TRUE);
if (NextCpuFeatureInOrder != NULL) {
AfterDep = DetectFeatureScope (NextCpuFeatureInOrder, FALSE);
} else {
AfterDep = NoneDepType;
}
//
// Assume only one of the depend is valid.
//
ASSERT (!(BeforeDep > ThreadDepType && AfterDep > ThreadDepType));
if (BeforeDep > ThreadDepType) {
CPU_REGISTER_TABLE_WRITE32 (ProcessorNumber, Semaphore, 0, BeforeDep);
}
if (AfterDep > ThreadDepType) {
CPU_REGISTER_TABLE_WRITE32 (ProcessorNumber, Semaphore, 0, AfterDep);
}
}
Entry = Entry->ForwardLink; Entry = Entry->ForwardLink;
NextEntry = Entry->ForwardLink;
} }
// //
@ -561,27 +707,77 @@ AnalysisProcessorFeatures (
} }
} }
/**
Increment semaphore by 1.
@param Sem IN: 32-bit unsigned integer
**/
VOID
LibReleaseSemaphore (
IN OUT volatile UINT32 *Sem
)
{
InterlockedIncrement (Sem);
}
/**
Decrement the semaphore by 1 if it is not zero.
Performs an atomic decrement operation for semaphore.
The compare exchange operation must be performed using
MP safe mechanisms.
@param Sem IN: 32-bit unsigned integer
**/
VOID
LibWaitForSemaphore (
IN OUT volatile UINT32 *Sem
)
{
UINT32 Value;
do {
Value = *Sem;
} while (Value == 0 ||
InterlockedCompareExchange32 (
Sem,
Value,
Value - 1
) != Value);
}
/** /**
Initialize the CPU registers from a register table. Initialize the CPU registers from a register table.
@param[in] ProcessorNumber The index of the CPU executing this function. @param[in] RegisterTable The register table for this AP.
@param[in] ApLocation AP location info for this ap.
@param[in] CpuStatus CPU status info for this CPU.
@param[in] CpuFlags Flags data structure used when program the register.
@note This service could be called by BSP/APs. @note This service could be called by BSP/APs.
**/ **/
VOID VOID
ProgramProcessorRegister ( ProgramProcessorRegister (
IN UINTN ProcessorNumber IN CPU_REGISTER_TABLE *RegisterTable,
IN EFI_CPU_PHYSICAL_LOCATION *ApLocation,
IN CPU_STATUS_INFORMATION *CpuStatus,
IN PROGRAM_CPU_REGISTER_FLAGS *CpuFlags
) )
{ {
CPU_FEATURES_DATA *CpuFeaturesData;
CPU_REGISTER_TABLE *RegisterTable;
CPU_REGISTER_TABLE_ENTRY *RegisterTableEntry; CPU_REGISTER_TABLE_ENTRY *RegisterTableEntry;
UINTN Index; UINTN Index;
UINTN Value; UINTN Value;
CPU_REGISTER_TABLE_ENTRY *RegisterTableEntryHead; CPU_REGISTER_TABLE_ENTRY *RegisterTableEntryHead;
volatile UINT32 *SemaphorePtr;
CpuFeaturesData = GetCpuFeaturesData (); UINT32 FirstThread;
RegisterTable = &CpuFeaturesData->RegisterTable[ProcessorNumber]; UINT32 PackageThreadsCount;
UINT32 CurrentThread;
UINTN ProcessorIndex;
UINTN ThreadIndex;
UINTN ValidThreadCount;
UINT32 *ValidCoreCountPerPackage;
// //
// Traverse Register Table of this logical processor // Traverse Register Table of this logical processor
@ -592,6 +788,21 @@ ProgramProcessorRegister (
RegisterTableEntry = &RegisterTableEntryHead[Index]; RegisterTableEntry = &RegisterTableEntryHead[Index];
DEBUG_CODE_BEGIN ();
AcquireSpinLock (&CpuFlags->ConsoleLogLock);
ThreadIndex = ApLocation->Package * CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount +
ApLocation->Core * CpuStatus->MaxThreadCount +
ApLocation->Thread;
DEBUG ((
DEBUG_INFO,
"Processor = %lu, Entry Index %lu, Type = %s!\n",
(UINT64)ThreadIndex,
(UINT64)Index,
mRegisterTypeStr[MIN ((REGISTER_TYPE)RegisterTableEntry->RegisterType, InvalidReg)]
));
ReleaseSpinLock (&CpuFlags->ConsoleLogLock);
DEBUG_CODE_END ();
// //
// Check the type of specified register // Check the type of specified register
// //
@ -654,10 +865,6 @@ ProgramProcessorRegister (
// The specified register is Model Specific Register // The specified register is Model Specific Register
// //
case Msr: case Msr:
//
// Get lock to avoid Package/Core scope MSRs programming issue in parallel execution mode
//
AcquireSpinLock (&CpuFeaturesData->MsrLock);
if (RegisterTableEntry->ValidBitLength >= 64) { if (RegisterTableEntry->ValidBitLength >= 64) {
// //
// If length is not less than 64 bits, then directly write without reading // If length is not less than 64 bits, then directly write without reading
@ -677,20 +884,19 @@ ProgramProcessorRegister (
RegisterTableEntry->Value RegisterTableEntry->Value
); );
} }
ReleaseSpinLock (&CpuFeaturesData->MsrLock);
break; break;
// //
// MemoryMapped operations // MemoryMapped operations
// //
case MemoryMapped: case MemoryMapped:
AcquireSpinLock (&CpuFeaturesData->MemoryMappedLock); AcquireSpinLock (&CpuFlags->MemoryMappedLock);
MmioBitFieldWrite32 ( MmioBitFieldWrite32 (
(UINTN)(RegisterTableEntry->Index | LShiftU64 (RegisterTableEntry->HighIndex, 32)), (UINTN)(RegisterTableEntry->Index | LShiftU64 (RegisterTableEntry->HighIndex, 32)),
RegisterTableEntry->ValidBitStart, RegisterTableEntry->ValidBitStart,
RegisterTableEntry->ValidBitStart + RegisterTableEntry->ValidBitLength - 1, RegisterTableEntry->ValidBitStart + RegisterTableEntry->ValidBitLength - 1,
(UINT32)RegisterTableEntry->Value (UINT32)RegisterTableEntry->Value
); );
ReleaseSpinLock (&CpuFeaturesData->MemoryMappedLock); ReleaseSpinLock (&CpuFlags->MemoryMappedLock);
break; break;
// //
// Enable or disable cache // Enable or disable cache
@ -706,6 +912,90 @@ ProgramProcessorRegister (
} }
break; break;
case Semaphore:
// Semaphore works logic like below:
//
// V(x) = LibReleaseSemaphore (Semaphore[FirstThread + x]);
// P(x) = LibWaitForSemaphore (Semaphore[FirstThread + x]);
//
// All threads (T0...Tn) waits in P() line and continues running
// together.
//
//
// T0 T1 ... Tn
//
// V(0...n) V(0...n) ... V(0...n)
// n * P(0) n * P(1) ... n * P(n)
//
SemaphorePtr = CpuFlags->SemaphoreCount;
switch (RegisterTableEntry->Value) {
case CoreDepType:
//
// Get Offset info for the first thread in the core which current thread belongs to.
//
FirstThread = (ApLocation->Package * CpuStatus->MaxCoreCount + ApLocation->Core) * CpuStatus->MaxThreadCount;
CurrentThread = FirstThread + ApLocation->Thread;
//
// First Notify all threads in current Core that this thread has ready.
//
for (ProcessorIndex = 0; ProcessorIndex < CpuStatus->MaxThreadCount; ProcessorIndex ++) {
LibReleaseSemaphore ((UINT32 *) &SemaphorePtr[FirstThread + ProcessorIndex]);
}
//
// Second, check whether all valid threads in current core have ready.
//
for (ProcessorIndex = 0; ProcessorIndex < CpuStatus->MaxThreadCount; ProcessorIndex ++) {
LibWaitForSemaphore (&SemaphorePtr[CurrentThread]);
}
break;
case PackageDepType:
ValidCoreCountPerPackage = (UINT32 *)(UINTN)CpuStatus->ValidCoreCountPerPackage;
//
// Get Offset info for the first thread in the package which current thread belongs to.
//
FirstThread = ApLocation->Package * CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount;
//
// Get the possible threads count for current package.
//
PackageThreadsCount = CpuStatus->MaxThreadCount * CpuStatus->MaxCoreCount;
CurrentThread = FirstThread + CpuStatus->MaxThreadCount * ApLocation->Core + ApLocation->Thread;
//
// Get the valid thread count for current package.
//
ValidThreadCount = CpuStatus->MaxThreadCount * ValidCoreCountPerPackage[ApLocation->Package];
//
// Different packages may have different valid cores in them. If driver maintail clearly
// cores number in different packages, the logic will be much complicated.
// Here driver just simply records the max core number in all packages and use it as expect
// core number for all packages.
// In below two steps logic, first current thread will Release semaphore for each thread
// in current package. Maybe some threads are not valid in this package, but driver don't
// care. Second, driver will let current thread wait semaphore for all valid threads in
// current package. Because only the valid threads will do release semaphore for this
// thread, driver here only need to wait the valid thread count.
//
//
// First Notify ALL THREADS in current package that this thread has ready.
//
for (ProcessorIndex = 0; ProcessorIndex < PackageThreadsCount ; ProcessorIndex ++) {
LibReleaseSemaphore ((UINT32 *) &SemaphorePtr[FirstThread + ProcessorIndex]);
}
//
// Second, check whether VALID THREADS (not all threads) in current package have ready.
//
for (ProcessorIndex = 0; ProcessorIndex < ValidThreadCount; ProcessorIndex ++) {
LibWaitForSemaphore (&SemaphorePtr[CurrentThread]);
}
break;
default:
break;
}
break;
default: default:
break; break;
} }
@ -724,10 +1014,36 @@ SetProcessorRegister (
IN OUT VOID *Buffer IN OUT VOID *Buffer
) )
{ {
UINTN ProcessorNumber; CPU_FEATURES_DATA *CpuFeaturesData;
CPU_REGISTER_TABLE *RegisterTable;
CPU_REGISTER_TABLE *RegisterTables;
UINT32 InitApicId;
UINTN ProcIndex;
UINTN Index;
ACPI_CPU_DATA *AcpiCpuData;
ProcessorNumber = GetProcessorIndex (); CpuFeaturesData = (CPU_FEATURES_DATA *) Buffer;
ProgramProcessorRegister (ProcessorNumber); AcpiCpuData = CpuFeaturesData->AcpiCpuData;
RegisterTables = (CPU_REGISTER_TABLE *)(UINTN)AcpiCpuData->RegisterTable;
InitApicId = GetInitialApicId ();
RegisterTable = NULL;
for (Index = 0; Index < AcpiCpuData->NumberOfCpus; Index++) {
if (RegisterTables[Index].InitialApicId == InitApicId) {
RegisterTable = &RegisterTables[Index];
ProcIndex = Index;
break;
}
}
ASSERT (RegisterTable != NULL);
ProgramProcessorRegister (
RegisterTable,
(EFI_CPU_PHYSICAL_LOCATION *)(UINTN)AcpiCpuData->ApLocation + ProcIndex,
&AcpiCpuData->CpuStatus,
&CpuFeaturesData->CpuFlags
);
} }
/** /**
@ -746,6 +1062,9 @@ CpuFeaturesDetect (
{ {
UINTN NumberOfCpus; UINTN NumberOfCpus;
UINTN NumberOfEnabledProcessors; UINTN NumberOfEnabledProcessors;
CPU_FEATURES_DATA *CpuFeaturesData;
CpuFeaturesData = GetCpuFeaturesData();
GetNumberOfProcessor (&NumberOfCpus, &NumberOfEnabledProcessors); GetNumberOfProcessor (&NumberOfCpus, &NumberOfEnabledProcessors);
@ -754,49 +1073,13 @@ CpuFeaturesDetect (
// //
// Wakeup all APs for data collection. // Wakeup all APs for data collection.
// //
StartupAPsWorker (CollectProcessorData); StartupAPsWorker (CollectProcessorData, NULL);
// //
// Collect data on BSP // Collect data on BSP
// //
CollectProcessorData (NULL); CollectProcessorData (CpuFeaturesData);
AnalysisProcessorFeatures (NumberOfCpus); AnalysisProcessorFeatures (NumberOfCpus);
} }
/**
Performs CPU features Initialization.
This service will invoke MP service to perform CPU features
initialization on BSP/APs per user configuration.
@note This service could be called by BSP only.
**/
VOID
EFIAPI
CpuFeaturesInitialize (
VOID
)
{
CPU_FEATURES_DATA *CpuFeaturesData;
UINTN OldBspNumber;
CpuFeaturesData = GetCpuFeaturesData ();
OldBspNumber = GetProcessorIndex();
CpuFeaturesData->BspNumber = OldBspNumber;
//
// Wakeup all APs for programming.
//
StartupAPsWorker (SetProcessorRegister);
//
// Programming BSP
//
SetProcessorRegister (NULL);
//
// Switch to new BSP if required
//
if (CpuFeaturesData->BspNumber != OldBspNumber) {
SwitchNewBsp (CpuFeaturesData->BspNumber);
}
}

71
UefiCpuPkg/Library/RegisterCpuFeaturesLib/DxeRegisterCpuFeaturesLib.c
View File

@ -15,6 +15,7 @@
#include <PiDxe.h> #include <PiDxe.h>
#include <Library/UefiBootServicesTableLib.h> #include <Library/UefiBootServicesTableLib.h>
#include <Library/UefiLib.h>
#include "RegisterCpuFeatures.h" #include "RegisterCpuFeatures.h"
@ -115,14 +116,20 @@ GetProcessorInformation (
@param[in] Procedure A pointer to the function to be run on @param[in] Procedure A pointer to the function to be run on
enabled APs of the system. enabled APs of the system.
@param[in] MpEvent A pointer to the event to be used later
to check whether procedure has done.
**/ **/
VOID VOID
StartupAPsWorker ( StartupAPsWorker (
IN EFI_AP_PROCEDURE Procedure IN EFI_AP_PROCEDURE Procedure,
IN EFI_EVENT MpEvent
) )
{ {
EFI_STATUS Status; EFI_STATUS Status;
EFI_MP_SERVICES_PROTOCOL *MpServices; EFI_MP_SERVICES_PROTOCOL *MpServices;
CPU_FEATURES_DATA *CpuFeaturesData;
CpuFeaturesData = GetCpuFeaturesData ();
MpServices = GetMpProtocol (); MpServices = GetMpProtocol ();
// //
@ -132,9 +139,9 @@ StartupAPsWorker (
MpServices, MpServices,
Procedure, Procedure,
FALSE, FALSE,
NULL, MpEvent,
0, 0,
NULL, CpuFeaturesData,
NULL NULL
); );
ASSERT_EFI_ERROR (Status); ASSERT_EFI_ERROR (Status);
@ -197,3 +204,61 @@ GetNumberOfProcessor (
ASSERT_EFI_ERROR (Status); ASSERT_EFI_ERROR (Status);
} }
/**
Performs CPU features Initialization.
This service will invoke MP service to perform CPU features
initialization on BSP/APs per user configuration.
@note This service could be called by BSP only.
**/
VOID
EFIAPI
CpuFeaturesInitialize (
VOID
)
{
CPU_FEATURES_DATA *CpuFeaturesData;
UINTN OldBspNumber;
EFI_EVENT MpEvent;
EFI_STATUS Status;
CpuFeaturesData = GetCpuFeaturesData ();
OldBspNumber = GetProcessorIndex();
CpuFeaturesData->BspNumber = OldBspNumber;
Status = gBS->CreateEvent (
EVT_NOTIFY_WAIT,
TPL_CALLBACK,
EfiEventEmptyFunction,
NULL,
&MpEvent
);
ASSERT_EFI_ERROR (Status);
//
// Wakeup all APs for programming.
//
StartupAPsWorker (SetProcessorRegister, MpEvent);
//
// Programming BSP
//
SetProcessorRegister (CpuFeaturesData);
//
// Wait all processors to finish the task.
//
do {
Status = gBS->CheckEvent (MpEvent);
} while (Status == EFI_NOT_READY);
ASSERT_EFI_ERROR (Status);
//
// Switch to new BSP if required
//
if (CpuFeaturesData->BspNumber != OldBspNumber) {
SwitchNewBsp (CpuFeaturesData->BspNumber);
}
}

2
UefiCpuPkg/Library/RegisterCpuFeaturesLib/DxeRegisterCpuFeaturesLib.inf
View File

@ -47,6 +47,8 @@
SynchronizationLib SynchronizationLib
UefiBootServicesTableLib UefiBootServicesTableLib
IoLib IoLib
UefiBootServicesTableLib
UefiLib
[Protocols] [Protocols]
gEfiMpServiceProtocolGuid ## CONSUMES gEfiMpServiceProtocolGuid ## CONSUMES

55
UefiCpuPkg/Library/RegisterCpuFeaturesLib/PeiRegisterCpuFeaturesLib.c
View File

@ -149,11 +149,15 @@ GetProcessorInformation (
**/ **/
VOID VOID
StartupAPsWorker ( StartupAPsWorker (
IN EFI_AP_PROCEDURE Procedure IN EFI_AP_PROCEDURE Procedure,
IN EFI_EVENT MpEvent
) )
{ {
EFI_STATUS Status; EFI_STATUS Status;
EFI_PEI_MP_SERVICES_PPI *CpuMpPpi; EFI_PEI_MP_SERVICES_PPI *CpuMpPpi;
CPU_FEATURES_DATA *CpuFeaturesData;
CpuFeaturesData = GetCpuFeaturesData ();
// //
// Get MP Services Protocol // Get MP Services Protocol
@ -175,7 +179,7 @@ StartupAPsWorker (
Procedure, Procedure,
FALSE, FALSE,
0, 0,
NULL CpuFeaturesData
); );
ASSERT_EFI_ERROR (Status); ASSERT_EFI_ERROR (Status);
} }
@ -257,3 +261,50 @@ GetNumberOfProcessor (
); );
ASSERT_EFI_ERROR (Status); ASSERT_EFI_ERROR (Status);
} }
/**
Performs CPU features Initialization.
This service will invoke MP service to perform CPU features
initialization on BSP/APs per user configuration.
@note This service could be called by BSP only.
**/
VOID
EFIAPI
CpuFeaturesInitialize (
VOID
)
{
CPU_FEATURES_DATA *CpuFeaturesData;
UINTN OldBspNumber;
CpuFeaturesData = GetCpuFeaturesData ();
OldBspNumber = GetProcessorIndex();
CpuFeaturesData->BspNumber = OldBspNumber;
//
// Known limitation: In PEI phase, CpuFeatures driver not
// support async mode execute tasks. So semaphore type
// register can't been used for this instance, must use
// DXE type instance.
//
//
// Wakeup all APs for programming.
//
StartupAPsWorker (SetProcessorRegister, NULL);
//
// Programming BSP
//
SetProcessorRegister (CpuFeaturesData);
//
// Switch to new BSP if required
//
if (CpuFeaturesData->BspNumber != OldBspNumber) {
SwitchNewBsp (CpuFeaturesData->BspNumber);
}
}

62
UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h
View File

@ -23,6 +23,7 @@
#include <Library/MemoryAllocationLib.h> #include <Library/MemoryAllocationLib.h>
#include <Library/SynchronizationLib.h> #include <Library/SynchronizationLib.h>
#include <Library/IoLib.h> #include <Library/IoLib.h>
#include <Library/LocalApicLib.h>
#include <AcpiCpuData.h> #include <AcpiCpuData.h>
@ -46,16 +47,27 @@ typedef struct {
CPU_FEATURE_INITIALIZE InitializeFunc; CPU_FEATURE_INITIALIZE InitializeFunc;
UINT8 *BeforeFeatureBitMask; UINT8 *BeforeFeatureBitMask;
UINT8 *AfterFeatureBitMask; UINT8 *AfterFeatureBitMask;
UINT8 *CoreBeforeFeatureBitMask;
UINT8 *CoreAfterFeatureBitMask;
UINT8 *PackageBeforeFeatureBitMask;
UINT8 *PackageAfterFeatureBitMask;
VOID *ConfigData; VOID *ConfigData;
BOOLEAN BeforeAll; BOOLEAN BeforeAll;
BOOLEAN AfterAll; BOOLEAN AfterAll;
} CPU_FEATURES_ENTRY; } CPU_FEATURES_ENTRY;
//
// Flags used when program the register.
//
typedef struct {
volatile UINTN ConsoleLogLock; // Spinlock used to control console.
volatile UINTN MemoryMappedLock; // Spinlock used to program mmio
volatile UINT32 *SemaphoreCount; // Semaphore used to program semaphore.
} PROGRAM_CPU_REGISTER_FLAGS;
typedef struct { typedef struct {
UINTN FeaturesCount; UINTN FeaturesCount;
UINT32 BitMaskSize; UINT32 BitMaskSize;
SPIN_LOCK MsrLock;
SPIN_LOCK MemoryMappedLock;
LIST_ENTRY FeatureList; LIST_ENTRY FeatureList;
CPU_FEATURES_INIT_ORDER *InitOrder; CPU_FEATURES_INIT_ORDER *InitOrder;
@ -64,9 +76,14 @@ typedef struct {
UINT8 *ConfigurationPcd; UINT8 *ConfigurationPcd;
UINT8 *SettingPcd; UINT8 *SettingPcd;
UINT32 NumberOfCpus;
ACPI_CPU_DATA *AcpiCpuData;
CPU_REGISTER_TABLE *RegisterTable; CPU_REGISTER_TABLE *RegisterTable;
CPU_REGISTER_TABLE *PreSmmRegisterTable; CPU_REGISTER_TABLE *PreSmmRegisterTable;
UINTN BspNumber; UINTN BspNumber;
PROGRAM_CPU_REGISTER_FLAGS CpuFlags;
} CPU_FEATURES_DATA; } CPU_FEATURES_DATA;
#define CPU_FEATURE_ENTRY_FROM_LINK(a) \ #define CPU_FEATURE_ENTRY_FROM_LINK(a) \
@ -118,10 +135,13 @@ GetProcessorInformation (
@param[in] Procedure A pointer to the function to be run on @param[in] Procedure A pointer to the function to be run on
enabled APs of the system. enabled APs of the system.
@param[in] MpEvent A pointer to the event to be used later
to check whether procedure has done.
**/ **/
VOID VOID
StartupAPsWorker ( StartupAPsWorker (
IN EFI_AP_PROCEDURE Procedure IN EFI_AP_PROCEDURE Procedure,
IN EFI_EVENT MpEvent
); );
/** /**
@ -170,4 +190,40 @@ DumpCpuFeature (
IN CPU_FEATURES_ENTRY *CpuFeature IN CPU_FEATURES_ENTRY *CpuFeature
); );
/**
Return feature dependence result.
@param[in] CpuFeature Pointer to CPU feature.
@param[in] Before Check before dependence or after.
@retval return the dependence result.
**/
CPU_FEATURE_DEPENDENCE_TYPE
DetectFeatureScope (
IN CPU_FEATURES_ENTRY *CpuFeature,
IN BOOLEAN Before
);
/**
Programs registers for the calling processor.
@param[in,out] Buffer The pointer to private data buffer.
**/
VOID
EFIAPI
SetProcessorRegister (
IN OUT VOID *Buffer
);
/**
Return ACPI_CPU_DATA data.
@return Pointer to ACPI_CPU_DATA data.
**/
ACPI_CPU_DATA *
GetAcpiCpuData (
VOID
);
#endif #endif

463
UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeaturesLib.c
View File

@ -112,6 +112,302 @@ IsBitMaskMatchCheck (
return FALSE; return FALSE;
} }
/**
Return feature dependence result.
@param[in] CpuFeature Pointer to CPU feature.
@param[in] Before Check before dependence or after.
@retval return the dependence result.
**/
CPU_FEATURE_DEPENDENCE_TYPE
DetectFeatureScope (
IN CPU_FEATURES_ENTRY *CpuFeature,
IN BOOLEAN Before
)
{
if (Before) {
if (CpuFeature->PackageBeforeFeatureBitMask != NULL) {
return PackageDepType;
}
if (CpuFeature->CoreBeforeFeatureBitMask != NULL) {
return CoreDepType;
}
if (CpuFeature->BeforeFeatureBitMask != NULL) {
return ThreadDepType;
}
return NoneDepType;
}
if (CpuFeature->PackageAfterFeatureBitMask != NULL) {
return PackageDepType;
}
if (CpuFeature->CoreAfterFeatureBitMask != NULL) {
return CoreDepType;
}
if (CpuFeature->AfterFeatureBitMask != NULL) {
return ThreadDepType;
}
return NoneDepType;
}
/**
Clear dependence for the specified type.
@param[in] CurrentFeature Cpu feature need to clear.
@param[in] Before Before or after dependence relationship.
**/
VOID
ClearFeatureScope (
IN CPU_FEATURES_ENTRY *CpuFeature,
IN BOOLEAN Before
)
{
if (Before) {
if (CpuFeature->BeforeFeatureBitMask != NULL) {
FreePool (CpuFeature->BeforeFeatureBitMask);
CpuFeature->BeforeFeatureBitMask = NULL;
}
if (CpuFeature->CoreBeforeFeatureBitMask != NULL) {
FreePool (CpuFeature->CoreBeforeFeatureBitMask);
CpuFeature->CoreBeforeFeatureBitMask = NULL;
}
if (CpuFeature->PackageBeforeFeatureBitMask != NULL) {
FreePool (CpuFeature->PackageBeforeFeatureBitMask);
CpuFeature->PackageBeforeFeatureBitMask = NULL;
}
} else {
if (CpuFeature->PackageAfterFeatureBitMask != NULL) {
FreePool (CpuFeature->PackageAfterFeatureBitMask);
CpuFeature->PackageAfterFeatureBitMask = NULL;
}
if (CpuFeature->CoreAfterFeatureBitMask != NULL) {
FreePool (CpuFeature->CoreAfterFeatureBitMask);
CpuFeature->CoreAfterFeatureBitMask = NULL;
}
if (CpuFeature->AfterFeatureBitMask != NULL) {
FreePool (CpuFeature->AfterFeatureBitMask);
CpuFeature->AfterFeatureBitMask = NULL;
}
}
}
/**
Base on dependence relationship to asjust feature dependence.
ONLY when the feature before(or after) the find feature also has
dependence with the find feature. In this case, driver need to base
on dependce relationship to decide how to insert current feature and
adjust the feature dependence.
@param[in] PreviousFeature CPU feature current before the find one.
@param[in] CurrentFeature Cpu feature need to adjust.
@param[in] Before Before or after dependence relationship.
@retval TRUE means the current feature dependence has been adjusted.
@retval FALSE means the previous feature dependence has been adjusted.
or previous feature has no dependence with the find one.
**/
BOOLEAN
AdjustFeaturesDependence (
IN OUT CPU_FEATURES_ENTRY *PreviousFeature,
IN OUT CPU_FEATURES_ENTRY *CurrentFeature,
IN BOOLEAN Before
)
{
CPU_FEATURE_DEPENDENCE_TYPE PreDependType;
CPU_FEATURE_DEPENDENCE_TYPE CurrentDependType;
PreDependType = DetectFeatureScope(PreviousFeature, Before);
CurrentDependType = DetectFeatureScope(CurrentFeature, Before);
//
// If previous feature has no dependence with the find featue.
// return FALSE.
//
if (PreDependType == NoneDepType) {
return FALSE;
}
//
// If both feature have dependence, keep the one which needs use more
// processors and clear the dependence for the other one.
//
if (PreDependType >= CurrentDependType) {
ClearFeatureScope (CurrentFeature, Before);
return TRUE;
} else {
ClearFeatureScope (PreviousFeature, Before);
return FALSE;
}
}
/**
Base on dependence relationship to asjust feature order.
@param[in] FeatureList Pointer to CPU feature list
@param[in] FindEntry The entry this feature depend on.
@param[in] CurrentEntry The entry for this feature.
@param[in] Before Before or after dependence relationship.
**/
VOID
AdjustEntry (
IN LIST_ENTRY *FeatureList,
IN OUT LIST_ENTRY *FindEntry,
IN OUT LIST_ENTRY *CurrentEntry,
IN BOOLEAN Before
)
{
LIST_ENTRY *PreviousEntry;
CPU_FEATURES_ENTRY *PreviousFeature;
CPU_FEATURES_ENTRY *CurrentFeature;
//
// For CPU feature which has core or package type dependence, later code need to insert
// AcquireSpinLock/ReleaseSpinLock logic to sequency the execute order.
// So if driver finds both feature A and B need to execute before feature C, driver will
// base on dependence type of feature A and B to update the logic here.
// For example, feature A has package type dependence and feature B has core type dependence,
// because package type dependence need to wait for more processors which has strong dependence
// than core type dependence. So driver will adjust the feature order to B -> A -> C. and driver
// will remove the feature dependence in feature B.
// Driver just needs to make sure before feature C been executed, feature A has finished its task
// in all all thread. Feature A finished in all threads also means feature B have finshed in all
// threads.
//
if (Before) {
PreviousEntry = GetPreviousNode (FeatureList, FindEntry);
} else {
PreviousEntry = GetNextNode (FeatureList, FindEntry);
}
CurrentFeature = CPU_FEATURE_ENTRY_FROM_LINK (CurrentEntry);
RemoveEntryList (CurrentEntry);
if (IsNull (FeatureList, PreviousEntry)) {
//
// If not exist the previous or next entry, just insert the current entry.
//
if (Before) {
InsertTailList (FindEntry, CurrentEntry);
} else {
InsertHeadList (FindEntry, CurrentEntry);
}
} else {
//
// If exist the previous or next entry, need to check it before insert curent entry.
//
PreviousFeature = CPU_FEATURE_ENTRY_FROM_LINK (PreviousEntry);
if (AdjustFeaturesDependence (PreviousFeature, CurrentFeature, Before)) {
//
// Return TRUE means current feature dependence has been cleared and the previous
// feature dependence has been kept and used. So insert current feature before (or after)
// the previous feature.
//
if (Before) {
InsertTailList (PreviousEntry, CurrentEntry);
} else {
InsertHeadList (PreviousEntry, CurrentEntry);
}
} else {
if (Before) {
InsertTailList (FindEntry, CurrentEntry);
} else {
InsertHeadList (FindEntry, CurrentEntry);
}
}
}
}
/**
Checks and adjusts current CPU features per dependency relationship.
@param[in] FeatureList Pointer to CPU feature list
@param[in] CurrentEntry Pointer to current checked CPU feature
@param[in] FeatureMask The feature bit mask.
@retval return Swapped info.
**/
BOOLEAN
InsertToBeforeEntry (
IN LIST_ENTRY *FeatureList,
IN LIST_ENTRY *CurrentEntry,
IN UINT8 *FeatureMask
)
{
LIST_ENTRY *CheckEntry;
CPU_FEATURES_ENTRY *CheckFeature;
BOOLEAN Swapped;
Swapped = FALSE;
//
// Check all features dispatched before this entry
//
CheckEntry = GetFirstNode (FeatureList);
while (CheckEntry != CurrentEntry) {
CheckFeature = CPU_FEATURE_ENTRY_FROM_LINK (CheckEntry);
if (IsBitMaskMatchCheck (CheckFeature->FeatureMask, FeatureMask)) {
AdjustEntry (FeatureList, CheckEntry, CurrentEntry, TRUE);
Swapped = TRUE;
break;
}
CheckEntry = CheckEntry->ForwardLink;
}
return Swapped;
}
/**
Checks and adjusts current CPU features per dependency relationship.
@param[in] FeatureList Pointer to CPU feature list
@param[in] CurrentEntry Pointer to current checked CPU feature
@param[in] FeatureMask The feature bit mask.
@retval return Swapped info.
**/
BOOLEAN
InsertToAfterEntry (
IN LIST_ENTRY *FeatureList,
IN LIST_ENTRY *CurrentEntry,
IN UINT8 *FeatureMask
)
{
LIST_ENTRY *CheckEntry;
CPU_FEATURES_ENTRY *CheckFeature;
BOOLEAN Swapped;
Swapped = FALSE;
//
// Check all features dispatched after this entry
//
CheckEntry = GetNextNode (FeatureList, CurrentEntry);
while (!IsNull (FeatureList, CheckEntry)) {
CheckFeature = CPU_FEATURE_ENTRY_FROM_LINK (CheckEntry);
if (IsBitMaskMatchCheck (CheckFeature->FeatureMask, FeatureMask)) {
AdjustEntry (FeatureList, CheckEntry, CurrentEntry, FALSE);
Swapped = TRUE;
break;
}
CheckEntry = CheckEntry->ForwardLink;
}
return Swapped;
}
/** /**
Checks and adjusts CPU features order per dependency relationship. Checks and adjusts CPU features order per dependency relationship.
@ -128,11 +424,13 @@ CheckCpuFeaturesDependency (
CPU_FEATURES_ENTRY *CheckFeature; CPU_FEATURES_ENTRY *CheckFeature;
BOOLEAN Swapped; BOOLEAN Swapped;
LIST_ENTRY *TempEntry; LIST_ENTRY *TempEntry;
LIST_ENTRY *NextEntry;
CurrentEntry = GetFirstNode (FeatureList); CurrentEntry = GetFirstNode (FeatureList);
while (!IsNull (FeatureList, CurrentEntry)) { while (!IsNull (FeatureList, CurrentEntry)) {
Swapped = FALSE; Swapped = FALSE;
CpuFeature = CPU_FEATURE_ENTRY_FROM_LINK (CurrentEntry); CpuFeature = CPU_FEATURE_ENTRY_FROM_LINK (CurrentEntry);
NextEntry = CurrentEntry->ForwardLink;
if (CpuFeature->BeforeAll) { if (CpuFeature->BeforeAll) {
// //
// Check all features dispatched before this entry // Check all features dispatched before this entry
@ -153,6 +451,7 @@ CheckCpuFeaturesDependency (
CheckEntry = CheckEntry->ForwardLink; CheckEntry = CheckEntry->ForwardLink;
} }
if (Swapped) { if (Swapped) {
CurrentEntry = NextEntry;
continue; continue;
} }
} }
@ -179,60 +478,59 @@ CheckCpuFeaturesDependency (
CheckEntry = CheckEntry->ForwardLink; CheckEntry = CheckEntry->ForwardLink;
} }
if (Swapped) { if (Swapped) {
CurrentEntry = NextEntry;
continue; continue;
} }
} }
if (CpuFeature->BeforeFeatureBitMask != NULL) { if (CpuFeature->BeforeFeatureBitMask != NULL) {
// Swapped = InsertToBeforeEntry (FeatureList, CurrentEntry, CpuFeature->BeforeFeatureBitMask);
// Check all features dispatched before this entry
//
CheckEntry = GetFirstNode (FeatureList);
while (CheckEntry != CurrentEntry) {
CheckFeature = CPU_FEATURE_ENTRY_FROM_LINK (CheckEntry);
if (IsBitMaskMatchCheck (CheckFeature->FeatureMask, CpuFeature->BeforeFeatureBitMask)) {
//
// If there is dependency, swap them
//
RemoveEntryList (CurrentEntry);
InsertTailList (CheckEntry, CurrentEntry);
Swapped = TRUE;
break;
}
CheckEntry = CheckEntry->ForwardLink;
}
if (Swapped) { if (Swapped) {
CurrentEntry = NextEntry;
continue; continue;
} }
} }
if (CpuFeature->AfterFeatureBitMask != NULL) { if (CpuFeature->AfterFeatureBitMask != NULL) {
// Swapped = InsertToAfterEntry (FeatureList, CurrentEntry, CpuFeature->AfterFeatureBitMask);
// Check all features dispatched after this entry
//
CheckEntry = GetNextNode (FeatureList, CurrentEntry);
while (!IsNull (FeatureList, CheckEntry)) {
CheckFeature = CPU_FEATURE_ENTRY_FROM_LINK (CheckEntry);
if (IsBitMaskMatchCheck (CheckFeature->FeatureMask, CpuFeature->AfterFeatureBitMask)) {
//
// If there is dependency, swap them
//
TempEntry = GetNextNode (FeatureList, CurrentEntry);
RemoveEntryList (CurrentEntry);
InsertHeadList (CheckEntry, CurrentEntry);
CurrentEntry = TempEntry;
Swapped = TRUE;
break;
}
CheckEntry = CheckEntry->ForwardLink;
}
if (Swapped) { if (Swapped) {
CurrentEntry = NextEntry;
continue; continue;
} }
} }
//
// No swap happened, check the next feature if (CpuFeature->CoreBeforeFeatureBitMask != NULL) {
// Swapped = InsertToBeforeEntry (FeatureList, CurrentEntry, CpuFeature->CoreBeforeFeatureBitMask);
if (Swapped) {
CurrentEntry = NextEntry;
continue;
}
}
if (CpuFeature->CoreAfterFeatureBitMask != NULL) {
Swapped = InsertToAfterEntry (FeatureList, CurrentEntry, CpuFeature->CoreAfterFeatureBitMask);
if (Swapped) {
CurrentEntry = NextEntry;
continue;
}
}
if (CpuFeature->PackageBeforeFeatureBitMask != NULL) {
Swapped = InsertToBeforeEntry (FeatureList, CurrentEntry, CpuFeature->PackageBeforeFeatureBitMask);
if (Swapped) {
CurrentEntry = NextEntry;
continue;
}
}
if (CpuFeature->PackageAfterFeatureBitMask != NULL) {
Swapped = InsertToAfterEntry (FeatureList, CurrentEntry, CpuFeature->PackageAfterFeatureBitMask);
if (Swapped) {
CurrentEntry = NextEntry;
continue;
}
}
CurrentEntry = CurrentEntry->ForwardLink; CurrentEntry = CurrentEntry->ForwardLink;
} }
} }
@ -265,8 +563,8 @@ RegisterCpuFeatureWorker (
CpuFeaturesData = GetCpuFeaturesData (); CpuFeaturesData = GetCpuFeaturesData ();
if (CpuFeaturesData->FeaturesCount == 0) { if (CpuFeaturesData->FeaturesCount == 0) {
InitializeListHead (&CpuFeaturesData->FeatureList); InitializeListHead (&CpuFeaturesData->FeatureList);
InitializeSpinLock (&CpuFeaturesData->MsrLock); InitializeSpinLock (&CpuFeaturesData->CpuFlags.MemoryMappedLock);
InitializeSpinLock (&CpuFeaturesData->MemoryMappedLock); InitializeSpinLock (&CpuFeaturesData->CpuFlags.ConsoleLogLock);
CpuFeaturesData->BitMaskSize = (UINT32) BitMaskSize; CpuFeaturesData->BitMaskSize = (UINT32) BitMaskSize;
} }
ASSERT (CpuFeaturesData->BitMaskSize == BitMaskSize); ASSERT (CpuFeaturesData->BitMaskSize == BitMaskSize);
@ -328,6 +626,31 @@ RegisterCpuFeatureWorker (
} }
CpuFeatureEntry->AfterFeatureBitMask = CpuFeature->AfterFeatureBitMask; CpuFeatureEntry->AfterFeatureBitMask = CpuFeature->AfterFeatureBitMask;
} }
if (CpuFeature->CoreBeforeFeatureBitMask != NULL) {
if (CpuFeatureEntry->CoreBeforeFeatureBitMask != NULL) {
FreePool (CpuFeatureEntry->CoreBeforeFeatureBitMask);
}
CpuFeatureEntry->CoreBeforeFeatureBitMask = CpuFeature->CoreBeforeFeatureBitMask;
}
if (CpuFeature->CoreAfterFeatureBitMask != NULL) {
if (CpuFeatureEntry->CoreAfterFeatureBitMask != NULL) {
FreePool (CpuFeatureEntry->CoreAfterFeatureBitMask);
}
CpuFeatureEntry->CoreAfterFeatureBitMask = CpuFeature->CoreAfterFeatureBitMask;
}
if (CpuFeature->PackageBeforeFeatureBitMask != NULL) {
if (CpuFeatureEntry->PackageBeforeFeatureBitMask != NULL) {
FreePool (CpuFeatureEntry->PackageBeforeFeatureBitMask);
}
CpuFeatureEntry->PackageBeforeFeatureBitMask = CpuFeature->PackageBeforeFeatureBitMask;
}
if (CpuFeature->PackageAfterFeatureBitMask != NULL) {
if (CpuFeatureEntry->PackageAfterFeatureBitMask != NULL) {
FreePool (CpuFeatureEntry->PackageAfterFeatureBitMask);
}
CpuFeatureEntry->PackageAfterFeatureBitMask = CpuFeature->PackageAfterFeatureBitMask;
}
CpuFeatureEntry->BeforeAll = CpuFeature->BeforeAll; CpuFeatureEntry->BeforeAll = CpuFeature->BeforeAll;
CpuFeatureEntry->AfterAll = CpuFeature->AfterAll; CpuFeatureEntry->AfterAll = CpuFeature->AfterAll;
@ -410,6 +733,8 @@ SetCpuFeaturesBitMask (
@retval RETURN_UNSUPPORTED Registration of the CPU feature is not @retval RETURN_UNSUPPORTED Registration of the CPU feature is not
supported due to a circular dependency between supported due to a circular dependency between
BEFORE and AFTER features. BEFORE and AFTER features.
@retval RETURN_NOT_READY CPU feature PCD PcdCpuFeaturesUserConfiguration
not updated by Platform driver yet.
@note This service could be called by BSP only. @note This service could be called by BSP only.
**/ **/
@ -431,12 +756,20 @@ RegisterCpuFeature (
UINT8 *FeatureMask; UINT8 *FeatureMask;
UINT8 *BeforeFeatureBitMask; UINT8 *BeforeFeatureBitMask;
UINT8 *AfterFeatureBitMask; UINT8 *AfterFeatureBitMask;
UINT8 *CoreBeforeFeatureBitMask;
UINT8 *CoreAfterFeatureBitMask;
UINT8 *PackageBeforeFeatureBitMask;
UINT8 *PackageAfterFeatureBitMask;
BOOLEAN BeforeAll; BOOLEAN BeforeAll;
BOOLEAN AfterAll; BOOLEAN AfterAll;
FeatureMask = NULL; FeatureMask = NULL;
BeforeFeatureBitMask = NULL; BeforeFeatureBitMask = NULL;
AfterFeatureBitMask = NULL; AfterFeatureBitMask = NULL;
CoreBeforeFeatureBitMask = NULL;
CoreAfterFeatureBitMask = NULL;
PackageBeforeFeatureBitMask = NULL;
PackageAfterFeatureBitMask = NULL;
BeforeAll = FALSE; BeforeAll = FALSE;
AfterAll = FALSE; AfterAll = FALSE;
@ -449,6 +782,10 @@ RegisterCpuFeature (
!= (CPU_FEATURE_BEFORE | CPU_FEATURE_AFTER)); != (CPU_FEATURE_BEFORE | CPU_FEATURE_AFTER));
ASSERT ((Feature & (CPU_FEATURE_BEFORE_ALL | CPU_FEATURE_AFTER_ALL)) ASSERT ((Feature & (CPU_FEATURE_BEFORE_ALL | CPU_FEATURE_AFTER_ALL))
!= (CPU_FEATURE_BEFORE_ALL | CPU_FEATURE_AFTER_ALL)); != (CPU_FEATURE_BEFORE_ALL | CPU_FEATURE_AFTER_ALL));
ASSERT ((Feature & (CPU_FEATURE_CORE_BEFORE | CPU_FEATURE_CORE_AFTER))
!= (CPU_FEATURE_CORE_BEFORE | CPU_FEATURE_CORE_AFTER));
ASSERT ((Feature & (CPU_FEATURE_PACKAGE_BEFORE | CPU_FEATURE_PACKAGE_AFTER))
!= (CPU_FEATURE_PACKAGE_BEFORE | CPU_FEATURE_PACKAGE_AFTER));
if (Feature < CPU_FEATURE_BEFORE) { if (Feature < CPU_FEATURE_BEFORE) {
BeforeAll = ((Feature & CPU_FEATURE_BEFORE_ALL) != 0) ? TRUE : FALSE; BeforeAll = ((Feature & CPU_FEATURE_BEFORE_ALL) != 0) ? TRUE : FALSE;
AfterAll = ((Feature & CPU_FEATURE_AFTER_ALL) != 0) ? TRUE : FALSE; AfterAll = ((Feature & CPU_FEATURE_AFTER_ALL) != 0) ? TRUE : FALSE;
@ -459,6 +796,14 @@ RegisterCpuFeature (
SetCpuFeaturesBitMask (&BeforeFeatureBitMask, Feature & ~CPU_FEATURE_BEFORE, BitMaskSize); SetCpuFeaturesBitMask (&BeforeFeatureBitMask, Feature & ~CPU_FEATURE_BEFORE, BitMaskSize);
} else if ((Feature & CPU_FEATURE_AFTER) != 0) { } else if ((Feature & CPU_FEATURE_AFTER) != 0) {
SetCpuFeaturesBitMask (&AfterFeatureBitMask, Feature & ~CPU_FEATURE_AFTER, BitMaskSize); SetCpuFeaturesBitMask (&AfterFeatureBitMask, Feature & ~CPU_FEATURE_AFTER, BitMaskSize);
} else if ((Feature & CPU_FEATURE_CORE_BEFORE) != 0) {
SetCpuFeaturesBitMask (&CoreBeforeFeatureBitMask, Feature & ~CPU_FEATURE_CORE_BEFORE, BitMaskSize);
} else if ((Feature & CPU_FEATURE_CORE_AFTER) != 0) {
SetCpuFeaturesBitMask (&CoreAfterFeatureBitMask, Feature & ~CPU_FEATURE_CORE_AFTER, BitMaskSize);
} else if ((Feature & CPU_FEATURE_PACKAGE_BEFORE) != 0) {
SetCpuFeaturesBitMask (&PackageBeforeFeatureBitMask, Feature & ~CPU_FEATURE_PACKAGE_BEFORE, BitMaskSize);
} else if ((Feature & CPU_FEATURE_PACKAGE_AFTER) != 0) {
SetCpuFeaturesBitMask (&PackageAfterFeatureBitMask, Feature & ~CPU_FEATURE_PACKAGE_AFTER, BitMaskSize);
} }
Feature = VA_ARG (Marker, UINT32); Feature = VA_ARG (Marker, UINT32);
} }
@ -470,6 +815,10 @@ RegisterCpuFeature (
CpuFeature->FeatureMask = FeatureMask; CpuFeature->FeatureMask = FeatureMask;
CpuFeature->BeforeFeatureBitMask = BeforeFeatureBitMask; CpuFeature->BeforeFeatureBitMask = BeforeFeatureBitMask;
CpuFeature->AfterFeatureBitMask = AfterFeatureBitMask; CpuFeature->AfterFeatureBitMask = AfterFeatureBitMask;
CpuFeature->CoreBeforeFeatureBitMask = CoreBeforeFeatureBitMask;
CpuFeature->CoreAfterFeatureBitMask = CoreAfterFeatureBitMask;
CpuFeature->PackageBeforeFeatureBitMask = PackageBeforeFeatureBitMask;
CpuFeature->PackageAfterFeatureBitMask = PackageAfterFeatureBitMask;
CpuFeature->BeforeAll = BeforeAll; CpuFeature->BeforeAll = BeforeAll;
CpuFeature->AfterAll = AfterAll; CpuFeature->AfterAll = AfterAll;
CpuFeature->GetConfigDataFunc = GetConfigDataFunc; CpuFeature->GetConfigDataFunc = GetConfigDataFunc;
@ -489,13 +838,12 @@ RegisterCpuFeature (
} }
/** /**
Allocates boot service data to save ACPI_CPU_DATA. Return ACPI_CPU_DATA data.
@return Pointer to allocated ACPI_CPU_DATA. @return Pointer to ACPI_CPU_DATA data.
**/ **/
STATIC
ACPI_CPU_DATA * ACPI_CPU_DATA *
AllocateAcpiCpuData ( GetAcpiCpuData (
VOID VOID
) )
{ {
@ -508,9 +856,20 @@ AllocateAcpiCpuData (
UINTN Index; UINTN Index;
EFI_PROCESSOR_INFORMATION ProcessorInfoBuffer; EFI_PROCESSOR_INFORMATION ProcessorInfoBuffer;
AcpiCpuData = (ACPI_CPU_DATA *) (UINTN) PcdGet64 (PcdCpuS3DataAddress);
if (AcpiCpuData != NULL) {
return AcpiCpuData;
}
AcpiCpuData = AllocatePages (EFI_SIZE_TO_PAGES (sizeof (ACPI_CPU_DATA))); AcpiCpuData = AllocatePages (EFI_SIZE_TO_PAGES (sizeof (ACPI_CPU_DATA)));
ASSERT (AcpiCpuData != NULL); ASSERT (AcpiCpuData != NULL);
//
// Set PcdCpuS3DataAddress to the base address of the ACPI_CPU_DATA structure
//
Status = PcdSet64S (PcdCpuS3DataAddress, (UINT64)(UINTN)AcpiCpuData);
ASSERT_EFI_ERROR (Status);
GetNumberOfProcessor (&NumberOfCpus, &NumberOfEnabledProcessors); GetNumberOfProcessor (&NumberOfCpus, &NumberOfEnabledProcessors);
AcpiCpuData->NumberOfCpus = (UINT32)NumberOfCpus; AcpiCpuData->NumberOfCpus = (UINT32)NumberOfCpus;
@ -606,7 +965,6 @@ CpuRegisterTableWriteWorker (
IN UINT64 Value IN UINT64 Value
) )
{ {
EFI_STATUS Status;
CPU_FEATURES_DATA *CpuFeaturesData; CPU_FEATURES_DATA *CpuFeaturesData;
ACPI_CPU_DATA *AcpiCpuData; ACPI_CPU_DATA *AcpiCpuData;
CPU_REGISTER_TABLE *RegisterTable; CPU_REGISTER_TABLE *RegisterTable;
@ -614,17 +972,8 @@ CpuRegisterTableWriteWorker (
CpuFeaturesData = GetCpuFeaturesData (); CpuFeaturesData = GetCpuFeaturesData ();
if (CpuFeaturesData->RegisterTable == NULL) { if (CpuFeaturesData->RegisterTable == NULL) {
AcpiCpuData = (ACPI_CPU_DATA *) (UINTN) PcdGet64 (PcdCpuS3DataAddress); AcpiCpuData = GetAcpiCpuData ();
if (AcpiCpuData == NULL) { ASSERT ((AcpiCpuData != NULL) && (AcpiCpuData->RegisterTable != 0));
AcpiCpuData = AllocateAcpiCpuData ();
ASSERT (AcpiCpuData != NULL);
//
// Set PcdCpuS3DataAddress to the base address of the ACPI_CPU_DATA structure
//
Status = PcdSet64S (PcdCpuS3DataAddress, (UINT64)(UINTN)AcpiCpuData);
ASSERT_EFI_ERROR (Status);
}
ASSERT (AcpiCpuData->RegisterTable != 0);
CpuFeaturesData->RegisterTable = (CPU_REGISTER_TABLE *) (UINTN) AcpiCpuData->RegisterTable; CpuFeaturesData->RegisterTable = (CPU_REGISTER_TABLE *) (UINTN) AcpiCpuData->RegisterTable;
CpuFeaturesData->PreSmmRegisterTable = (CPU_REGISTER_TABLE *) (UINTN) AcpiCpuData->PreSmmInitRegisterTable; CpuFeaturesData->PreSmmRegisterTable = (CPU_REGISTER_TABLE *) (UINTN) AcpiCpuData->PreSmmInitRegisterTable;
} }