The helper that updates live page table entries writes a zero entry,
invalidates the covered address range from the TLBs, and finally writes
the actual entry. This ensures that no TLB conflicts can occur.
Writing the final entry needs to complete before any translations can be
performed, as otherwise, the zero entry, which describes an invalid
translation, may be observed by the page table walker, resulting in a
translation fault. For this reason, the final write is followed by a DSB
barrier instruction.
However, this barrier will not stall the pipeline, and instruction
fetches may still hit this invalid translation, as has been observed and
reported by Oliver. To ensure that the new translation is fully active
before returning from this helper, we have to insert an ISB barrier as
well.
Reported-by: Oliver Steffen <osteffen@redhat.com>
Tested-by: Oliver Steffen <osteffen@redhat.com>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
Acked-by: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
As the ASM_FUNC() macro performs a section switch, the preceding
.balign directive applies the alignment constraint to the current
location in the previous section. As the linker may not merge the
sections in-order, ArmReplaceLiveTranslationEntry() may be left
unaligned.
Replace the explicit invocation of .balign with the ASM_FUNC_ALIGN()
macro, which guarantees the alignment constraint is applied correctly.
To make sure related issues are reliably caught in the future, align the
end of the function before checking the total occupied size. This
ensures crossing a 0x200 boundary will cause a compilation error.
Signed-off-by: Marvin Häuser <mhaeuser@posteo.de>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
When updating a page table descriptor in a way that requires break
before make, we temporarily disable the MMU to ensure that we don't
unmap the memory region that the code itself is executing from.
However, this is a condition we can check in a straight-forward manner,
and if the regions are disjoint, we don't have to bother with the MMU
controls, and we can just perform an ordinary break before make.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Leif Lindholm <quic_llindhol@quicinc.com>
Currently, we always invalidate the TLBs entirely after making
any modification to the page tables. Now that we have introduced
strict memory permissions in quite a number of places, such
modifications occur much more often, and it is better for performance
to flush only those TLB entries that are actually affected by
the changes.
At the same time, relax some system wide data synchronization barriers
to non-shared. When running in UEFI, we don't share virtual address
translations with other masters, unless we are running under virt, but
in that case, the host will upgrade them as appropriate (by setting
an override at EL2)
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Annotate functions with ASM_FUNC() so that they are emitted into
separate sections.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
The function ArmReplaceLiveTranslationEntry() has been moved to
ArmMmuLib, so remove the old implementation from ArmLib.
Note that the new implementation was not exported from the object file,
and so references to it were satisfied by the old version residing in
ArmLib. Since we are removing that one, we need to export the new one
at the same time to prevent the linker from bailing with undefined
reference errors.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
This base library encapsulates the MMU manipulation routines that have been
factored out of ArmLib. The functionality covers initial creation of the 1:1
mapping in the page tables, and remapping regions to change permissions or
cacheability attributes.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Ard Biesheuvel
Marvin Häuser
Michael D Kinney
Ard Biesheuvel