Commit 48485eee authored by Jeffrey Lee's avatar Jeffrey Lee
Browse files

Add new ARMops. Add macros which map the ARMv7/v8 cache/TLB maintenance...

Add new ARMops. Add macros which map the ARMv7/v8 cache/TLB maintenance mnemonics (as featured in recent ARM ARMs) to MCR ops.

Detail:
  - Docs/HAL/ARMop_API - Document the new ARMops. These ops are intended to help with future work (DMA without OS_Memory 0 "make temp uncacheable", and minimising cache maintenance when unmapping pages) and aren't in use just yet.
  - hdr/Copro15ops - Add new macros for ARMv7+ which map the mnemonics seen in recent ARM ARMs to the corresponding MCR ops. This should make things easier when cross-referencing docs and reduce the risk of typos.
  - hdr/KernelWS - Shuffle kernel workspace a bit to make room for the new ARMops
  - hdr/OSMisc - Expose new ARMops via OS_MMUControl 2
  - s/ARMops - Implement the new ARMops. Change the ARMv7+ ARMops to use the new mnemonic macros. Also get rid of myDSB / myISB usage from ARMv7+ code paths; use DSB/ISB/etc. directly to ensure correct behaviour
  - s/HAL - Mnemonic + ISB/DSB updates. Change software RAM clear to do 16 bytes at a time for kernel workspace instead of 32 to allow the kernel workspace tweaks to work.
Admin:
  Binary diff shows that mnemonics map to the original MCR ops correctly
  Note: Raspberry Pi builds will now emit lots of warnings due to increased DSB/ISB instruction use. However it should be safe to ignore these as they should only be present in v7+ code paths.
  Note: New ARMops haven't been tested yet, will be disabled (or at least hidden from user code) in a future checkin


Version 5.68. Tagged as 'Kernel-5_68'
parent 4a6150dc
......@@ -238,6 +238,25 @@ involved in any currently active interrupts. In other words, it is expected
and desirable that interrupts remain enabled during any extended clean
operation, in order to avoid impact on interrupt latency.
-- Cache_CleanRange
The cache or caches are to be cleaned for (at least) the given range.
Invalidation is not required.
entry: r0 = logical address of start of range
r1 = logical address of end of range (exclusive)
Note that r0 and r1 are aligned on cache line boundaries
exit: -
Note that any write buffer draining should also be performed by this
operation, so that memory is fully updated with respect to any writeaback
data.
The OS only expects the invalidation to be with respect to instructions/data
that are not involved in any currently active interrupts. In other words, it
is expected and desirable that interrupts remain enabled during any extended
clean operation, in order to avoid impact on interrupt latency.
-- Cache_InvalidateAll
The cache or caches are to be globally invalidated. Cleaning of any writeback
......@@ -250,6 +269,33 @@ This call is only required for special restart use, since it implies that
any writeback data are either irrelevant or not valid. It should be a very
simple operation on all ARMs.
-- Cache_InvalidateRange
The cache or caches are to be invalidated for the given range. Cleaning of any
writeback data is not to be performed.
entry: r0 = logical address of start of range
r1 = logical address of end of range (exclusive)
Note that r0 and r1 are aligned on cache line boundaries
exit: -
This call is intended for use in situations where another bus master (e.g. DMA)
has written to an area of cacheable memory, and stale data is to be cleared
from the ARM's cache so that software can see the new values.
It is important that only the indicated region is invalidated - neighbouring
cache lines may contain valid data that has not yet been written back. Because
software should not have been writing to the DMA buffer while the DMA was in
progress, it is permissible for this operation to both clean and invalidate.
E.g. if a write-back cache is in use, it would be incorrect to promote a large
invalidate to a global invalidate, and so an implementation could instead
perform a global clean+invalidate.
The OS only expects the invalidation to be with respect to instructions/data
that are not involved in any currently active interrupts. In other words, it
is expected and desirable that interrupts remain enabled during any extended
clean operation, in order to avoid impact on interrupt latency.
-- Cache_RangeThreshold
Return a threshold value for an address range, above which it is advisable
......@@ -291,6 +337,30 @@ Return information about a given cache level
For unified caches, r1-r2 will match r3-r4. This call mainly exists for the
benefit of OS_PlatformFeatures 33.
-- ICache_InvalidateAll
The instruction cache is to be globally invalidated.
entry: -
exit: -
This operation should only act on instruction caches - not data or unified
caches. If only data or unified caches are present then the operation can be
implemented as a NOP.
-- ICache_InvalidateRange
The instruction cache is to be invalidated for the given range.
entry: r0 = logical address of start of range
r1 = logical address of end of range (exclusive)
Note that r0 and r1 are aligned on cache line boundaries
exit: -
This operation should only act on instruction caches - not data or unified
caches. If only data or unified caches are present then the operation can be
implemented as a NOP.
Memory barrier ARMops
=====================
......
......@@ -11,13 +11,13 @@
GBLS Module_HelpVersion
GBLS Module_ComponentName
GBLS Module_ComponentPath
Module_MajorVersion SETS "5.67"
Module_Version SETA 567
Module_MajorVersion SETS "5.68"
Module_Version SETA 568
Module_MinorVersion SETS ""
Module_Date SETS "13 Dec 2016"
Module_ApplicationDate SETS "13-Dec-16"
Module_ComponentName SETS "Kernel"
Module_ComponentPath SETS "castle/RiscOS/Sources/Kernel"
Module_FullVersion SETS "5.67"
Module_HelpVersion SETS "5.67 (13 Dec 2016)"
Module_FullVersion SETS "5.68"
Module_HelpVersion SETS "5.68 (13 Dec 2016)"
END
/* (5.67)
/* (5.68)
*
* This file is automatically maintained by srccommit, do not edit manually.
* Last processed by srccommit version: 1.1.
*
*/
#define Module_MajorVersion_CMHG 5.67
#define Module_MajorVersion_CMHG 5.68
#define Module_MinorVersion_CMHG
#define Module_Date_CMHG 13 Dec 2016
#define Module_MajorVersion "5.67"
#define Module_Version 567
#define Module_MajorVersion "5.68"
#define Module_Version 568
#define Module_MinorVersion ""
#define Module_Date "13 Dec 2016"
......@@ -18,6 +18,6 @@
#define Module_ComponentName "Kernel"
#define Module_ComponentPath "castle/RiscOS/Sources/Kernel"
#define Module_FullVersion "5.67"
#define Module_HelpVersion "5.67 (13 Dec 2016)"
#define Module_LibraryVersionInfo "5:67"
#define Module_FullVersion "5.68"
#define Module_HelpVersion "5.68 (13 Dec 2016)"
#define Module_LibraryVersionInfo "5:68"
......@@ -166,48 +166,6 @@ C15 CN 15
TEQ$cond $tmp,#1
MEND
[ :LNOT: HAL
;flush whole TLB (both data and instruction for StrongARM)
;trashes $temp
MACRO
ARM_flush_TLB $temp
ARM_read_ID $temp
AND $temp,$temp,#&F000
CMP $temp,#&8000 ;ARM 8?
CMPNE $temp,#&A000 ;or StrongARM?
MCRNE ARM_config_cp,0,R0,ARM67_TLBflush_reg,C0,0
MCREQ ARM_config_cp,0,R0,ARM8A_TLB_reg,C7,0
MEND
;flush whole cache (both data and instruction for StrongARM),
;without worrying about any cache cleaning
;trashes $temp
MACRO
ARM_flush_cache $temp
ARM_read_ID $temp
AND $temp,$temp,#&F000
CMP $temp,#&8000 ;ARM 8?
CMPNE $temp,#&A000 ;or StrongARM?
MCRNE ARM_config_cp,0,R0,ARM67_cacheflush_reg,C0,0
MCREQ ARM_config_cp,0,R0,ARM8A_cache_reg,C7,0
MEND
;flush whole TLB and cache (both data and instruction for StrongARM),
;without worrying about any cache cleaning
;trashes $temp
MACRO
ARM_flush_cacheandTLB $temp
ARM_read_ID $temp
AND $temp,$temp,#&F000
CMP $temp,#&8000 ;ARM 8?
CMPNE $temp,#&A000 ;or StrongARM?
MCRNE ARM_config_cp,0,R0,ARM67_cacheflush_reg,C0,0
MCRNE ARM_config_cp,0,R0,ARM67_TLBflush_reg,C0,0
MCREQ ARM_config_cp,0,R0,ARM8A_cache_reg,C7,0
MCREQ ARM_config_cp,0,R0,ARM8A_TLB_reg,C7,0
MEND
]
;
; -------------- ARM 6,7 only --------------------------------------------
;
......@@ -584,7 +542,7 @@ C15 CN 15
]
MEND
; Data Synchronisation Barrier - aka drain write buffer/data write barrier. Stalls pipeline until all preceeding memory accesses (including cache/TLB/BTC ops complete.
; Data Synchronisation Barrier - aka drain write buffer/data write barrier. Stalls pipeline until all preceeding memory accesses (including cache/TLB/BTC ops) complete.
MACRO
myDSB $cond,$temp,$option,$quick
[ NoARMv6
......@@ -686,5 +644,139 @@ C15 CN 15
]
MEND
[ SupportARMv7
; ARMv7/v8 cache, TLB and branch predictor maintenance operations
; These are named directly after the acronyms that appear in recent ARM ARM
; revisions (and for AArch64, the assembler will even recognise them as
; mnemonics)
; Branch predictor invalidate all (local core only)
MACRO
BPIALL$cond
MCR$cond p15,0,a1,c7,c5,6
MEND
; Branch predictor invalidate all (inner shareable - MP extensions)
MACRO
BPIALLIS$cond
MCR$cond p15,0,a1,c7,c1,6
MEND
; Branch predictor invalidate by MVA
MACRO
BPIMVA$cond $reg
MCR$cond p15,0,$reg,c7,c5,7
MEND
; Data cache clean & invalidate by MVA to point of coherency
MACRO
DCCIMVAC$cond $reg
MCR$cond p15,0,$reg,c7,c14,1
MEND
; Data cache clean and invalidate by set/way (local core only)
MACRO
DCCISW$cond $reg
MCR$cond p15,0,$reg,c7,c14,2
MEND
; Data cache clean by MVA to point of coherency
MACRO
DCCMVAC$cond $reg
MCR$cond p15,0,$reg,c7,c10,1
MEND
; Data cache clean by MVA to point of unification
MACRO
DCCMVAU$cond $reg
MCR$cond p15,0,$reg,c7,c11,1
MEND
; Data cache clean by set/way (local core only)
MACRO
DCCSW$cond $reg
MCR$cond p15,0,$reg,c7,c10,2
MEND
; Data cache invalidate by MVA to point of coherency
MACRO
DCIMVAC$cond $reg
MCR$cond p15,0,$reg,c7,c6,1
MEND
; Data cache invalidate by set/way (local core only)
MACRO
DCISW$cond $reg
MCR$cond p15,0,$reg,c7,c6,2
MEND
; Instruction cache + branch predictor invalidate all to point of unification (local core only)
MACRO
ICIALLU$cond
MCR$cond p15,0,a1,c7,c5,0
MEND
; Instruction cache + branch predictor invalidate all to point of unification (inner shareable - MP extensions)
MACRO
ICIALLUIS$cond
MCR$cond p15,0,a1,c7,c1,0
MEND
; Instruction cache invalidate by MVA to point of unification
MACRO
ICIMVAU$cond $reg
MCR$cond p15,0,$reg,c7,c5,1
MEND
; Invalidate entire TLB (local core only)
MACRO
TLBIALL$cond
MCR$cond p15,0,a1,c8,c7,0
MEND
; Invalidate entire TLB (inner shareable - MP extensions)
MACRO
TLBIALLIS$cond
MCR$cond p15,0,a1,c8,c3,0
MEND
; Invalidate TLB by ASID (local core only)
MACRO
TLBIASID$cond $reg
MCR$cond p15,0,$reg,c8,c7,2
MEND
; Invalidate TLB by ASID (inner shareable - MP extensions)
MACRO
TLBIASIDIS$cond $reg
MCR$cond p15,0,$reg,c8,c3,2
MEND
; Invalidate TLB by MVA, all ASID or global (local core only - MP extensions)
MACRO
TLBIMVAA$cond $reg
MCR$cond p15,0,$reg,c8,c7,3
MEND
; Invalidate TLB by MVA, all ASID or global (inner shareable - MP extensions)
MACRO
TLBIMVAAIS$cond $reg
MCR$cond p15,0,$reg,c8,c3,3
MEND
; Invalidate TLB by MVA, indicated ASID or global (local core only)
MACRO
TLBIMVA$cond $reg
MCR$cond p15,0,$reg,c8,c7,1
MEND
; Invalidate TLB by MVA, indicated ASID or global (inner shareable - MP extensions)
MACRO
TLBIMVAIS$cond $reg
MCR$cond p15,0,$reg,c8,c3,1
MEND
] ; SupportARMv7
END
......@@ -1074,21 +1074,27 @@ EvtHan # 4
; (256 words) which is no longer adequate, so we can reuse it
JordanWS # 0
# 3*4 ; SPARE
Serv_SysChains # 4 ;anchor for block handling 'system' service numbers, in range 1 to 255
Serv_UsrChains # 4 ;anchor for block handling 'user' service numbers, > 255
Serv_AwkwardChain # 4 ;anchor for chain handling non-compliant modules (no service table)
[ :DEF: ShowWS
! 0, "Serv_SysChains at ":CC::STR:(Serv_SysChains)
! 0, "Serv_UsrChains at ":CC::STR:(Serv_UsrChains)
! 0, "Serv_AwkwardChain at ":CC::STR:(Serv_AwkwardChain)
]
DAList # 4 ; Pointer to first node on dynamic area list
AlignSpace 16
AMBControl_ws # 4 ; workspace anchor word for AMBControl
DynArea_ws # 4 ; workspace anchor word for data structures to accelerate OS SWIs for dynamic areas
SyncCodeA_sema # 1 ; re-entrancy semaphore for SynchroniseCodeAreas (full address space version)
[ :DEF: ShowWS
! 0, "AMBControl_ws at ":CC::STR:(AMBControl_ws)
! 0, "DynArea_ws at ":CC::STR:(DynArea_ws)
! 0, "SyncCodeA_sema (byte) at ":CC::STR:(SyncCodeA_sema)
]
AlignSpace 4
Oscli_CmdHashSum # 4 ;for hashed command lookup
Oscli_CmdHashLists # 4 ;anchor for hashed command lists structure
[ :DEF: ShowWS
......@@ -1096,19 +1102,10 @@ Oscli_CmdHashLists # 4 ;anchor for hashed command lists structure
! 0, "Oscli_CmdHashLists at ":CC::STR:(Oscli_CmdHashLists)
]
Serv_SysChains # 4 ;anchor for block handling 'system' service numbers, in range 1 to 255
Serv_UsrChains # 4 ;anchor for block handling 'user' service numbers, > 255
Serv_AwkwardChain # 4 ;anchor for chain handling non-compliant modules (no service table)
[ :DEF: ShowWS
! 0, "Serv_SysChains at ":CC::STR:(Serv_SysChains)
! 0, "Serv_UsrChains at ":CC::STR:(Serv_UsrChains)
! 0, "Serv_AwkwardChain at ":CC::STR:(Serv_AwkwardChain)
]
AlignSpace 32 ; skipped bit must start on 32-byte boundary (due to speedup)
AlignSpace 16 ; skipped bit must start on 16-byte boundary (ClearPhysRAM does 4 words at a time for skipped areas)
SkippedTables # 0
PhysRamTable # 0 ; Pairs of words (physaddr, size+flags)
; indicating RAM present in machine
; Unused entries have size of zero
......@@ -1173,9 +1170,13 @@ MMU_PPLAccess # 4
Proc_Cache_CleanInvalidateAll # 4
Proc_Cache_CleanInvalidateRange # 4
Proc_Cache_CleanAll # 4
Proc_Cache_CleanRange # 4
Proc_Cache_InvalidateAll # 4
Proc_Cache_InvalidateRange # 4
Proc_Cache_RangeThreshold # 4
Proc_Cache_Examine # 4
Proc_ICache_InvalidateAll # 4
Proc_ICache_InvalidateRange # 4
Proc_TLB_InvalidateAll # 4
Proc_TLB_InvalidateEntry # 4
Proc_DSB_ReadWrite # 4
......@@ -1229,7 +1230,7 @@ DebuggerSpace_Size * &1000
IICBus_Count * 5 ; 5 buses is enough for all current machines
IICBus_Base # IICBus_Size*IICBus_Count
AlignSpace 32 ; skipped bit must end on 32-byte boundary (due to speedup)
AlignSpace 16 ; skipped bit must end on 16-byte boundary (ClearPhysRAM does 4 words at a time for skipped areas)
SkippedTablesEnd # 0
; NVRAM support
......
......@@ -146,6 +146,10 @@ ARMop_DMB_ReadWrite # 1 ; 18
ARMop_DMB_Write # 1 ; 19
ARMop_DMB_Read # 1 ; 20
ARMop_Cache_CleanInvalidateRange # 1 ; 21
ARMop_Cache_CleanRange # 1 ; 22
ARMop_Cache_InvalidateRange # 1 ; 23
ARMop_ICache_InvalidateAll # 1 ; 24
ARMop_ICache_InvalidateRange # 1 ; 25
ARMop_Max # 0
; SeriousErrorV reason codes (R2)
......
This diff is collapsed.
......@@ -1397,7 +1397,7 @@ HAL_InvalidateCache_ARMvF
CMP r9, #2
BLT %FT40 ; no cache or only instruction cache at this level
MCR p15, 2, r11, c0, c0, 0 ; write CSSELR from r11
myISB ,r9
ISB
MRC p15, 1, r9, c0, c0, 0 ; read current CSSIDR to r9
AND r10, r9, #CCSIDR_LineSize_mask ; extract the line length field
ADD r10, r10, #4 ; add 4 for the line length offset (log2 16 bytes)
......@@ -1411,12 +1411,12 @@ HAL_InvalidateCache_ARMvF
30 ; Loop3
ORR r14, r11, r8, LSL r13 ; factor in the way number and cache number into r14
ORR r14, r14, r9, LSL r10 ; factor in the index number
MCR p15, 0, r14, c7, c6, 2 ; Invalidate
DCISW r14 ; Invalidate
SUBS r9, r9, #1 ; decrement the index
BGE %BT30
SUBS r8, r8, #1 ; decrement the way number
BGE %BT20
myDSB ,r8 ; Cortex-A7 errata 814220: DSB required when changing cache levels when using set/way operations. This also counts as our end-of-maintenance DSB.
DSB ; Cortex-A7 errata 814220: DSB required when changing cache levels when using set/way operations. This also counts as our end-of-maintenance DSB.
MRC p15, 1, r8, c0, c0, 1
40 ; Skip
ADD r11, r11, #2
......@@ -1982,7 +1982,6 @@ ClearPhysRAM ROUT
TEQ r0, r1
TEQNE r0, r5
STMNEIA r0!, {r8-r11}
STMNEIA r0!, {r8-r11}
BNE %BT10
TEQ r0, r1
BEQ %FT20
......@@ -2117,8 +2116,9 @@ CPR_skipped
MACRO
MakeSkipTable $addr, $size
ASSERT ($addr :AND: 31) = 0
ASSERT ($size :AND: 31) = 0
ASSERT ($addr :AND: 15) = 0
ASSERT ($size :AND: 15) = 0
ASSERT ($addr-ZeroPage) < 16*1024
& $addr, $size
MEND
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment