Commit 2704c756 authored by Jeffrey Lee's avatar Jeffrey Lee
Browse files

Place restrictions on the use of cacheable doubly-mapped DAs

Detail:
  The kernel has always allowed software to create cacheable doubly-mapped DAs, despite the fact that the VIVT caches used on ARMv5 and below would have no way of keeping both of the mappings coherent
  This change places restrictions the following restrictions on doubly-mapped areas, to ensure that cache settings which can't be supported by the cache architecture of the CPU can't be selected:
  * On ARMv6 and below, cacheable doubly-mapped areas aren't supported.
    * Although ARMv6 has VIPT data caches, it's also subject to page colouring constraints which would require us to force the DA size to be a multiple of 16k. So for now keep things simple and disallow cacheable doubly-mapped areas on ARMv6.
  * On ARMv7 and above, cacheable doubly-mapped areas are allowed, but only if they are marked non-executable
    * The blocker to allowing executable cacheable doubly-mapped areas are the VIPT instruction caches; OS_SynchroniseCodeAreas (or callers of it) would need to know that a doubly-mapped area is in use so that they can flush both mappings from the I-cache. Although some chips do have PIPT instruction caches, again it isn't really worth supporting executable cacheable doubly-mapped areas at the moment.
  These changes also allow us to get rid of the expensive 'sledgehammer' logic when dealing with doubly-mapped areas
  File changes:
  - s/ARM600, s/VMSAv6 - Remove the sledgehammer logic, only perform cache/TLB maintenance for the required areas
  - s/ChangeDyn - Implement the required checks
  - s/MemMap2 - Move some cache maintenance logic into RemoveCacheabilityR0ByMinusR2, which previously would have had to be performed by the caller due to the sledgehammer paranoia
Admin:
  Cacheable doubly-mapped DAs tested on iMx6 (tried making screen memory write-through cacheable; decent performance gain seen)
  Note OS_Memory 0 "make temporarily uncacheable" doesn't work on doubly-mapped areas, so cacheable doubly-mapped areas are not yet safe for general DMA


Version 5.69. Tagged as 'Kernel-5_69'
parent 48485eee
......@@ -11,13 +11,13 @@
GBLS Module_HelpVersion
GBLS Module_ComponentName
GBLS Module_ComponentPath
Module_MajorVersion SETS "5.68"
Module_Version SETA 568
Module_MajorVersion SETS "5.69"
Module_Version SETA 569
Module_MinorVersion SETS ""
Module_Date SETS "13 Dec 2016"
Module_ApplicationDate SETS "13-Dec-16"
Module_ComponentName SETS "Kernel"
Module_ComponentPath SETS "castle/RiscOS/Sources/Kernel"
Module_FullVersion SETS "5.68"
Module_HelpVersion SETS "5.68 (13 Dec 2016)"
Module_FullVersion SETS "5.69"
Module_HelpVersion SETS "5.69 (13 Dec 2016)"
END
/* (5.68)
/* (5.69)
*
* This file is automatically maintained by srccommit, do not edit manually.
* Last processed by srccommit version: 1.1.
*
*/
#define Module_MajorVersion_CMHG 5.68
#define Module_MajorVersion_CMHG 5.69
#define Module_MinorVersion_CMHG
#define Module_Date_CMHG 13 Dec 2016
#define Module_MajorVersion "5.68"
#define Module_Version 568
#define Module_MajorVersion "5.69"
#define Module_Version 569
#define Module_MinorVersion ""
#define Module_Date "13 Dec 2016"
......@@ -18,6 +18,6 @@
#define Module_ComponentName "Kernel"
#define Module_ComponentPath "castle/RiscOS/Sources/Kernel"
#define Module_FullVersion "5.68"
#define Module_HelpVersion "5.68 (13 Dec 2016)"
#define Module_LibraryVersionInfo "5:68"
#define Module_FullVersion "5.69"
#define Module_HelpVersion "5.69 (13 Dec 2016)"
#define Module_LibraryVersionInfo "5:69"
......@@ -167,10 +167,7 @@ BangL2PT ; internal entry point used only
MOV r6, r0
TST r11, #PageFlags_Unsafe
BNE %FT30
TST r11, #DynAreaFlags_DoublyMapped
BNE BangL2PT_sledgehammer ;if doubly mapped, don't try to be clever
BNE BangL2PT_unsafe
;In order to safely map out a cacheable page and remove it from the
;cache, we need to perform the following process:
......@@ -190,7 +187,16 @@ BangL2PT ; internal entry point used only
LDR lr, [r1, r3, LSR #10] ;get current L2PT entry
BIC lr, lr, #TempUncache_L2PTMask ;remove current attributes
ORR lr, lr, r0
STR lr, [r1, r3, LSR #10] ;Make uncacheable
STR lr, [r1, r3, LSR #10]! ;Make uncacheable
TST r11, #DynAreaFlags_DoublyMapped
BEQ %FT19
STR lr, [r1, r9, LSR #10] ;Update 2nd mapping too if required
ADD r0, r3, r9
ARMop MMU_ChangingUncachedEntry,,, r4 ; TLB flush
ADD r0, r3, r9
ADD r1, r0, #4096
ARMop Cache_CleanInvalidateRange,,, r4 ; Cache flush
19
MOV r0, r3
ARMop MMU_ChangingUncachedEntry,,, r4 ; TLB flush
MOV r0, r3
......@@ -198,28 +204,23 @@ BangL2PT ; internal entry point used only
ARMop Cache_CleanInvalidateRange,,, r4 ; Cache flush
LDR r1, =L2PT
20 STR r6, [r1, r3, LSR #10] ;update L2PT entry
20 STR r6, [r1, r3, LSR #10]! ;update L2PT entry
TST r11, #DynAreaFlags_DoublyMapped
BEQ %FT21
STR r6, [r1, r9, LSR #10] ;Update 2nd mapping
MOV r0, r3
ARMop MMU_ChangingUncachedEntry,,, r4 ; TLB flush for 1st mapping
ADD r3, r3, r9 ;restore r3 back to 2nd copy
21
Pull "lr"
MOV r0, r3
ARMop MMU_ChangingUncachedEntry,,tailcall,r4
BangL2PT_sledgehammer
;sledgehammer is super cautious and does cache/TLB coherency on a global basis
;should only be used for awkward cases
;
TEQ r6, #0 ;EQ if mapping out
TSTEQ r11, #DynAreaFlags_NotCacheable ;EQ if also cacheable (overcautious for temp uncache+illegal PCB combos)
ADR lr, %FT30
LDR r4, =ZeroPage
ARMop MMU_Changing, EQ, tailcall, r4
ARMop MMU_ChangingUncached, NE, tailcall, r4
30 STR r6, [r1, r3, LSR #10]! ; update level 2 page table (and update pointer so we can use bank-to-bank offset
BangL2PT_unsafe
STR r6, [r1, r3, LSR #10]! ; update level 2 page table (and update pointer so we can use bank-to-bank offset
TST r11, #DynAreaFlags_DoublyMapped ; if area doubly mapped
STRNE r6, [r1, r9, LSR #10] ; then store entry for 2nd copy as well
ADDNE r3, r3, r9 ; and point logical address back at 2nd copy
Pull "pc"
[ ARM6support
......
......@@ -833,6 +833,28 @@ DynArea_ErrorTranslateAndExit
BEQ DynArea_ErrorTranslateAndExit
21
; Check cacheable doubly-mapped area restrictions
; * On ARMv5 and below we disallow cacheable doubly-mapped areas outright, because the virtually tagged caches simply can't deal with them (at least when it comes to writes)
; * ARMv6 can support cacheable doubly-mapped areas, but only if we comply with the page colouring restrictions, which is something we currently don't do. So disallow there as well.
; * ARMv7+ doesn't have page colouring restrictions, but due to the potential of virtually tagged instruction caches (which would complicate OS_SynchroniseCodeAreas), for simplicity we only allow the area to be cacheable if it's non-executable
AND r11, r4, #DynAreaFlags_DoublyMapped+DynAreaFlags_NotCacheable
TEQ r11, #DynAreaFlags_DoublyMapped
BNE %FT22
; Cheesy architecture check: check the identified cache/ARMop type
LDR r11, =ZeroPage
LDRB lr, [r11, #Cache_Type]
TEQ lr, #CT_ctype_WB_CR7_Lx
ADRNE r0, ErrorBlock_BadDynamicAreaOptions
BNE DynArea_ErrorTranslateAndExit
; Check the supplied access policy is XN
LDR r11, [r11, #MMU_PPLAccess]
AND lr, r4, #DynAreaFlags_APBits
LDR r11, [r11, lr, LSL #2]
TST r11, #CMA_Partially_UserXN+CMA_Partially_PrivXN ; n.b. XN flag sense is inverted in PPLAccess, so NE means executable
ADRNE r0, ErrorBlock_BadDynamicAreaOptions
BNE DynArea_ErrorTranslateAndExit
22
; now validate maximum size of area
[ DynArea_QuickHandles
......
......@@ -203,7 +203,7 @@ CamMapBroke
;
; 4K page size assumed!
RemoveCacheabilityR0ByMinusR2 ROUT
Entry "r1-r5"
Entry "r0-r5"
MOV r5, #DynAreaFlags_NotCacheable
MOV r1, #-1
10
......@@ -232,11 +232,26 @@ RemoveCacheabilityR0ByMinusR2 ROUT
SUBS r2, r2, #4096
BNE %BT10
MOV r6, r5
FRAMSTR r0
; Perform the required cache/TLB maintenance
; TODO - Fix MMU_ChangingEntries to do TLB maintenance first so that we can just use that in cacheable case
LDR r4, =ZeroPage
FRAMLDR r1,,r2
MOV r1,r1,LSR #Log2PageSize
ARMop MMU_ChangingUncachedEntries,,,r4
TST r6, #DynAreaFlags_NotCacheable
BNE %FT91
FRAMLDR r0
FRAMLDR r1,,r2
ARMop Cache_CleanInvalidateRange,,,r4
91
EXIT
]
; Move pages at [R0-R4,R0) to [R1-R4,R1)
[ FastCDA_Bulk
; Pages must be uncacheable
]
; R0 < R1
MoveUncacheableR0ToR1ByMinusR4 ROUT
Entry "r0,r1,r4,r6,r9"
......@@ -263,11 +278,15 @@ MoveUncacheableR0ToR1ByMinusR4 ROUT
EXIT
; Move pages at [R0,R0+R4) to [R1,R1+R4)
[ FastCDA_Bulk
; Pages must currently be uncacheable
]
; R0 > R1
MoveUncacheableR0ToR1ByR4WithAccessR6 ROUT
Entry "r0,r1,r4,r6,r9"
[ FastCDA_Bulk
ORR r6, r6, #PageFlags_Unsafe
]
MOV r9, #0 ; no funny business while moving these pages
15
BL MovePageAtR0ToR1WithAccessR6 ; move page
......@@ -297,7 +316,7 @@ MoveUncacheableR0ToR1ByR4WithAccessR6 ROUT
ShuffleDoublyMappedRegionForGrow ROUT
Entry "r0,r1"
[ FastCDA_Bulk
; Perform sledgehammer logic upfront
; Perform cache maintenance upfront
MOV r0, r1
Push "r2,r6"
MOV r2, r4
......@@ -422,30 +441,15 @@ DoTheShrink ROUT
[ FastCDA_Bulk
Push "r0"
BL RemoveCacheabilityR0ByMinusR2
LDR r4, [r11, #DANode_Flags]
ADR lr, %FT19
TST r4, #DynAreaFlags_DoublyMapped
LDR r4, =ZeroPage
BNE %FT18
; Interacting with singly-mapped region - use regular logic
MOV r1, r2, LSR #Log2PageSize
ARMop MMU_ChangingUncachedEntries,,, r4
TST r6, #DynAreaFlags_NotCacheable
BNE %FT19
LDR r1, [sp]
SUB r0, r1, r2
ARMop Cache_CleanInvalidateRange,,, r4
B %FT19
18
; Interacting with doubly-mapped region - use sledgehammer logic
LDR r6, [r11, #DANode_Flags]
TST r6, #DynAreaFlags_DoublyMapped
BEQ %FT19
; Interacting with doubly-mapped region - make entireity of lower mapping uncacheable too
LDR r0, [r11, #DANode_Base]
MOV r1, r2
LDR r2, [sp, #4] ; Grab old source size (pushed r3)
BL RemoveCacheabilityR0ByMinusR2 ; Remove cacheability for entireity of lower mapping (whole thing needs shuffling)
BL RemoveCacheabilityR0ByMinusR2
MOV r2, r1
ARMop MMU_ChangingUncached,,, r4
TST r6, #DynAreaFlags_NotCacheable
ARMop Cache_CleanInvalidateAll, EQ,, r4
19
Pull "r0"
]
......
......@@ -175,9 +175,6 @@ BangL2PT ; internal entry point used only
TST r11, #PageFlags_Unsafe
BNE BangL2PT_unsafe
TST r11, #DynAreaFlags_DoublyMapped
BNE BangL2PT_sledgehammer ;if doubly mapped, don't try to be clever
;In order to safely map out a cacheable page and remove it from the
;cache, we need to perform the following process:
;* Make the page uncacheable
......@@ -199,8 +196,17 @@ BangL2PT ; internal entry point used only
LDR r4, =TempUncache_L2PTMask
BIC lr, lr, r4 ;remove current attributes
ORR lr, lr, r0
STR lr, [r1, r3, LSR #10] ;Make uncacheable
STR lr, [r1, r3, LSR #10]! ;Make uncacheable
TST r11, #DynAreaFlags_DoublyMapped
LDR r4, =ZeroPage
BEQ %FT19
STR lr, [r1, r9, LSR #10] ;Update 2nd mapping too if required
ADD r0, r3, r9
ARMop MMU_ChangingUncachedEntry,,, r4 ; TLB flush
ADD r0, r3, r9
ADD r1, r0, #4096
ARMop Cache_CleanInvalidateRange,,, r4 ; Cache flush
19
MOV r0, r3
ARMop MMU_ChangingUncachedEntry,,, r4 ; TLB flush
MOV r0, r3
......@@ -208,35 +214,18 @@ BangL2PT ; internal entry point used only
ARMop Cache_CleanInvalidateRange,,, r4 ; Cache flush
LDR r1, =L2PT
20 STR r6, [r1, r3, LSR #10] ;update L2PT entry
20 STR r6, [r1, r3, LSR #10]! ;update L2PT entry
TST r11, #DynAreaFlags_DoublyMapped
BEQ %FT21
STR r6, [r1, r9, LSR #10] ;Update 2nd mapping
MOV r0, r3
ARMop MMU_ChangingUncachedEntry,,, r4 ; TLB flush for 1st mapping
ADD r3, r3, r9 ;restore r3 back to 2nd copy
21
Pull "lr"
MOV r0, r3
ARMop MMU_ChangingUncachedEntry,,tailcall,r4
BangL2PT_sledgehammer
;sledgehammer is super cautious and does cache/TLB coherency on a global basis
;should only be used for awkward cases
;
TEQ r6, #0 ;EQ if mapping out
TSTEQ r11, #DynAreaFlags_NotCacheable ;EQ if also cacheable (overcautious for temp uncache+illegal PCB combos)
ADR lr, %FT30
LDR r4, =ZeroPage
ARMop MMU_Changing, EQ, tailcall, r4
ARMop MMU_ChangingUncached, NE, tailcall, r4
30 STR r6, [r1, r3, LSR #10]! ; update level 2 page table (and update pointer so we can use bank-to-bank offset
TST r11, #DynAreaFlags_DoublyMapped ; if area doubly mapped
STRNE r6, [r1, r9, LSR #10] ; then store entry for 2nd copy as well
ADDNE r3, r3, r9 ; and point logical address back at 2nd copy
; In order to guarantee that the result of a page table write is
; visible, the ARMv6+ memory order model requires us to perform TLB
; maintenance (equivalent to the MMU_ChangingUncached ARMop) after we've
; performed the write. Performing the maintenance beforehand (as we've
; done traditionally) will work most of the time, but not always.
Pull "lr"
ARMop MMU_ChangingUncached,,tailcall,r4
BangL2PT_unsafe
STR r6, [r1, r3, LSR #10]! ; update level 2 page table (and update pointer so we can use bank-to-bank offset
TST r11, #DynAreaFlags_DoublyMapped ; if area doubly mapped
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment