GitLab

Detail:
  For a while we've known that the correct way of doing cache maintenance on ARMv6+ (e.g. when converting a page from cacheable to non-cacheable) is as follows:
  1. Write new page table entry
  2. Flush old entry from TLB
  3. Clean cache + drain write buffer
  The MMU_Changing ARMops (e.g. MMU_ChangingEntry) implement the last two items, but in the wrong order. This has caused the operations to fall out of favour and cease to be used, even in pre-ARMv6 code paths where the effects of improper cache/TLB management perhaps weren't as readily visible.
  This change re-specifies the relevant ARMops so that they perform their sub-operations in the correct order to make them useful on modern ARMs, updates the implementations, and updates the kernel to make use of the ops whereever relevant.
  File changes:
  - Docs/HAL/ARMop_API - Re-specify all the MMU_Changing ARMops to state that they are for use just after a page table entry has been changed (as opposed to before - e.g. 5.00 kernel behaviour). Re-specify the cacheable ones to state that the TLB invalidatation comes first.
  - s/ARM600, s/ChangeDyn, s/HAL, s/MemInfo, s/VMSAv6, s/AMBControl/memmap - Replace MMU_ChangingUncached + Cache_CleanInvalidate pairs with equivalent MMU_Changing op
  - s/ARMops - Update ARMop implementations to do everything in the correct order
  - s/MemMap2 - Update ARMop usage, and get rid of some lingering sledgehammer logic from ShuffleDoublyMappedRegionForGrow
Admin:
  Tested on pretty much everything currently supported


Version 5.70. Tagged as 'Kernel-5_70'

Add new ARMops. Add macros which map the ARMv7/v8 cache/TLB maintenance mnemonics (as featured in recent ARM ARMs) to MCR ops.

Detail:
  - Docs/HAL/ARMop_API - Document the new ARMops. These ops are intended to help with future work (DMA without OS_Memory 0 "make temp uncacheable", and minimising cache maintenance when unmapping pages) and aren't in use just yet.
  - hdr/Copro15ops - Add new macros for ARMv7+ which map the mnemonics seen in recent ARM ARMs to the corresponding MCR ops. This should make things easier when cross-referencing docs and reduce the risk of typos.
  - hdr/KernelWS - Shuffle kernel workspace a bit to make room for the new ARMops
  - hdr/OSMisc - Expose new ARMops via OS_MMUControl 2
  - s/ARMops - Implement the new ARMops. Change the ARMv7+ ARMops to use the new mnemonic macros. Also get rid of myDSB / myISB usage from ARMv7+ code paths; use DSB/ISB/etc. directly to ensure correct behaviour
  - s/HAL - Mnemonic + ISB/DSB updates. Change software RAM clear to do 16 bytes at a time for kernel workspace instead of 32 to allow the kernel workspace tweaks to work.
Admin:
  Binary diff shows that mnemonics map to the original MCR ops correctly
  Note: Raspberry Pi builds will now emit lots of warnings due to increased DSB/ISB instruction use. However it should be safe to ignore these as they should only be present in v7+ code paths.
  Note: New ARMops haven't been tested yet, will be disabled (or at least hidden from user code) in a future checkin


Version 5.68. Tagged as 'Kernel-5_68'

Detail:
  This set of changes tackles two main issues:
  * Before mapping out a cacheable page or making it uncacheable, the OS performs a cache clean+invalidate op. However this leaves a small window where data may be fetched back into the cache, either accidentally (dodgy interrupt handler) or via agressive prefetch (as allowed for by the architecture). This rogue data can then result in coherency issues once the pages are mapped out or made uncacheable a short time later.
    The fix for this is to make the page uncacheable before performing the cache maintenance (although this isn't ideal, as prior to ARMv7 it's implementation defined whether address-based cache maintenance ops affect uncacheable pages or not - and on ARM11 it seems that they don't, so for that CPU we currently force a full cache clean instead)
  * Modern ARMs generally ignore unexpected cache hits, so there's an interrupt hole in the current OS_Memory 0 "make temporarily uncacheable" implementation where the cache is being flushed after the page has been made uncacheable (consider the case of a page that's being used by an interrupt handler, but the page is being made uncacheable so it can also be used by DMA). As well as affecting ARMv7+ devices this was found to affect XScale (and ARM11, although untested for this issue, would have presumably suffered from the "can't clean uncacheable pages" limitation)
    The fix for this is to disable IRQs around the uncache sequence - however FIQs are currently not being dealt with, so there's still a potential issue there.
  File changes:
  - Docs/HAL/ARMop_API, hdr/KernelWS, hdr/OSMisc - Add new Cache_CleanInvalidateRange ARMop
  - s/ARM600, s/VMSAv6 - BangCam updated to make the page uncacheable prior to flushing the cache. Add GetTempUncache macro to help with calculating the page flags required for making pages uncacheable. Fix abort in OS_MMUControl on Raspberry Pi - MCR-based ISB was resetting ZeroPage pointer to 0
  - s/ARMops - Cache_CleanInvalidateRange implementations. PL310 MMU_ChangingEntry/MMU_ChangingEntries refactored to rely on Cache_CleanInvalidateRange_PL310, which should be a more optimal implementation of the cache cleaning code that was previously in MMU_ChangingEntry_PL310.
  - s/ChangeDyn - Rename FastCDA_UpFront to FastCDA_Bulk, since the cache maintenance is no longer performed upfront. CheckCacheabilityR0ByMinusR2 now becomes RemoveCacheabilityR0ByMinusR2. PMP LogOp implementation refactored quite a bit to perform cache/TLB maintenance after making page table changes instead of before. One flaw with this new implementation is that mapping out large areas of cacheable pages will result in multiple full cache cleans while the old implementation would have (generally) only performed one - a two-pass approach over the page list would be needed to solve this.
  - s/GetAll - Change file ordering so GetTempUncache macro is available earlier
  - s/HAL - ROM decompression changed to do full MMU_Changing instead of MMU_ChangingEntries, to make sure earlier cached data is truly gone from the cache. ClearPhysRAM changed to make page uncacheable before flushing cache.
  - s/MemInfo - OS_Memory 0 interrupt hole fix
  - s/AMBControl/memmap - AMB_movepagesout_L2PT now split into cacheable+non-cacheable variants. Sparse map out operation now does two passes through the page list so that they can all be made uncacheable prior to the cache flush + map out.
Admin:
  Tested on StrongARM, XScale, ARM11, Cortex-A7, Cortex-A9, Cortex-A15, Cortex-A53
  Appears to fix the major issues plaguing SATA on IGEPv5


Version 5.35, 4.79.2.306. Tagged as 'Kernel-5_35-4_79_2_306'

Detail:
  s/ARMops - Change VMSAv6 cache policy mapping so that default NCB policy maps to Normal, non-cacheable memory rather than Device memory. This allows unaligned accesses to work, and also delivers some major performance improvements for some activities (write performance seems about 2x better than Device)
  Docs/HAL/ARMop_API - Wrap the new barrier ARMop description text to match the rest of the file
Admin:
  Tested on ARM11, Cortex-A7, -A8, -A9
  Note - relaxed memory consistency rules for Normal vs. Device mean that now more than ever, developers authoring device drivers for ARMv6+ need to be careful to use memory barriers in all the right places (preferably the new ARMop barriers exposed by OS_MMUControl 2 to ensure compatibility with all machines)


Version 5.35, 4.79.2.280. Tagged as 'Kernel-5_35-4_79_2_280'

Detail:
  - Docs/HAL/ARMop_API - Updated with documentation for the new ARMops.
  - s/ARMops - Set up pointers for the new memory barrier ARMops. Add full implementations for ARMv6 & ARMv7; older architectures should be able to get by with a mix of null ops & write buffer drain ops. Update ARMopPtrTable to validate structure against the list in hdr/OSMisc
  - hdr/KernelWS - Reserve workspace for new ARMops. Free up a bit of space by limiting ourselves to 2 cache levels with ARMv7. Remove some unused definitions.
  - hdr/OSMisc - New header defining OS_PlatformFeatures & OS_MMUControl reason codes, OS_PlatformFeatures 0 flags, and OS_MMUControl 2 ARMop indices
  - Makefile - Add export rules for OSMisc header
  - hdr/ARMops, s/ARM600, s/VMSAv6 - Remove CPUFlag_* and MMUCReason_* definitions. Update OS_MMUControl write buffer drain to use DSB_ReadWrite ARMop (which is what most existing write buffer drain implementations have been renamed to).
  - s/GetAll - Get Hdr:OSMisc
  - s/Kernel - Use OS_PlatformFeatures reason code symbols
  - s/vdu/vdudecl - Remove unused definition
Admin:
  Tested on ARM11, Cortex-A8, Cortex-A9


Version 5.35, 4.79.2.279. Tagged as 'Kernel-5_35-4_79_2_279'

Improve support for VMSAv6 cache policies & memory types. Expose raw ARMops via OS_MMUControl & cache information via OS_PlatformFeatures.

Detail:
  Docs/HAL/ARMop_API - Document two new ARMops: Cache_Examine and IMB_List
  hdr/KernelWS - Shuffle workspace round a bit to allow space for the two new ARMops. IOSystemType now deleted (has been deprecated and fixed at 0 for some time)
  s/ARM600 - Cosmetic changes to BangCam to make it clearer what's going on. Add OS_MMUControl 2 (get ARMop) implementation.
  s/ARMops - Switch out different ARMop implementations and XCB tables depending on MMU model - helps reduce assembler warnings and make it clearer what code paths are and aren't possible. Add implementations of the two new ARMops. Simplify ARM_Analyse_Fancy by removing some tests which we know will have certain results. Use CCSIDR constants in ARMv7 ARMops instead of magic numbers. Update XCB table comments, and add a new table for VMSAv6
  s/ChangeDyn - Define constant for the new NCB 'idempotent' cache ...

Version 5.35, 4.79.2.18. Tagged as 'Kernel-5_35-4_79_2_18'

kernel now attempts to substitute video mode numbers in face of h/w with limited bits-per-pixel support (not tested yet)

HAL_API document added - early draft only, of interest to those
writing or modifying HALs for new h/w
ARMop_API document added - early draft only, of interest only
to those modifying kernel to support new ARM cores
*** polite comments on HAL_API welcome ***

Version 5.35, 4.79.2.15. Tagged as 'Kernel-5_35-4_79_2_15'