GitLab

Detail:
  With this set of changes, each AMB node is now the owner of a fake DANode which is linked to a PMP.
  From a user's perspective the behaviour of AMBControl is the same as before, but rewriting it to use PMPs internally offers the following (potential) benefits:
  * Reduction in the amount of code which messes with the CAM & page tables, simplifying future work/maintenance. Some of the AMB ops (grow, shrink) now just call through to OS_ChangeDynamicArea. However all of the old AMB routines were well-optimised, so to avoid a big performance hit for common operations not all of them have been removed (e.g. mapslot / mapsome). Maybe one day these optimal routines will be made available for use by regular PMP DAs.
  * Removal of the slow Service_MemoryMoved / Service_PagesSafe handlers that had to do page list fixup after the core kernel had reclaimed/moved pages. Since everything is a PMP, the kernel will now deal with this on behalf of AMB.
  * Removal of a couple of other slow code paths (e.g. Do_AMB_MakeUnsparse calls from OS_ChangeDynamicArea)
  * Potential for more flexible mapping of application space in future, e.g. sparse allocation of memory to the wimp slot
  * Simpler transition to an ASID-based task swapping scheme on ARMv6+?
  Other changes of note:
  * AMB_LazyMapIn switch has been fixed up to work correctly (i.e. turning it off now disables lazy task swapping and all associated code instead of producing a build error)
  * The DANode for the current app should be accessed via the GetAppSpaceDANode macro. This will either return the current AMB DANode, or AppSpaceDANode (if e.g. pre-Wimp). However be aware that AppSpaceDANode retains the legacy behaviour of having a base + size relative to &0, while the AMB DANodes (identifiable via the PMP flag) are sane and have their base + size relative to &8000.
  * Mostly-useless DebugAborts switch removed
  * AMBPhysBin (page number -> phys addr lookup table) removed. Didn't seem to give any tangible performance benefit, and was imposing hidden restrictions on memory usage (all phys RAM fragments in PhysRamTable must be multiple of 512k). And if it really was a good optimisation, surely it should have been applied to all areas of the kernel, not just AMB!
  Other potential future improvements:
  * Turn the fake DANodes into real dynamic areas, reducing the amount of special code needed in some places, but allow the DAs to be hidden from OS_DynamicArea 3 so that apps/users won't get too confused
  * Add a generic abort trapping system to PMPs/DAs (lazy task swapping abort handler is still a special case)
  File changes:
  - s/ARM600, s/VMSAv6, s/ExtraSWIs - Remove DebugAborts
  - s/ArthurSWIs - Remove AMB service call handler dispatch
  - s/ChangeDyn - AMB_LazyMapIn switch fixes. Add alternate internal entry points for some PMP ops to allow the DANode to be specified (used by AMB)
  - s/Exceptions - Remove DebugAborts, AMB_LazyMapIn switch fixes
  - s/Kernel - Define GetAppSpaceDANode macro, AMB_LazyMapIn switch fix
  - s/MemInfo - AMB_LazyMapIn switch fixes
  - s/AMBControl/AMB - Update GETs
  - s/AMBControl/Memory - Remove block size quantisation, AMB_BlockResize (page list blocks are now allocated by PMP code)
  - s/AMBControl/Options - Remove PhysBin definitions, AMBMIRegWords (moved to Workspace file), AMB_LimpidFreePool switch. Add AMB_Debug switch.
  - s/AMBControl/Workspace - Update AMBNode to contain an embedded DANode. Move AMBMIRegWords here from Options file.
  - s/AMBControl/allocate - Fake DA node initialisation
  - s/AMBControl/deallocate - Add debug output
  - s/AMBControl/growp, growshrink, mapslot, mapsome, shrinkp - Rewrite to use PMP ops where possible, add debug output
  - s/AMBControl/main - Remove PhysBin initialisation. Update the enumerate/mjs_info call.
  - s/AMBControl/memmap - Low-level memory mapping routines updated or rewritten as appropriate.
  - s/AMBControl/readinfo - Update to cope with DANode
  - s/AMBControl/service - Remove old service call handlers
  - s/AMBControl/handler - DA handler for responding to PMP calls from OS_ChangeDynamicArea; just calls through to growpages/shrinkpages as appropriate.
Admin:
  Tested on pretty much everything currently supported


Version 5.66. Tagged as 'Kernel-5_66'

Detail:
  This set of changes:
  * Refactors page table entry encoding/decoding so that it's (mostly) performed via functions in the MMU files (s.ARM600, s.VMSAv6) rather than on an ad-hoc basis as was the case previously
  * Page table entry encoding/decoding performed during ROM init is also handled via the MMU functions, which resolves some cases where the wrong cache policy was in use on ARMv6+
  * Adds basic support for shareable pages - on non-uniprocessor systems all pages will be marked as shareable (however, we are currently lacking ARMops which broadcast cache maintenance operations to other cores, so safe sharing of cacheable regions isn't possible yet)
  * Adds support for the VMSA XN flag and the "privileged ROM" access permission. These are exposed via RISC OS access privileges 4 and above, taking advantage of the fact that 4 bits have always been reserved for AP values but only 4 values were defined
  * Adds OS_Memory 17 and 18 to convert RWX-style access flags to and from RISC OS access privelege numbers; this allows us to make arbitrary changes to the mappings of AP values 4+ between different OS/hardware versions, and allows software to more easily cope with cases where the most precise AP isn't available (e.g. no XN on <=ARMv5)
  * Extends OS_Memory 24 (CheckMemoryAccess) to return executability information
  * Adds exported OSMem header containing definitions for OS_Memory and OS_DynamicArea
  File changes:
  - Makefile - export C and assembler versions of hdr/OSMem
  - Resources/UK/Messages - Add more text for OS_Memory errors
  - hdr/KernelWS - Correct comment regarding DCacheCleanAddress. Allocate workspace for MMU_PPLTrans and MMU_PPLAccess.
  - hdr/OSMem - New file containing exported OS_Memory and OS_DynamicArea constants, and public page flags
  - hdr/Options - Reduce scope of ARM6support to only cover builds which require ARMv3 support
  - s/AMBControl/Workspace - Clarify AMBNode_PPL usage
  - s/AMBControl/growp, mapslot, mapsome, memmap - Use AreaFlags_ instead of AP_
  - s/AMBControl/main, memmap - Use GetPTE instead of generating page table entry manually
  - s/ARM600 - Remove old coments relating to lack of stack. Update BangCam to use GetPTE. Update PPL tables, removing PPLTransL1 (L1 entries are now derived from L2 table on demand) and adding a separate table for ARM6. Implement the ARM600 versions of the Get*PTE ('get page table entry') and Decode*Entry functions
  - s/ARMops - Add Init_PCBTrans function to allow relevant MMU_PPLTrans/MMU_PCBTrans pointers to be set up during the pre-MMU stage of ROM init. Update ARM_Analyse to set up the pointers that are used post MMU init.
  - s/ChangeDyn - Move a bunch of flags to hdr/OSMem. Rename the AP_ dynamic area flags to AreaFlags_ to avoid name clashes and confusion with the page table AP_ values exported by Hdr:MEMM.ARM600/Hdr:MEMM.VMSAv6. Also generate the relevant flags for OS_Memory 24 so that it can refer to the fixed areas by their name instead of hardcoding the permissions.
  - s/GetAll - GET Hdr:OSMem
  - s/HAL - Change initial page table setup to use DA/page flags and GetPTE instead of building page table entries manually. Simplify AllocateL2PT by removing the requirement for the user to supply the access perimssions that will be used for the area; instead for ARM6 we just assume that cacheable memory is the norm and set L1_U for any L1 entry we create here.
  - s/Kernel - Add GetPTE macro (for easier integration of Get*PTE functions) and GenPPLAccess macro (for easy generation of OS_Memory 24 flags)
  - s/MemInfo - Fixup OS_Memory 0 to not fail on seeing non-executable pages. Implement OS_Memory 17 & 18. Tidy up some error generation. Make OS_Memory 13 use GetPTE. Extend OS_Memory 24 to return (non-) executability information, to use the named CMA_ constants generated by s/ChangeDyn, and to use the Decode*Entry functions when it's necessary to decode page table entries.
  - s/NewReset - Use AreaFlags_ instead of AP_
  - s/VMSAv6 - Remove old comments relating to lack of stack. Update BangCam to use GetPTE. Update PPL tables, removing PPLTransL1 (L1 entries are now derived from L2 table on demand) and adding a separate table for shareable pages. Implement the VMSAv6 versions of the Get*PTE and Decode*Entry functions.
Admin:
  Tested on Raspberry Pi 1, Raspberry Pi 3, Iyonix, RPCEmu (ARM6 & ARM7), comparing before and after CAM and page table dumps to check for any unexpected differences


Version 5.55. Tagged as 'Kernel-5_55'

Detail:
  s/ChangeDyn - Fix OS_DynamicArea 20 to work properly with sparse & PMP DAs. It now checks against the max extent of the area rather than the current size; this matches the logic used for checking fixed system workspace areas. The call only determines the ownership of a logical address, and it's considered the caller's responsibility to check if there's actually a page at the given address.
  s/ChangeDyn - Revise OS_DynamicArea 25 to remove the redundant 'PMP page flags' entry, and to allow pages to be looked up by either PMP page index, phys page number, or DA page index
  s/ChangeDyn - Tidy up InitDynamicAreas by adding the NextFreePage routine to help determine the next page to be added to the free pool.
  s/AMBControl/Workspace, s/AMBControl/main, s/AMBControl/memmap - Fix lazy mapping in of pages to use the correct L2PT flags for the default CB cache policy
  s/AMBControl/allocate - Get rid of magic constant when extracting page flags from DA flags, and make note of the fact that assorted bits of code ignore the flags
  s/AMBControl/growp, s/AMBControl/shrinkp - Reverse the page order when growing/shrinking areas, to match OS_ChangeDynamicArea. This helps both DAs and application space to have pages allocated to them in contiguous physical order - which in turn helps produce shorter, more optimal scatter lists for DMA
Admin:
  Tested on Pandaboard


Version 5.35, 4.79.2.287. Tagged as 'Kernel-5_35-4_79_2_287'

Detail:
  This set of changes adds support for "physical memory pools" (aka PMPs), a new type of dynamic area which allow physical pages to be claimed/allocated without mapping them in to the logical address space. PMPs have full control over which physical pages they use (similar to DAs which request specific physical pages), and also have full control over the logical mapping of their pages (which pages go where, and per-page access/cacheability control).
  Currently the OS makes use of two PMPs: one for the free pool (which now has a logical size of zero - freeing up gigabytes of logical space), and one for the RAM disc (logical size of 1MB, allowing for a physical size limited only by the amount of free memory)
  Implementing these changes has required a number of other changes to be made:
  * The CAM has been expanded from 8 bytes per entry to 16 bytes per entry, in order to allow each RAM page to store information about its PMP association
  * The system heap has been expanded to 32MB in size (from just under 4MB), in order to allow it to be used to store PMP page lists (1 word needed per page, but PMP pages may not always have physical pages assigned to them - so to allow multiple large PMPs to exist we need more than just 1 word per RAM page)
  * The &FA000000-&FBFFFFFF area of fixed kernel workspace has been shuffled around to accomodate the larger CAM, and the system heap is now located just above the RMA.
  * SoftResets code stripped out (unlikely we'll ever want to fix and re-enable it)
  * A couple of FastCDA options are now permanently on
  * Internal page flags shuffled around a bit. PageFlags_Unavailable now publicly exposed so that PMP clients can lock/unlock pages at will.
  * When OS_ChangeDynamicArea is asked to grow or shrink the free pool, it now implicitly converts it into a shrink or grow of application space (which is what would happen anyway). This simplifies the implementation; during a grow, pages (or replacement pages) are always sourced from the free pool, and during a shrink pages are always sent to the free pool.
  File changes:
  - hdr/KernelWS - Extend DANode structure. Describe CAM format. Adjust kernel workspace.
  - hdr/OSRSI6, s/Middle - Add new item to expose the CAM format
  - hdr/Options - Remove SoftResets switch. Add some PMP switches.
  - s/ARM600, s/VMSAv6 - Updated for new CAM format. Note that although the CAM stores PMP information, BangCamUpdate currently doesn't deal with updating that data - it's the caller's responsibility to do so where appropriate.
  - s/ChangeDyn - Lots of changes to implement PMP support, and to cope with the new CAM format.
  - s/HAL - Updated to cope with new CAM format, and lack of logical mapping of free pool.
  - s/MemInfo - Updated to cope with new CAM format. OS_Memory 0 updated to cope with converting PPN to PA for pages which are mapped out. OS_Memory 24 updated to decode the access permissions on a per-page basis for PMPs, and fixed its HWM usage for sparse DAs.
  - s/NewReset - Soft reset code and unused AddCamEntries function removed. Updated to cope with new CAM format, PMP free pool, PMP RAMFS
  - s/AMBControl/allocate - Update comment (RMA hasn't been used for AMBControl nodes for a long time)
  - s/AMBControl/growp, s/AMBControl/memmap, s/AMBControl/shrinkp - Update for new CAM format + PMP free pool
  - s/vdu/vdudriver - Strip out soft reset code.
Admin:
  Tested on Pandaboard
  This is just a first iteration of the PMP feature, with any luck future changes will improve functionality. This means APIs are subject to change as well.


Version 5.35, 4.79.2.284. Tagged as 'Kernel-5_35-4_79_2_284'

Detail:
  A whole mass of changes to add high processor vectors + zero page relocation support to the Cortex branch of the kernel
  At the moment the code can only cope with two ZeroPage locations, &0 and &FFFF0000. But with a bit more tweaking those restrictions can probably be lifted, allowing ZeroPage to be hidden at almost any address (assuming it's fixed at compile time). If I've done my job right, these restrictions should all be enforced by asserts.
  There's a new option, HiProcVecs, in hdr/Options to control whether high processor vectors are used. When enabling it and building a ROM, remember:
  * FPEmulator needs to be built with the FPEAnchor=High option specified in the components file (not FPEAnchorType=High as my FPEmulator commit comments suggested)
  * ShareFS needs unplugging/removing since it can't cope with it yet
  * Iyonix users will need to use the latest ROOL boot sequence, to ensure the softloaded modules are compatible (OMAP, etc. don't really softload much so they're OK with older sequences)
  * However VProtect also needs patching to fix a nasty bug there - http://www.riscosopen.org/tracker/tickets/294
  The only other notable thing I can think of is that the ProcessTransfer code in s/ARM600 & s/VMSAv6 is disabled if high processor vectors are in use (it's fairly safe to say that code is obsolete in HAL builds anyway?)
  Fun challenge for my successor: Try setting ZeroPage to &FFFF00FF (or similar) so its value can be loaded with MVN instead of LDR. Then use positive/negative address offsets to access the contents.
  File changes:
  - hdr/ARMops - Modified ARMop macro to take the ZeroPage pointer as a parameter instead of 'zero'
  - hdr/Copro15ops - Corrected $quick handling in myISB macro
  - hdr/Options - Added ideal setting for us to use for HiProcVecs
  - s/AMBControl/allocate, s/AMBControl/growp, s/AMBControl/mapslot, s/AMBControl/memmap, s/AMBControl/service, s/AMBControl/shrinkp, s/Arthur2, s/Arthur3, s/ArthurSWIs, s/ChangeDyn, s/ExtraSWIs, s/HAL, s/HeapMan, s/Kernel, s/MemInfo, s/Middle, s/ModHand, s/MoreSWIs, s/MsgCode, s/NewIRQs, s/NewReset, s/Oscli, s/PMF/buffer, s/PMF/IIC, s/PMF/i2cutils, s/PMF/key, s/PMF/mouse, s/PMF/osbyte, s/PMF/oseven, s/PMF/osinit, s/PMF/osword, s/PMF/oswrch, s/SWINaming, s/Super1, s/SysComms, s/TickEvents, s/Utility, s/vdu/vdu23, s/vdu/vdudriver, s/vdu/vdugrafl, s/vdu/vdugrafv, s/vdu/vdupalxx, s/vdu/vdupointer, s/vdu/vduswis, s/vdu/vduwrch - Lots of updates to deal with zero page relocation
  - s/ARM600 - UseProcessTransfer option. Zero page relocation support. Deleted pre-HAL ClearPhysRAM code to tidy the file up a bit.
  - s/ARMops - Zero page relocation support. Set CPUFlag_HiProcVecs when high vectors are in use.
  - s/KbdResPC - Disable compilation of dead code
  - s/VMSAv6 - UseProcessTransfer option. Zero page relocation support.
Admin:
  Tested with OMAP & Iyonix ROM softloads, both with high & low zero page.
  High zero page hasn't had extensive testing, but boot sequence + ROM apps seem to work.


Version 5.35, 4.79.2.98.2.48. Tagged as 'Kernel-5_35-4_79_2_98_2_48'

Version 5.35, 4.79.2.12. Tagged as 'Kernel-5_35-4_79_2_12'