Commit cf85a5d6 authored by Robert Sprowson's avatar Robert Sprowson

Fix for mistargetted write with a file size > 2G when a background scatter...

Fix for mistargetted write with a file size > 2G when a background scatter list straddles a fragment

When large transfers are being buffered to be written in the background they are attached to the Fcb without considering how the underlying disc fragments are lined up.
When the background process grabs a handful of buffers to build into a scatter list to pass to the low level DiscOp it picks the head of the list and goes forward (SkipWriteBehind) to the fragment end and back (BackwardSkipWriteBehind).
However, BackwardsSkipWriteBehind uses DefFileFrag which substitutes a default offset of 0, and the signed compare resulted in this being left as the believed start of fragment.
In turn, far too many buffers got associated with the fragment, and written to the wrong place on the disc.
In general this doesn't occur, but if the group of buffers happens to straddle a fragment, and that group of buffers is > 2G, the signed compare would trip up.
As fsbash uses the same random number seed this conspired that testing the same drive twice (with and without background transfers enabled) would appear to pass the integrity check step. A new harddisc was used to show this up.

FileCore80.s: Recoded the compare to be in sectors, so 4G files aren't top bit set, but shared files not at the fragment start still are.
Docs/AdfsBuffers: Added some notes before I forget what I just wrote meant.


Version 3.58. Tagged as 'FileCore-3_58'
parent 7a9d7b7d
......@@ -38,6 +38,33 @@ for us, pair extension is disabled. That is, the scatter list never
gets extended just when the driver least expects it, as it causes
all sorts of headaches - allegedly, of course.
There's one process per controller (one for floppies and one for fixed discs).
Each process maintains some state in its respective process control block,
this is kept for the duration of the processing:
* The drive number (so that scatter lists can be sent to the
right drive on that controller)
* Start offset & end offset, in bytes, for the portion of the transfer
currently being scattered. Note that scatter lists are just attached
blindly by the foreground process without any consideration of where
they cross fragment boundaries - it is only when a new chunk of a
scatter list is being passed (via the DiscOp low level entry) to the
client filing system that the fragment boundaries are respected
* Fcb, the file control block to which the scatter list entries relate
* FragEnd, the end offset, in bytes, of the portion of the scatter list
that is currently being processed.
* Error & Status (part of the client API) defined to be held at -4
and -8 from the scatter list start
When idle FileCore will look to see if any more background work could be
started. If writes are possible the first buffer (attached to the Fcb) will
be picked and using its BufFileOff this is mapped to a fragment. More buffers
will be considered (using SkipWriteBehind and BackwardsSkipWriteBehind) to
see how many buffers cover the corresponding fragment. These are then inserted
in one go onto a scatter list and consumed.
Later, the above steps will repeat. Due to the way the buffers are attached
to the Fcb this results in writing backwards through the file as it happens.
Process Scatter Lists
------- ------- -----
......@@ -50,77 +77,22 @@ to either a buffer or a direct transfer. A scatter list entry
cannot cover multiple buffers as data for adjacent buffers in
a transfer will not conjoin. Does this apply for sub-buffers?
DiscAdjust
----------
DiscAdjust/RamAdjust
--------------------
This is an adjustment offset which expresses either the originating
buffer (when RamAdjust) or target disc address (when DiscAdjust)
relative to the file offset.
By example
This seems to be an adjustment offset which would generally appear
to be added to any offset within a file to give the actual disc
address for that offset - thus DiscAdjust only remains valid
when working within a single fragment. It is added to the offset
into a file to give the disc address corresponding to that offset.
Buffer = 0x12340000
FileOff = 0x00010000
RamAdjust = 0x12330000
so as you read bytes at FileOff, there's no need to keep two indexes
in sync, the buffer address can be recreated by just adding RamAdjust.
DiscAdjust works the same, only it's a disc address rather than RAM.
I'm thinking that DiscAdjust must be always sector aligned. If
this assumption breaks I'll have fun.
The next three sections list what needs modified.
Functions which will need modified:
--------- ----- ---- ---- --------
FileFrag ... SBP: done
BackGroundFileCacheOp ... SBP: done
GetBytesEntry ... SBP: done
PutBytesEntry ... SBP: done
BackGroundOps ... SBP: done
Functions which will not need modified:
--------- ----- ---- --- ---- --------
FindSubBuf
UpdateProcess
GetPutCommon
FcbCommon
ReadAheadCommon
R3LoadNewMap
BufsToRam
BackGroundTidyUp
ReduceWriteBehind
AddBuffer
SkipEmpty
SkipWriteBehind
BackwardsSkipWriteBehind
DefFileFrag
FindSubBuf
UpdateBufState
ClaimFileCache
ReleaseFileCache
IncReadAhead
ExtendUp
ExtendDown
Flush
TestFileCacheGrowable
RestartCheck
FloppyOpDone
WinnieOpDone
TickerEntry
UpdateProcesses
ScanBuffers
FindFreeBuf
EmptyBuffers
UnlinkAllocChain
LinkAllocChain
UnlinkFileChain
LinkFileChain
LessValid
MoreValid
WaitForControllerFree
DiscWriteBehindWait
DriveWriteBehindWait
LockDisc
FindDisc
FreeFcb
ClaimController
ReleaseController
AddPair
FragLeft
DefFileFrag (veneer to FileFrag)
......@@ -11,13 +11,13 @@
GBLS Module_HelpVersion
GBLS Module_ComponentName
GBLS Module_ComponentPath
Module_MajorVersion SETS "3.57"
Module_Version SETA 357
Module_MajorVersion SETS "3.58"
Module_Version SETA 358
Module_MinorVersion SETS ""
Module_Date SETS "25 Mar 2013"
Module_ApplicationDate SETS "25-Mar-13"
Module_Date SETS "01 Apr 2013"
Module_ApplicationDate SETS "01-Apr-13"
Module_ComponentName SETS "FileCore"
Module_ComponentPath SETS "castle/RiscOS/Sources/FileSys/FileCore"
Module_FullVersion SETS "3.57"
Module_HelpVersion SETS "3.57 (25 Mar 2013)"
Module_FullVersion SETS "3.58"
Module_HelpVersion SETS "3.58 (01 Apr 2013)"
END
/* (3.57)
/* (3.58)
*
* This file is automatically maintained by srccommit, do not edit manually.
* Last processed by srccommit version: 1.1.
*
*/
#define Module_MajorVersion_CMHG 3.57
#define Module_MajorVersion_CMHG 3.58
#define Module_MinorVersion_CMHG
#define Module_Date_CMHG 25 Mar 2013
#define Module_Date_CMHG 01 Apr 2013
#define Module_MajorVersion "3.57"
#define Module_Version 357
#define Module_MajorVersion "3.58"
#define Module_Version 358
#define Module_MinorVersion ""
#define Module_Date "25 Mar 2013"
#define Module_Date "01 Apr 2013"
#define Module_ApplicationDate "25-Mar-13"
#define Module_ApplicationDate "01-Apr-13"
#define Module_ComponentName "FileCore"
#define Module_ComponentPath "castle/RiscOS/Sources/FileSys/FileCore"
#define Module_FullVersion "3.57"
#define Module_HelpVersion "3.57 (25 Mar 2013)"
#define Module_LibraryVersionInfo "3:57"
#define Module_FullVersion "3.58"
#define Module_HelpVersion "3.58 (01 Apr 2013)"
#define Module_LibraryVersionInfo "3:58"
......@@ -4490,12 +4490,11 @@ FileFrag ROUT
MOVLO FragEnd, R4 ;pick smaller of end-of-this-fragment and end-of-alloclen
]
SUB LR, R2, LR ;length of frag before FileOff
SUB LR, FileOff, LR, LSL R7 ;frag start file offset
CMPS LR, R0
MOVGT R0, LR ;signed compare as shared files may not start
SUB DiscAdjust, R2, FileOff, LSR R7 ;at frag start
SUB LR, R2, LR ;length of frag before FileOff (in sectors)
RSB LR, LR, FileOff, LSR R7 ;project back to what FileOff would be at frag start (in sectors)
CMPS LR, R0, LSR R7 ;pick larger of R0 (reexpressed in sectors) and projected start
MOVGT R0, LR, LSL R7 ;a -ve projected start is allowed as shared files may not start at frag start
SUB DiscAdjust, R2, FileOff, LSR R7
|
TEQS FileOff,FragEnd
SUBEQ FileOff,FileOff,#1 ;nasty - non sector aligned?
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment