GitLab

The PipeFS dynamic area was being created with AP 0, granting usermode
full read/write/execute access. Attempt to restrict it to just the
minimum required permissions - privileged read/write for PipeFS access,
and usermode read for TaskWindow (or other potential UpCall_Sleep
handlers) checking pollwords.

This isn't an ideal situation, both because we're exposing the contents
to everyone, and because systems which are using the long descriptor
page table format will be forced to use the kernel's AP1 emulation to
provide usermode read access, adding unnecessary overhead to pollword
checks.

Moving the pollwords to the RMA could be one solution to this (although
it will re-introducing the possibility for user code to scribble on
them)

Version 0.25. Tagged as 'PipeFS-0_25'

When handling fsfile_ReadInfo, PipeFS wants to wait for any writer to
close the file before returning the file information. To do this it
repeatedly calls FindFileOrDirectory & blockifwriting in a loop until
that condition is reached (or the file ceases to exist). However the
call to FindFileOrDirectory was replacing the filename in R1 with a the
internal file handle (i.e. a pointer to a struct), so on the second loop
round it would always exit out with a "file not found" response.

This bug had the effect that attempting to read the size of a file (e.g.
using OS_File 5 or *FileInfo) which is open for write would always
report that the file doesn't exist, even if the blockifwriting call
successfully blocks until the writer closes the file.

Restoring R1 before looping back round will give the behaviour the
programmer originally intended, for better or worse (e.g. current
versions of FileSwitch use fsfile_ReadInfo to check the file before
trying to open it, so trying to use OPENIN on a file which is open for
write will unintentionally block - a bug that was hidden on earlier
versions of RISC OS because prior to TaskWindow 0.65, TaskWindow didn't
make any attempt to wait until the pollword was non-zero before
returning, so PipeFS would pretty quickly exit out with a harmless "file
not found" response due to the register corruption bug)

Add debug messages to all FS entry points, and call Debug_Open &
Debug_Close

This fixes a problem where a reader which is asleep waiting for more
data to arrive won't be woken when the writer closes its end of the
pipe.

Behaviour 1:

* Two taskwindows open the same pipe, one for reading, one for writing
* Writer writes data to the pipe
* Writer closes the pipe
* Reader enters an OS_BGet loop to read until EOF

Following the above steps, the reader will be able to read all of the
data and the read loop will terminate.

Behaviour 2:

* Two taskwindows open the same pipe, one for reading, one for writing
* Writer writes data to the pipe
* Reader enters an OS_BGet loop to read until EOF
* Wait for the reader to read all the data; it'll go to sleep waiting
for more
* Writer closes the pipe

Following the above steps, the reader will remain asleep inside OS_BGet,
even through the writer has signalled that it's finished with the pipe
by closing it. This looks like a textbook race condition bug; if the
writer closed the file with slightly different timing (e.g. while there
are still unread bytes in the pipe, or while the pipe is empty but
before the reader calls OS_BGet and is put to sleep) then the reader
will behave correctly.

This change fixes things so that any reader will be woken when the
writer closes its end of the pipe.

This separates the "reads blocked" pollword from the Size variable,
without altering the behaviour. This is to allow for future changes
which will alter the behaviour of the readblocked pollword.

Since PipeFS registers with FileSwitch with both dontuseload and dontusesave there's no need to implement the corresponding FSEntry_File subreasons. This might have been left around because RISC OS 2 ignored FS info word flags b23-16, but we're not worrying about RISC OS 2 compatibility now.

Version 0.24. Tagged as 'PipeFS-0_24'

PipeFS was carefully preserving R6 (the special field pointer, which is 0 for PipeFS) in the places where FileSwitch expects to get the leafname back when *OPT1,2 is enabled, resulting in a NULL pointer dereference in FileSwitch. Update R6 per the spec.
Delete old switch justreturnextent from version 0.06.
Delete unused Help & Syntax, these are now in Resources/UK.

Version 0.23. Tagged as 'PipeFS-0_23'

Use objasm {TRUE} and {FALSE}.
Get error numbers from Hdr:NewErrors and file types from Hdr:FileTypes.
Parameterise PipeFS_BufferSize and PipeFS_AllocSize.
Deduce international_help switch locally since International_Help is a bitfield in Hdr:ModHand.
Use Push/Pop macros.

Version 0.22. Tagged as 'PipeFS-0_22'

Detail:
  PipeFS uses UpCall 6 (Sleep) to signal that the reading process can sleep,
using the size of the file as a pollword.  Thus when any data is written into
the pipe, the sleeping process will be woken up.  If an attempt is made to
close the file handle, UpCall 7 (Sleep No More) is broadcast to allow anyone
to object to the closure occurring.  Pipe_Close wasn't checking the error
return from making this call, which was an error.  It will now propogate this
error up to fileswitch upon an attempt being made to close the file while a
process is sleeping on it.

  32-bitness involved changing a load of TEQPs for RestPSRs, and fixing some
functions which had forgotten to use [sp, #Proc_RegOffset].

  Also added !Mk and added stripdepend to MkClean.

Admin:
  Tested on dev board and RiscPC running a fairly recent Ursula build.
Version 0.21. Tagged as 'PipeFS-0_21'

Detail:
    Converted to use ObjASM and shared makefiles. Added MkRes and MkExp obey
files.

Admin:
    Built. This module is *not* yet 32-bit compatible.

Version 0.20. Tagged as 'PipeFS-0_20'

Detail:
  Use of the RMA by PipeFS could lead to fragmentation, and is slower
  than a private dynamic area as the RMA's a very busy heap. The new
  dynamic area is set to 16MB maximum size, called "PipeFS", and the
  code is compile-time switched, not run-time.

  PipeFS has no block handling code so its get/put byte routines are
  called by the OS in a tight loop. Unfortunately for each byte it
  reads/writes it issues UpCall_ModifyingFile, and moreover does this
  without specifying a filename, so *all* directories are rescanned -
  for every single byte read or written to PipeFS. This is why PipeFS
  is so extremely slow prior to this version. Now, a callback is set,
  and the UpCall only issued when this goes off - after any block
  transfers, for example. A typical effect is a 171K file save from a
  text editor to PipeFS used to take around 20 seconds, and now takes
  less than a quarter of a second. Streaming data through PipeFS is
  now a sensible option.

  There are two variants of the callback code; one maintans an array
  of filename pointers and goes through the list each time the call-
  back fires; the other just issues a general UpCall each time. The
  first one should work OK, and is there in case anyone ever works
  out how to use R1 properly for the UpCall.
Admin:
  Copied files into PipeFS up to the DA limit and verified sensible
  behaviour at this point; memory released as expected on file
  removal; files appear to be intact; MP3 streaming at 128kbps from
  network or disc through PipeFS is reliable; Filer window behaviour
  with "full info" on and Task Manager display of dynamic area size
  act as expected.

  By default, the compile-time options mean the module will only run on
  RISC OS 3.5 or later. Tested on RISC OS 3.70 only.

Version 0.19. Tagged as 'PipeFS-0_19'

Detail
------
  * Under the resources: rule the dependency resources-{$CMDHELP} was
    spelt as resource-{$CMDHELP} which is not a valid dependency.
Admin
-----
  * Tested on Ursula build.

Version 0.18. Tagged as 'PipeFS-0_18'

Moved to srccommit.
Added inclusion of Machine header for new CMOS header.

Version 0.17. Tagged as 'PipeFS-0_17'