GitLab

  Fix inconsistency in handling illegal byte sequences.
  Convert surrogate codepoints and U+FFFE, U+FFFF to U+FFFD.
  Also, a few extra mappings.
Detail:
  enc_utf8.c: 0x80 is a continuation byte. Map stray ones to U+FFFD.
              Reset the count of expected continuation bytes to 0 when
              encountering illegal byte sequences. Previously, if the character
              callback returned non-zero, this count would not be reset, thus
              leaving the codec in an inconsistent state. Additionally, we no
              longer consume the illegal continuation byte: instead, we process
              it as a start byte next time round.
  encoding.c: Do not load extension tables for ISO-8859-{1,2,9,10,15,16}
              If these are needed, it's probably best that different charset
              names are used rather than overloading 8859-n.
  iso2022.c:  Permit SS2/3 escape sequences for EUC encode/decode.
              Disable C1 chara...

Version 0.55. Tagged as 'Unicode-0_55'

  (This isn't integrated with ISO 2022 processing though - it's standalone).
* Added a Dstroke -> Eth second-attempt conversion in various write routines,
  primarily for ISO 6937 -> Latin1 conversion (ISO 6937 unifies them).

Version 0.54. Tagged as 'Unicode-0_54'

Detail:
  Builds on 32-bit machine even with 26-bit environment.
  Fixed c.encoding so that it builds with newer tools.
Admin:
  Works in Baseline 500 build.

Version 0.53. Tagged as 'Unicode-0_53'

> Summary:
>   Merged changes from branch tree
>   Reversed previous change
> Detail:
>
> * Merged a few changes/fixes from the Unicode library in
>  branch's tree.
>
> * Reversed Steve's change from version 0.50. The change wasn't
>  necessary, and with the changed definition of NOT_USED in this
>  version, it compiles fine with cc 5.45.
>
> * Small comment change in unix.c. It now states that the file
>  isn't equivalent to any in the branch tree.
>
> Admin:
>   Built and briefly tested using TextConv utility on Risc PC.

Version 0.52. Tagged as 'Unicode-0_52'

Changed (C) message ready for release with C tools.
Module form renamed to "unicodelibm".

Version 0.51. Tagged as 'Unicode-0_51'

Detail:
  This version now builds with cc-5_45. Note: it has not been verified as
  actually functioning correctly.
Admin:
  Tested in DSL Baseline build.

Version 0.50. Tagged as 'Unicode-0_50'

Detail:
  Regenerated this header file from the iso publications on the web
  to make sure that it's up to date and spotted some syntax errors
  which must have been introduced in the generation of the previous
  version of this header file. No other differences.
Admin:
  Should work.

Version 0.49. Tagged as 'Unicode-0_49'

  Fixed a comparison of a plain char (signedness issue)
Admin:
  These were from NCBrowser 5.28 too - but got forgot in the last checkin :-(
  I've not tried using this library.


Version 0.48. Tagged as 'Unicode-0_48'

Detail:
  Buffer overrun fixed; some buffer counting problems fixed too.  There is
    now helpful initialisation and tidyup routines you can call too (called
    encoding_initialise and encoding_tidyup)
Admin:
  I've built this with cc 5.45 in basic build environment - it built OK.
  This source code now matches that in NCBrowser 5.28.


Version 0.47. Tagged as 'Unicode-0_47'

Detail:
  Added some changes from Unicode lib in branched tree.  All basically
   type changes.  This appears to be because other compilers are
   more picky about types than armcc.

Admin:
  Will add 0.46 VersionNum file into branched tree, and all will be
   syncronised fully.


Version 0.46. Tagged as 'Unicode-0_46'

Detail:
  Copyright messages changed from E-14 to Pace throughout, filename
   placed at top of file throughout, instead of in just some files.

  Merged branch's fixes into our code base, plus made it possible to
   get nice debug output in branched tree, and vfprintf() to stderr in
   RISC OS tree.  Exactly same source used in branched tree now (apart
   from OS specific files riscos.c and unix.c moving into layers
   directory structure).

Admin:
  Built for branched, both Unix and RISC OS.
  Built in RISC OS tree, and compiled into TextConv.


Version 0.45. Tagged as 'Unicode-0_45'

Minor change to SCSU encoder. If someone feeds us a byte-order mark (U+FEFF) as the first character of a stream, we encode it as SQU FE FF. This is recommended by revision 3.1 of Unicode Technical Report #6.

Version 0.44. Tagged as 'Unicode-0_44'

ISO 8859-8 is now ISO-IR 198 (05/14).

Version 0.43. Tagged as 'Unicode-0_43'

Improved handling of SIP ideographs.
Added ISO-8859-11 (csISOLatinThai).
Renamed Latin13 to Latin7.

Version 0.42. Tagged as 'Unicode-0_42'

UTF-8 encoder handles out-of-space conditions correctly.
ISO 2022 encoder/decoder doesn't try to load table 7E (the null table).
encoding_new() does identify a null MIME string with auto-detect Japanese.
UnicodeData 3.0.0 imported.

Version 0.41. Tagged as 'Unicode-0_41'

Changed default language of Latin-5 (ISO 8859-9) from English to Turkish.

Version 0.40. Tagged as 'Unicode-0_40'

Modified encoding.c so that Chinese encodings use the correct
country code as a secondary tag to the language code so that
we can distinguish Chinese Simplified and Traditional.

Version 0.39. Tagged as 'Unicode-0_39'

Fixed encoding_table_remove_unused() which totally failed to work correctly would most likely crash as soon as it tried to free any tables.

Verified that fixed version does work within branched.

Version 0.38. Tagged as 'Unicode-0_38'

Fixed encoding_new() so that it returns NULL if an encoding is chosen that does not have an encoding structure with it.

e.g. encoding 0 or AutoDetectJP.

Version 0.37. Tagged as 'Unicode-0_37'

Changed encoding_table_remove_unused() so that it takes a parameter giving the depth from which to start purging.

Fixed ISO2022 write code to free search tables.
Added unix.c for unix-targeted builds.
Updated cross-compile build.
Added unix-targeted build of library and textconv tool
in ccsolaris directory.

Version 0.36. Tagged as 'Unicode-0_36'

x-Current encoding didn't work if International 1.50 wasn't loaded.
Adjusted various ISO 2022 escape sequence tables to change prioritisation.
ISO 2022 writer won't shift character set until required.

Version 0.35. Tagged as 'Unicode-0_35'

Added encoding_set_flags().
Proper handling of byte order marks in UTF-16 and UCS-4.
Fixed UTF-16 surrogate writing.
Adjusted various MIME charset identifiers.
Incorporated latest Unicode Character Database (2.1.8).
Added "current system alphabet" encoding.
Created "TextConv" command line character set conversion utility.

Version 0.34. Tagged as 'Unicode-0_34'

Created new file riscos.c for RISC OS specific functions. Rest of library should remain portable. Moved function to load a map file into that new file. Added #defines for directory separator and wild card characters and updated the various file names.

Version 0.33. Tagged as 'Unicode-0_33'

DOS code page 866 (Russian) added.

Version 0.32. Tagged as 'Unicode-0_32'

More messing with the state machine - some state transitions were still totally wrong, causing some EUC pages to come up as Shift-JIS. Maybe this one will work?

Version 0.31. Tagged as 'Unicode-0_31'

Version 0.30. Tagged as 'Unicode-0_30'

Version 0.29. Tagged as 'Unicode-0_29'

Updated all the writers to ignore the NULL_UCS4 character (as had been previously added to the iso2022_escapes case). Any new writers should flush any pending characters they may have at this point.

Also udpated enc_UCS4.c and utf8.c to turn all illegal characeters
(top bit set) into FFFD.

Version 0.28. Tagged as 'Unicode-0_28'

Added new function encoding_default_mime_type() which given an encoding number returns the first mime type from the matching entry in the table.

Version 0.27. Tagged as 'Unicode-0_27'

Version 0.26. Tagged as 'Unicode-0_26'

Version 0.25. Tagged as 'Unicode-0_25'

Version 0.24. Tagged as 'Unicode-0_24'

Changed Shift-JIS 7-bit set to match MS CP932. It is now ASCII, except that character &5C is yen instead of backslash.

Version 0.23. Tagged as 'Unicode-0_23'

Version 0.22. Tagged as 'Unicode-0_22'

Version 0.21. Tagged as 'Unicode-0_21'

Version 0.20. Tagged as 'Unicode-0_20'

Version 0.19. Tagged as 'Unicode-0_19'

Version 0.18. Tagged as 'Unicode-0_18'

Version 0.16. Tagged as 'Unicode-0_16'