Next

• Fix writing 0-terminated multi-byte / UTF32 strings when using SpanReader and SpanWriter of Syroot.BinaryData.Memory.

• First release of Syroot.BinaryData.Memory package focussing new Span functionality.
• Common functionality has been moved to Syroot.BinaryData.Core package.
• Fixes missing thread-safety when checking enumeration values and serializing objects.

• Added methods to BinaryStream to support writing data not requiring any of the data codings configurable by the BinaryStream properties without having to use extension methods.
• Added MoveAsync to support simulating seeks asynchronously.
• Fixed a typo of StreamExtensions.Write7BitInt32Async writing multiple values. The correct name is now Write7BitInt32sAsync.
• The signed package has been dropped, this legacy use case is no longer supported.

Package split

• The NuGet package Syroot.IO.BinaryData has been split into Syroot.BinaryData and Syroot.BinaryData.Serialization, but continuing with version 5 to prevent confusion.
  • Syroot.BinaryData includes the core features of the library, excluding Read/WriteObject methods and any other serialization features.
  • Syroot.BinaryData.Serialization is dependent on Syroot.BinaryData, and adds the Read/WriteObject methods and serialization features.
  • Signed packages are available as Syroot.BinaryData.Signed and Syroot.BinaryData.Serialization.Signed correspondingly.
• The namespaces Syroot.BinaryData.Serialization and Syroot.BinaryData.Extensions have been removed. All features now reside in the main Syroot.BinaryData namespace.
• The IO package name prefix has been dropped, as it was meaningless anyway after version 3 removed the corresponding sub namespace.
  • Projects using the old Syroot.IO.BinaryData package will not receive updates automatically. Due to the drastic API changes, an automatic update is not recommended anyway as existing code will require changes.

New features

• Asynchronous programming is now supported with new Async methods. In the following, both synchronous and asynchronous methods are meant if the name of only the synchronous method is mentioned.
• ReadMany / WriteMany has been added to invoke a reading or writing method for the given number of times.
  • Multi-value overloads (like ReadInt32s) are now simply calling these new methods, as they've previously done the same thing internally with duplicate logic.
  • ReadBytes is an exception to this for performance reasons.
• Methods writing 7-bit encoded Int32 values, like those used for .NET string length prefixes, are now public.
• Additional named methods for the many Write overloads, e.g. WriteInt32, to prevent errors from mistakenly used overloads. They simply forward to the classic Write overloads.
• There are new non-generic overloads for Read/WriteEnum and Read/WriteObject which allow passing the type to read or write as a parameter, determining the type at runtime rather than compile-time. (These were previously internal to the library but have been made public after some polishing.)
• WriteEnums methods now accept an IEnumerable parameter of values rather than being limited to IEnumerable<T> (the type is validated by the logic afterwards anyway).
• Stream.ReadBytes now throws an EndOfStreamException if the requested number of bytes could not be read, behaving like the other ReadXxx methods (and unlike the default Stream.Read method).
• The ByteConverter instances use the new Span<T> types for optimal performance. Since they're used internally by all other features of the library, this speeds up reading and writing binary data in general.

Dropped features

• BinaryDataReader and BinaryDataWriter have been removed. The replacement is the Stream-wrapping class BinaryStream, if the Stream extension methods should be insufficient.
  • It supports the features the previous BinaryDataReader/Writer have been reduced to in a much cleaner fashion: It remembers default Boolean / DateTime and String formats, String encoding and byte order to reduce the number of parameters required to be specified for each call to a Stream extension method. Due to being a Stream itself, all extension methods are still available on it to override the remembered defaults for a single call.
• Offset has been removed. The size and contents of an offset are highly implementation specific and fit badly into the generic approach of the library.
• "Thread-safety" has been removed. This decision was made because Stream locking has to be handled in a meaningful (and more performant) way by the user at a higher level, and not at a low level reading / writing function. .NET's SynchronizedStream should be created if a similar behavior comparable to old versions of the library is required.

Other breaking changes

• In an effort to further shorten commonly used class and enumeration names, the following name changes were made:
  • BooleanDataFormat has been renamed to BooleanCoding
  • DateTimeDataFormat has been renamed to DateTimeCoding
  • StringDataFormat has been renamed to StringCoding
    • DynamicByteCount has been renamed to VariableByteCount to prevent confusion with the DLR dynamicfeatures.
  • SeekTask has been renamed to Seek to prevent confusion with asynchronous Task features.
• ByteConverter instances now make use of Span<T> for even better performance. Due to this, you can no longer pass in a startIndex manually, but have to create a view on the span.

• Adds deprecation warning to the NuGet package in favor of the new Syroot.BinaryData package. No other changes were made.

• Fix big endian Single and Double conversion through ByteConverter.

• Fixes serialization of arrays (https://github.com/Syroot/BinaryData/issues/11)
• Fixes serialization of UInt64 values (https://github.com/Syroot/BinaryData/issues/12)

• Fixes serializing UInt64 values which were erroneously written as UInt32 values (https://github.com/Syroot/BinaryData/issues/7)
• Adds feature to write strings with StringDataFormat.Raw (https://github.com/Syroot/BinaryData/issues/9)

• Fixes members of instances (de-)serialized with Read/WriteObject methods not being read or written in any deterministc order.
• Members are now sorted alphabetically (ordinally).
• A custom, prioritized order can be set with the BinaryMemberAttribute.Order property.

Note that this breaks reading objects written with earlier versions and that you have to specify the order of members manually first before being able to read these again.

Please consult the wiki page for more information.

• Add method to write multiple strings. (https://github.com/Syroot/BinaryData/issues/5)
• Allow reading and writing objects even if the stream is not seekable. (https://github.com/Syroot/BinaryData/issues/3)

General

• The assembly now targets .NET Standard 2.0 rather than 1.5. Support for .NET 4.5 stays unchanged.
• The new NuGet package Syroot.IO.BinaryData.Signed provides a signed version of the library.
• Unit tests have been added to verify the correctness of all methods.

Extension methods

The functionality of BinaryDataReader and BinaryDataWriter has been rewritten as extension methods for Stream instances, available through the Syroot.BinaryData.Extensions namespace.

• It is thus no longer required (but still possible) to create reader or writer instances.

ByteConverter

The new ByteConverter class is an endian-aware, extended version of the .NET BitConverter class. It is used internally to convert values from and to byte arrays, and can also be used externally.

• A converter handling little endian is available as ByteConverter.LittleEndian
• A converter handling big endian is available as ByteConverter.BigEndian
• A converter handling the executing system's byte order is available as ByteConverter.System.
• The static ByteConverter.GetConverter(ByteOrder) method can be used to retrieve a converter corresponding to the passed byte order.
• If possible, parts do not rely on .NET's internal BitConverter, eliminating the need to reverse byte arrays, using optimized methods to convert data instead.
• It can be passed to the extension methods to one-time change the endianness of the data to be read or written.
• ByteOrder.System has been added, signaling to effectively use the executing system's byte order.

Stream navigation

• The Align() method now returns the resulting position in the stream.
  • BinaryDataWriter.Align has a new optional parameter grow. If the alignment moved the position behind the end of the stream and this parameter is true, the stream will be resized up to the current position.
• BinaryDataWriter now also has an EndOfStream and a Length property. Length can also be set to resize the underlying stream.

Thread safety

• Reading strings is now thread-safe.
• Reading multiple values at once is now thread-safe.

Breaking changes

• The extension methods required bigger rewrites, changing the IBinaryConverter interface which now no longer takes BinaryDataReader and BinaryDataWriter instances, but the base Stream instances instead. Additonally, the ByteConverter to use when parsing multi-byte data is passed as a new parameter. You can update your implementations by using the new extension methods directly instead of creating a reader or writer instance.

• ByteOrderHelper.SystemByteOrder has been removed. This is now accessible through the ByteConverter.System.ByteOrder property.

• BinaryDataReader/Writer.NeedsReversion has been removed. Check the need to reverse multi-byte data with ByteConverter.ByteOrder != ByteConverter.System.ByteOrder, or use the ByteConverter property of each instance directly.

• String parsing has been completely rewritten to fix faulty encoding logic in the previous implementation. This can cause different, but now correct behavior when reading Unicode strings.

• The binary format enums and their members have been renamed in an effort to shorten names and clarify meaning (especially for StringDataFormat).

  Old Name	New Name
  BinaryBooleanFormat	BooleanDataFormat
  BinaryBooleanFormat.NonZeroByte	BooleanDataFormat.Byte
  BinaryBooleanFormat.NonZeroWord	BooleanDataFormat.Word
  BinaryBooleanFormat.NonZeroDword	BooleanDataFormat.Dword
  BinaryDateTimeFormat	DateTimeDataFormat
  BinaryStringFormat	StringDataFormat
  BinaryStringFormat.VariableLengthPrefix	StringDataFormat.DynamicByteCount
  BinaryStringFormat.ByteLengthPrefix	StringDataFormat.ByteCharCount
  BinaryStringFormat.WordLengthPrefix	StringDataFormat.Int16CharCount
  BinaryStringFormat.DwordLengthPrefix	StringDataFormat.Int32CharCount
  BinaryStringFormat.NoPrefixOrTermination	StringDataFormat.Raw

• Add support for 64-bit CTime (time_t) format.

• Added parameterless BinaryDataWriter.TemporarySeek() method, remembering and seeking back to the current position as known from BinaryDataReader.TemporarySeek().

• Added automated support for loading and writing classes and structs with ReadObject, ReadObjects and WriteObject, which behavior can be configured with BinaryObjectAttribute and BinaryMemberAttribute. S. the wiki for more details and documentation.
• NeedsReversion member is now public to help in respecting endianness when implementing extension methods.

Breaking changes when updating from 1.x versions:

• Classes now reside in the Syroot.BinaryData rather than the Syroot.IO namespace. Simply searching for and replacing the namespace should suffice.
• BinaryBooleanFormat.NonZeroByte, BinaryDateTimeFormat.NetTicks and BinaryStringFormat.VariableLengthPrefix are now the default (0) values of the enums to reflect the default formats of .NET and the library . This should not be a problem if you did not use the underlying values directly.
• The library now requires at least .NET Standard 1.5 rather than .NET Standard 1.1. The support for .NET 4.5 is unchanged.

An example commit making use of the new features and applying the required changes to work with 2.0.0 can be found here.

• Fix reading and writing of unsigned enum values.

• Added support for writing multiple values from IEnumerable<T> instances rather than just arrays.
• Sped-up writing of multiple values by eliminating unrequired array access.

• Fixed issue in enum validation method for big enums.
• Added DebuggerDisplayAttribute to directly show the current position in the underlying stream of BinaryDataReader and BinaryDataWriter.