Merge branch 'dmgass/string' into 'alpha_4'

Add string type.

See merge request dangass/plum!18

contribution_waiver.txt

@@ -91,7 +91,7 @@ By adding my name to this signature list and dating, I acknowledge:
     Date       Contributor Name/Email
     ---------  --------------------------------------------------------
     01Mar2019  Jan Novak (job.jan.novak@gmail.com)
-    07Mar2019  Daniel Mark Gass (dan.gass@gmail.com)
+    13Mar2019  Daniel Mark Gass (dan.gass@gmail.com)
 
 
 ****************

docs/about.rst

@@ -47,12 +47,8 @@ What's Next?
 
 - Add types.
 
-    - Bool
-    - Float
-    - Nil
     - IpAddr
     - MacAddr
-    - String
     - Pointer
 
 - Increase performance.

docs/alias.txt

 
 .. |API reference| replace:: :doc:`API reference </reference/index>`
 .. |Array| replace:: :class:`~plum.array.Array`
+.. |AsciiStr| replace:: :class:`~plum.str.AsciiStr`
+.. |AsciiZeroTermStr| replace:: :class:`~plum.str.AsciiZeroTermStr`
 .. |BitFields| replace:: :class:`~plum.int.bitfields.BitFields`
 .. |BitFlags| replace:: :class:`~plum.int.bitflags.BitFlags`
+.. |ByteArray| replace:: :class:`~plum.bytearray.ByteArray`
 .. |ExcessMemoryError| replace:: :class:`~plum.ExcessMemoryError`
+.. |Float| replace:: :class:`~plum.float.Float`
 .. |InsufficientMemoryError| replace:: :class:`~plum.InsufficientMemoryError`
 .. |Int| replace:: :class:`~plum.int.Int`
 .. |Memory| replace:: :class:`~plum.Memory`
+.. |Nil| replace:: :class:`~plum.nil.Nil`
 .. |pack()| replace:: :func:`~plum.pack`
 .. |PackError| replace:: :class:`~plum.PackError`
 .. |plum| replace:: :mod:`plum`
@@ -15,7 +20,10 @@
 .. |SizedArray| replace:: :class:`~plum.sizedarray.SizedArray`
 .. |SizedObject| replace:: :class:`~plum.sizedobject.SizedObject`
 .. |SizeError| replace:: :class:`~plum.SizeError`
+.. |Str| replace:: :class:`~plum.str.Str`
 .. |Structure| replace:: :class:`~plum.structure.Structure`
 .. |Tutorials| replace:: :doc:`Tutorials </tutorials/index>`
-.. |unpack| replace:: :func:`~plum.unpack`
+.. |unpack()| replace:: :func:`~plum.unpack`
 .. |UnpackError| replace:: :class:`~plum.UnpackError`
+.. |Utf8Str| replace:: :class:`~plum.str.Utf8Str`
+.. |Utf8ZeroTermStr| replace:: :class:`~plum.str.Utf8ZeroTermStr`

docs/index.rst

@@ -25,7 +25,17 @@ more complete usage examples. The following summarizes the
 +===========================+======================+=====================================================+
 | :mod:`plum.array`         | |Array|              | list of uniformly typed items                       |
 +---------------------------+----------------------+-----------------------------------------------------+
-| :mod:`plum.int`           | |Int|                | integer base class                                  |
+| :mod:`plum.bytearray`     | |ByteArray|          | array of bytes                                      |
++---------------------------+----------------------+-----------------------------------------------------+
+| :mod:`plum.float`         | |Float|              | float base class                                    |
++---------------------------+----------------------+-----------------------------------------------------+
+| :mod:`plum.float.big`     | Float32, Float64     | big endian floating point numbers                   |
++---------------------------+----------------------+-----------------------------------------------------+
+| :mod:`plum.float.little`  | Float32, Float64     | little endian floating point numbers                |
++---------------------------+----------------------+-----------------------------------------------------+
+| :mod:`plum.float.native`  | Float32, Float64     | native endian floating point numbers                |
++---------------------------+----------------------+-----------------------------------------------------+
+| :mod:`plum.int`           | |Int|                | integer (and enumeration) base class                |
 +---------------------------+----------------------+-----------------------------------------------------+
 | :mod:`plum.int.bitfields` | |BitFields|          | integer with bitfield accessors                     |
 +---------------------------+----------------------+-----------------------------------------------------+
@@ -43,8 +53,19 @@ more complete usage examples. The following summarizes the
 |                           +----------------------+-----------------------------------------------------+
 |                           | UInt8, 16, 32, 64    | native endian unsigned integers                     |
 +---------------------------+----------------------+-----------------------------------------------------+
+| :mod:`plum.nil`           | |Nil|                | no memory bytes                                     |
++---------------------------+----------------------+-----------------------------------------------------+
 | :mod:`plum.sequence`      | |Sequence|           | list of uniquely typed items                        |
 +---------------------------+----------------------+-----------------------------------------------------+
+| :mod:`plum.str`           | |Str|                | string base class                                   |
+|                           +----------------------+-----------------------------------------------------+
+|                           | |AsciiStr|           | ASCII encoded string                                |
+|                           +----------------------+-----------------------------------------------------+
+|                           | |AsciiZeroTermStr|   | ASCII encoded zero terminated string                |
+|                           +----------------------+-----------------------------------------------------+
+|                           | |Utf8Str|            | UTF-8 encoded string                                |
+|                           +----------------------+-----------------------------------------------------+
+|                           | |Utf8ZeroTermStr|    | UTF-8 encoded zero terminated string                |
++---------------------------+----------------------+-----------------------------------------------------+
 | :mod:`plum.structure`     | |Structure|          | dictionary of named, uniquely typed items           |
 +---------------------------+----------------------+-----------------------------------------------------+
-

docs/reference/index.rst

@@ -81,13 +81,20 @@ Modules
     Module                          Description
     ==============================  ================================================
     :mod:`plum.array`               list of uniformly typed items
-    :mod:`plum.int`                 integer base class
+    :mod:`plum.bytearray`           array of bytes
+    :mod:`plum.float`               float base class
+    :mod:`plum.float.big`           big endian floats
+    :mod:`plum.float.little`        little endian floats
+    :mod:`plum.float.native`        native endian floats
+    :mod:`plum.int`                 integer (and enumeration) base class
     :mod:`plum.int.big`             big endian integers
     :mod:`plum.int.bitfields`       integer with bitfield accessors
     :mod:`plum.int.bitflags`        integer with bit accessors
     :mod:`plum.int.little`          little endian integers
     :mod:`plum.int.native`          native endian integers
+    :mod:`plum.nil`                 no memory bytes
     :mod:`plum.sequence`            list of uniquely typed items
+    :mod:`plum.str`                 string
     :mod:`plum.structure`           dictionary of named, uniquely typed items
     ==============================  ================================================
 
@@ -96,11 +103,18 @@ Modules
     :hidden:
 
     array <modules/array.rst>
+    bytearray <modules/bytearray.rst>
+    float <modules/float.rst>
+    float.big <modules/float_big.rst>
+    float.little <modules/float_little.rst>
+    float.native <modules/float_native.rst>
     int <modules/int.rst>
     int.big <modules/int_big.rst>
     int.bitfields <modules/int_bitfields.rst>
     int.bitflags <modules/int_bitflags.rst>
     int.little <modules/int_little.rst>
     int.native <modules/int_native.rst>
+    nil <modules/nil.rst>
     sequence <modules/sequence.rst>
+    str <modules/str.rst>
     structure <modules/structure.rst>

docs/reference/modules/bytearray.rst

+#################################
+[plum.bytearray] Module Reference
+#################################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.bytearray
+
+.. autoclass:: ByteArray()
+
+.. autoclass:: ByteArrayType()
+
+.. seealso::
+
+    :doc:`ByteArray Tutorial </tutorials/bytearray>`
+

docs/reference/modules/float.rst

+#############################
+[plum.float] Module Reference
+#############################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.float
+
+.. autoclass:: Float()
+
+.. autoclass:: FloatType()
+
+.. seealso::
+
+    :doc:`Float Tutorial </tutorials/float>`

docs/reference/modules/float_big.rst

+#################################
+[plum.float.big] Module Reference
+#################################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.float.big
+
+.. autoclass:: Float32(x=0)
+
+.. autoclass:: Float64(x=0)

docs/reference/modules/float_little.rst

+####################################
+[plum.float.little] Module Reference
+####################################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.float.little
+
+.. autoclass:: Float32(x=0)
+
+.. autoclass:: Float64(x=0)

docs/reference/modules/float_native.rst

+####################################
+[plum.float.native] Module Reference
+####################################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.float.native
+
+
+.. class:: Float32([x], base=10)
+
+    Single precision native endian floating point number.
+
+
+.. class:: Float64([x], base=10)
+
+    Double precision native endian floating point number.

docs/reference/modules/nil.rst

+###########################
+[plum.nil] Module Reference
+###########################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.nil
+
+.. autoclass:: Nil()
+
+.. autodata:: nil
+    :annotation: = singleton instance
+
+.. seealso::
+
+    :doc:`Nil Tutorial </tutorials/nil>`
+

docs/reference/modules/str.rst

+###########################
+[plum.str] Module Reference
+###########################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.str
+
+.. autoclass:: Str(object, encoding='ascii', error='strict')
+
+.. autoclass:: StrType
+
+.. autoclass:: AsciiStr
+
+.. autoclass:: AsciiZeroTermStr
+
+.. autoclass:: Utf8Str
+
+.. autoclass:: Utf8ZeroTermStr

docs/reference/modules/structure.rst

@@ -10,6 +10,8 @@
 
 .. autoclass:: Structure(mapping, **kwargs)
 
+.. autoclass:: Varies
+
 .. autofunction:: limit_size
 
 .. autofunction:: measure_dims
@@ -18,6 +20,8 @@
 
 .. autofunction:: member
 
+.. autofunction:: switch_type
+
 .. autofunction:: transfer_dims
 
 .. seealso::

docs/release_notes.rst

@@ -8,6 +8,11 @@ Versions increment per `semver <http://semver.org/>`_ (except for alpha series).
 Alpha (0.1.0aX) Versions (only serial number X increments)
 **********************************************************
 
++ 0.1.0a4 (2019-03-13)
+    - Add ByteArray, Float, Nil, Str types.
+    - Add switch_type( ) to support varying structure member type based on
+      another structure member.
+
 + 0.1.0a3 (2019-03-07)
     - Change str() and repr() formats.
     - Remove Tuple, SizedArray, SizedObject types.

docs/tutorials/bytearray.rst

+###############################################
+[plum.bytearray] Tutorial: Using ByteArray Type
+###############################################
+
+.. include:: ../alias.txt
+
+
+The |ByteArray| type inherits :class:`bytearray` characteristics as a result of
+it being a subclass of it. The |plum| type collection includes this type for
+applications that require an array of bytes. This tutorial shows sample usages
+and the |ByteArray| type's properties.
+
+.. contents::
+    :local:
+
+
+*****************
+Out of Box Sample
+*****************
+
+By default, the |ByteArray| type consumes all bytes given to it when unpacking,
+packing or instantiating:
+
+    >>> from plum import pack, unpack
+    >>> from plum.bytearray import ByteArray
+    >>>
+    >>> unpack(ByteArray, b'\x00\x01')
+    ByteArray(b'\x00\x01')
+    >>>
+    >>> pack(ByteArray, [0, 1])
+    b'\x00\x01'
+    >>>
+    >>> ByteArray([0, 1])
+    ByteArray(b'\x00\x01')
+
+
+*******************
+Fixed Size Subclass
+*******************
+
+To lock the size of the byte array for use within a composite type such as |Structure|,
+subclass |ByteArray| and specify the number of bytes using the ``nbytes`` keyword
+argument:
+
+    >>> from plum import dump, pack, unpack
+    >>> from plum.bytearray import ByteArray
+    >>> from plum.int.little import UInt8
+    >>> from plum.structure import Structure
+    >>>
+    >>> class ByteArray1(ByteArray, nbytes=4):
+    ...     """Interpret memory as a byte array of length 4."""
+    ...
+    >>> class Struct1(Structure):
+    ...     barray: ByteArray1
+    ...     bookend: UInt8
+    ...
+    >>> struct1 = unpack(Struct1, b'\x00\x01\x02\x03\x99')
+    >>> dump(struct1)
+    +--------+------------+--------------+-------------+------------+
+    | Offset | Access     | Value        | Memory      | Type       |
+    +--------+------------+--------------+-------------+------------+
+    |        | x          |              |             | Struct1    |
+    |        |   .barray  |              |             | ByteArray1 |
+    | 0      |     [0:4]  | [0, 1, 2, 3] | 00 01 02 03 |            |
+    | 4      |   .bookend | 153          | 99          | UInt8      |
+    +--------+------------+--------------+-------------+------------+
+    >>>
+    >>> pack(Struct1, {'barray': [0, 1, 2, 3], 'bookend': 0x99})
+    b'\x00\x01\x02\x03\x99'
+
+
+**********************
+Sized Structure Member
+**********************
+
+To couple a size member to a byte array member, use the :func:`measure_size`
+and :func:`limit_size` functions. The following example demonstrates. See
+the :doc:`Sized Object Tutorial <structure/sizedobject>` for details.
+
+    >>> from plum import dump, pack, unpack
+    >>> from plum.bytearray import ByteArray
+    >>> from plum.int.little import UInt8
+    >>> from plum.structure import Structure, measure_size, limit_size
+    >>>
+    >>> class Struct2(Structure):
+    ...     size: UInt8 = measure_size(sizedmember='barray')
+    ...     barray: ByteArray = limit_size(sizemember='size')
+    ...     bookend: UInt8
+    ...
+    >>> # size member automatically filled in
+    >>> struct2 = Struct2(barray=ByteArray(24), bookend=0x99)
+    >>> dump(struct2)
+    +--------+-------------+--------------------------------------------------+-------------------------------------------------+-----------+
+    | Offset | Access      | Value                                            | Memory                                          | Type      |
+    +--------+-------------+--------------------------------------------------+-------------------------------------------------+-----------+
+    |        | x           |                                                  |                                                 | Struct2   |
+    |  0     |   .size     | 24                                               | 18                                              | UInt8     |
+    |        |   .barray   |                                                  |                                                 | ByteArray |
+    |  1     |     [0:16]  | [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |           |
+    | 17     |     [16:24] | [0, 0, 0, 0, 0, 0, 0, 0]                         | 00 00 00 00 00 00 00 00                         |           |
+    | 25     |   .bookend  | 153                                              | 99                                              | UInt8     |
+    +--------+-------------+--------------------------------------------------+-------------------------------------------------+-----------+
+    >>>
+    >>> dump(unpack(Struct2, b'\x01\x02\x99'))
+    +--------+------------+-------+--------+-----------+
+    | Offset | Access     | Value | Memory | Type      |
+    +--------+------------+-------+--------+-----------+
+    |        | x          |       |        | Struct2   |
+    | 0      |   .size    | 1     | 01     | UInt8     |
+    |        |   .barray  |       |        | ByteArray |
+    | 1      |     [0:1]  | [2]   | 02     |           |
+    | 2      |   .bookend | 153   | 99     | UInt8     |
+    +--------+------------+-------+--------+-----------+
+
+
+**********
+Properties
+**********
+
+The |ByteArray| constructor accepts the same argument variations as :class:`bytearray`:
+
+    >>> from plum.bytearray import ByteArray
+    >>>
+    >>> # iterable of ints
+    >>> ByteArray([0, 1, 2])
+    ByteArray(b'\x00\x01\x02')
+    >>>
+    >>> # string
+    >>> ByteArray('Hello World!', encoding='ascii', errors='strict')
+    ByteArray(b'Hello World!')
+    >>>
+    >>> # bytes or buffer
+    >>> ByteArray(b'\x00\x01\x02')
+    ByteArray(b'\x00\x01\x02')
+    >>>
+    >>> # integer size
+    >>> ByteArray(3)
+    ByteArray(b'\x00\x00\x00')
+    >>>
+    >>> # empty
+    >>> ByteArray()
+    ByteArray(b'')
+
+
+|ByteArray| types follow all other :class:`bytearray` method behaviors.

docs/tutorials/float.rst

+########################################
+[plum.float] Tutorial: Using Float Types
+########################################
+
+.. include:: ../alias.txt
+
+
+The |Float| types inherit :class:`float` characteristics as a result of
+it being a subclass of it. The |plum| type collection includes |Float| type
+variants for packing and unpacking floating point numbers. This tutorial
+shows sample usages and properties of |Float| types.
+
+.. contents::
+    :local:
+
+
+*******************
+Packing / Unpacking
+*******************
+
+The :mod:`plum.float` subpackage includes three modules, one for each endian
+byte order: :mod:`plum.float.big`, :mod:`plum.float.little`, and :mod:`plum.native`
+(byte order follows host architecture). Each module provides a single precision and
+double precision float type (``Float32`` and ``Float64`` respectively) for the
+particular byte order.
+
+    >>> from plum import pack, unpack
+    >>> from plum.float.little import Float32
+    >>>
+    >>> unpack(Float32, b'\x00\x00\x00\x00')
+    Float32(0.0)
+    >>>
+    >>> pack(Float32, 0.0)
+    b'\x00\x00\x00\x00'
+    >>>
+    >>> f = Float32()
+    >>> f
+    Float32(0.0)
+    >>> pack(f)
+    b'\x00\x00\x00\x00'
+
+
+**************
+Half-Precision
+**************
+
+To create half precision floating point types (16 bits in size), subclass |Float| and specify
+``2`` for the number of bytes and either ``'big'`` or ``'little'`` for the byte order:
+
+    >>> from plum import dump, pack, unpack
+    >>> from plum.float import Float
+    >>>
+    >>> class Float16(Float, nbytes=2, byteorder='little'):
+    ...     """Half precision little endian floating point number."""
+    ...
+    >>> unpack(Float16, b'\x00\x00')
+    Float16(0.0)
+    >>>
+    >>> pack(Float16, 0.0)
+    b'\x00\x00'
+
+
+**********
+Properties
+**********
+
+The float types accept the same argument variations as :class:`float` and share
+the same properties:
+
+    >>> from plum.float.little import Float32
+    >>>
+    >>> # default to 0
+    >>> Float32()
+    Float32(0.0)
+    >>>
+    >>> # accept string
+    >>> Float32("10")
+    Float32(10.0)
+    >>>
+    >>> # include all float properties/methods
+    >>> f = unpack(Float32, b'\x00\x00\x00\x00')
+    >>> f.as_integer_ratio()
+    (0, 1)
+    >>> f.conjugate()
+    0.0
+    >>> f.hex()
+    '0x0.0p+0'
+    >>> Float32.fromhex(f.hex())
+    Float32(0.0)
+    >>> f.imag
+    0.0
+    >>> f.is_integer()
+    True
+    >>> f.real
+    0.0

docs/tutorials/index.rst

@@ -5,6 +5,10 @@
 .. toctree::
     :maxdepth: 2
 
-    Arrays <array/index>
-    Structures <structure/index>
+    Array <array/index>
+    ByteArray <bytearray>
+    Float <float>
+    Nil <nil>
+    String <string/index>
+    Structure <structure/index>
     Utilities <utilities/index>

docs/tutorials/nil.rst

+###################################
+[plum.nil] Tutorial: Using Nil Type
+###################################
+
+.. include:: ../alias.txt
+
+
+The |Nil| type consumes no memory bytes when unpacked and produces no memory
+bytes when packed. Seldom needed, but important when a dynamic structure needs
+a placeholder. This tutorial shows a sample usage and the |Nil| type's properties.
+
+.. contents::
+    :local:
+
+
+************
+Sample Usage
+************
+
+One application where |Nil| may become necessary is when a structure member's
+datatype depends on another structure member. The :func:`switch_type` function
+as shown in the following example demonstrates a use case. The ``data`` member's
+type depends on the value of the ``datatype`` member as defined by the mapping
+passed to :func:`switch_type`. When ``datatype`` equals ``0``, no memory bytes
+are expected for the ``data`` member. When ``datatype`` equals ``1`` or ``2``,
+memory bytes for a ``UInt8`` or ``UInt16`` (respectively) are expected for the
+``data`` member:
+
+    >>> from plum import unpack, dump
+    >>> from plum.int.little import UInt8, UInt16
+    >>> from plum.nil import Nil
+    >>> from plum.structure import Structure, Varies, switch_type
+    >>>
+    >>> class Struct1(Structure):
+    ...     datatype: UInt8
+    ...     data: Varies = switch_type('datatype', {0: Nil, 1: UInt8, 2: UInt16})
+    ...
+    >>> dump(unpack(Struct1, b'\x00'))
+    +--------+-------------+-------+--------+---------+
+    | Offset | Access      | Value | Memory | Type    |
+    +--------+-------------+-------+--------+---------+
+    |        | x           |       |        | Struct1 |
+    | 0      |   .datatype | 0     | 00     | UInt8   |
+    |        |   .data     | nil   |        | Nil     |
+    +--------+-------------+-------+--------+---------+
+    >>>
+    >>> dump(unpack(Struct1, b'\x01\x02'))
+    +--------+-------------+-------+--------+---------+
+    | Offset | Access      | Value | Memory | Type    |
+    +--------+-------------+-------+--------+---------+
+    |        | x           |       |        | Struct1 |
+    | 0      |   .datatype | 1     | 01     | UInt8   |
+    | 1      |   .data     | 2     | 02     | UInt8   |
+    +--------+-------------+-------+--------+---------+
+
+
+**********
+Properties
+**********
+
+During instantiation or packing, the |Nil| type accepts either ``None``,
+the singleton ``nil`` object, or nothing:
+
+    >>> from plum.nil import Nil, nil
+    >>>
+    >>> Nil(None)
+    Nil()
+    >>> Nil(nil)
+    Nil()
+    >>> Nil()
+    Nil()
+
+
+Instantiation of |Nil| always yields the singleton ``nil`` object:
+
+    >>> x = Nil()
+    >>> y = Nil()
+    >>> x is y
+    True
+    >>> x is nil
+    True
+    >>> y is nil
+    True
+