Commit 9e617470 authored by Dan Gass's avatar Dan Gass
Browse files

Merge branch 'dmgass/string' into 'alpha_4'

Add string type.

See merge request dangass/plum!18
parent 7670d9b7
......@@ -91,7 +91,7 @@ By adding my name to this signature list and dating, I acknowledge:
Date Contributor Name/Email
--------- --------------------------------------------------------
01Mar2019 Jan Novak (job.jan.novak@gmail.com)
07Mar2019 Daniel Mark Gass (dan.gass@gmail.com)
13Mar2019 Daniel Mark Gass (dan.gass@gmail.com)
****************
......
......@@ -47,12 +47,8 @@ What's Next?
- Add types.
- Bool
- Float
- Nil
- IpAddr
- MacAddr
- String
- Pointer
- Increase performance.
......
.. |API reference| replace:: :doc:`API reference </reference/index>`
.. |Array| replace:: :class:`~plum.array.Array`
.. |AsciiStr| replace:: :class:`~plum.str.AsciiStr`
.. |AsciiZeroTermStr| replace:: :class:`~plum.str.AsciiZeroTermStr`
.. |BitFields| replace:: :class:`~plum.int.bitfields.BitFields`
.. |BitFlags| replace:: :class:`~plum.int.bitflags.BitFlags`
.. |ByteArray| replace:: :class:`~plum.bytearray.ByteArray`
.. |ExcessMemoryError| replace:: :class:`~plum.ExcessMemoryError`
.. |Float| replace:: :class:`~plum.float.Float`
.. |InsufficientMemoryError| replace:: :class:`~plum.InsufficientMemoryError`
.. |Int| replace:: :class:`~plum.int.Int`
.. |Memory| replace:: :class:`~plum.Memory`
.. |Nil| replace:: :class:`~plum.nil.Nil`
.. |pack()| replace:: :func:`~plum.pack`
.. |PackError| replace:: :class:`~plum.PackError`
.. |plum| replace:: :mod:`plum`
......@@ -15,7 +20,10 @@
.. |SizedArray| replace:: :class:`~plum.sizedarray.SizedArray`
.. |SizedObject| replace:: :class:`~plum.sizedobject.SizedObject`
.. |SizeError| replace:: :class:`~plum.SizeError`
.. |Str| replace:: :class:`~plum.str.Str`
.. |Structure| replace:: :class:`~plum.structure.Structure`
.. |Tutorials| replace:: :doc:`Tutorials </tutorials/index>`
.. |unpack| replace:: :func:`~plum.unpack`
.. |unpack()| replace:: :func:`~plum.unpack`
.. |UnpackError| replace:: :class:`~plum.UnpackError`
.. |Utf8Str| replace:: :class:`~plum.str.Utf8Str`
.. |Utf8ZeroTermStr| replace:: :class:`~plum.str.Utf8ZeroTermStr`
......@@ -25,7 +25,17 @@ more complete usage examples. The following summarizes the
+===========================+======================+=====================================================+
| :mod:`plum.array` | |Array| | list of uniformly typed items |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.int` | |Int| | integer base class |
| :mod:`plum.bytearray` | |ByteArray| | array of bytes |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.float` | |Float| | float base class |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.float.big` | Float32, Float64 | big endian floating point numbers |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.float.little` | Float32, Float64 | little endian floating point numbers |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.float.native` | Float32, Float64 | native endian floating point numbers |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.int` | |Int| | integer (and enumeration) base class |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.int.bitfields` | |BitFields| | integer with bitfield accessors |
+---------------------------+----------------------+-----------------------------------------------------+
......@@ -43,8 +53,19 @@ more complete usage examples. The following summarizes the
| +----------------------+-----------------------------------------------------+
| | UInt8, 16, 32, 64 | native endian unsigned integers |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.nil` | |Nil| | no memory bytes |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.sequence` | |Sequence| | list of uniquely typed items |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.str` | |Str| | string base class |
| +----------------------+-----------------------------------------------------+
| | |AsciiStr| | ASCII encoded string |
| +----------------------+-----------------------------------------------------+
| | |AsciiZeroTermStr| | ASCII encoded zero terminated string |
| +----------------------+-----------------------------------------------------+
| | |Utf8Str| | UTF-8 encoded string |
| +----------------------+-----------------------------------------------------+
| | |Utf8ZeroTermStr| | UTF-8 encoded zero terminated string |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.structure` | |Structure| | dictionary of named, uniquely typed items |
+---------------------------+----------------------+-----------------------------------------------------+
......@@ -81,13 +81,20 @@ Modules
Module Description
============================== ================================================
:mod:`plum.array` list of uniformly typed items
:mod:`plum.int` integer base class
:mod:`plum.bytearray` array of bytes
:mod:`plum.float` float base class
:mod:`plum.float.big` big endian floats
:mod:`plum.float.little` little endian floats
:mod:`plum.float.native` native endian floats
:mod:`plum.int` integer (and enumeration) base class
:mod:`plum.int.big` big endian integers
:mod:`plum.int.bitfields` integer with bitfield accessors
:mod:`plum.int.bitflags` integer with bit accessors
:mod:`plum.int.little` little endian integers
:mod:`plum.int.native` native endian integers
:mod:`plum.nil` no memory bytes
:mod:`plum.sequence` list of uniquely typed items
:mod:`plum.str` string
:mod:`plum.structure` dictionary of named, uniquely typed items
============================== ================================================
......@@ -96,11 +103,18 @@ Modules
:hidden:
array <modules/array.rst>
bytearray <modules/bytearray.rst>
float <modules/float.rst>
float.big <modules/float_big.rst>
float.little <modules/float_little.rst>
float.native <modules/float_native.rst>
int <modules/int.rst>
int.big <modules/int_big.rst>
int.bitfields <modules/int_bitfields.rst>
int.bitflags <modules/int_bitflags.rst>
int.little <modules/int_little.rst>
int.native <modules/int_native.rst>
nil <modules/nil.rst>
sequence <modules/sequence.rst>
str <modules/str.rst>
structure <modules/structure.rst>
#################################
[plum.bytearray] Module Reference
#################################
.. include:: ../../alias.txt
.. automodule:: plum.bytearray
.. autoclass:: ByteArray()
.. autoclass:: ByteArrayType()
.. seealso::
:doc:`ByteArray Tutorial </tutorials/bytearray>`
#############################
[plum.float] Module Reference
#############################
.. include:: ../../alias.txt
.. automodule:: plum.float
.. autoclass:: Float()
.. autoclass:: FloatType()
.. seealso::
:doc:`Float Tutorial </tutorials/float>`
#################################
[plum.float.big] Module Reference
#################################
.. include:: ../../alias.txt
.. automodule:: plum.float.big
.. autoclass:: Float32(x=0)
.. autoclass:: Float64(x=0)
####################################
[plum.float.little] Module Reference
####################################
.. include:: ../../alias.txt
.. automodule:: plum.float.little
.. autoclass:: Float32(x=0)
.. autoclass:: Float64(x=0)
####################################
[plum.float.native] Module Reference
####################################
.. include:: ../../alias.txt
.. automodule:: plum.float.native
.. class:: Float32([x], base=10)
Single precision native endian floating point number.
.. class:: Float64([x], base=10)
Double precision native endian floating point number.
###########################
[plum.nil] Module Reference
###########################
.. include:: ../../alias.txt
.. automodule:: plum.nil
.. autoclass:: Nil()
.. autodata:: nil
:annotation: = singleton instance
.. seealso::
:doc:`Nil Tutorial </tutorials/nil>`
###########################
[plum.str] Module Reference
###########################
.. include:: ../../alias.txt
.. automodule:: plum.str
.. autoclass:: Str(object, encoding='ascii', error='strict')
.. autoclass:: StrType
.. autoclass:: AsciiStr
.. autoclass:: AsciiZeroTermStr
.. autoclass:: Utf8Str
.. autoclass:: Utf8ZeroTermStr
......@@ -10,6 +10,8 @@
.. autoclass:: Structure(mapping, **kwargs)
.. autoclass:: Varies
.. autofunction:: limit_size
.. autofunction:: measure_dims
......@@ -18,6 +20,8 @@
.. autofunction:: member
.. autofunction:: switch_type
.. autofunction:: transfer_dims
.. seealso::
......
......@@ -8,6 +8,11 @@ Versions increment per `semver <http://semver.org/>`_ (except for alpha series).
Alpha (0.1.0aX) Versions (only serial number X increments)
**********************************************************
+ 0.1.0a4 (2019-03-13)
- Add ByteArray, Float, Nil, Str types.
- Add switch_type( ) to support varying structure member type based on
another structure member.
+ 0.1.0a3 (2019-03-07)
- Change str() and repr() formats.
- Remove Tuple, SizedArray, SizedObject types.
......
###############################################
[plum.bytearray] Tutorial: Using ByteArray Type
###############################################
.. include:: ../alias.txt
The |ByteArray| type inherits :class:`bytearray` characteristics as a result of
it being a subclass of it. The |plum| type collection includes this type for
applications that require an array of bytes. This tutorial shows sample usages
and the |ByteArray| type's properties.
.. contents::
:local:
*****************
Out of Box Sample
*****************
By default, the |ByteArray| type consumes all bytes given to it when unpacking,
packing or instantiating:
>>> from plum import pack, unpack
>>> from plum.bytearray import ByteArray
>>>
>>> unpack(ByteArray, b'\x00\x01')
ByteArray(b'\x00\x01')
>>>
>>> pack(ByteArray, [0, 1])
b'\x00\x01'
>>>
>>> ByteArray([0, 1])
ByteArray(b'\x00\x01')
*******************
Fixed Size Subclass
*******************
To lock the size of the byte array for use within a composite type such as |Structure|,
subclass |ByteArray| and specify the number of bytes using the ``nbytes`` keyword
argument:
>>> from plum import dump, pack, unpack
>>> from plum.bytearray import ByteArray
>>> from plum.int.little import UInt8
>>> from plum.structure import Structure
>>>
>>> class ByteArray1(ByteArray, nbytes=4):
... """Interpret memory as a byte array of length 4."""
...
>>> class Struct1(Structure):
... barray: ByteArray1
... bookend: UInt8
...
>>> struct1 = unpack(Struct1, b'\x00\x01\x02\x03\x99')
>>> dump(struct1)
+--------+------------+--------------+-------------+------------+
| Offset | Access | Value | Memory | Type |
+--------+------------+--------------+-------------+------------+
| | x | | | Struct1 |
| | .barray | | | ByteArray1 |
| 0 | [0:4] | [0, 1, 2, 3] | 00 01 02 03 | |
| 4 | .bookend | 153 | 99 | UInt8 |
+--------+------------+--------------+-------------+------------+
>>>
>>> pack(Struct1, {'barray': [0, 1, 2, 3], 'bookend': 0x99})
b'\x00\x01\x02\x03\x99'
**********************
Sized Structure Member
**********************
To couple a size member to a byte array member, use the :func:`measure_size`
and :func:`limit_size` functions. The following example demonstrates. See
the :doc:`Sized Object Tutorial <structure/sizedobject>` for details.
>>> from plum import dump, pack, unpack
>>> from plum.bytearray import ByteArray
>>> from plum.int.little import UInt8
>>> from plum.structure import Structure, measure_size, limit_size
>>>
>>> class Struct2(Structure):
... size: UInt8 = measure_size(sizedmember='barray')
... barray: ByteArray = limit_size(sizemember='size')
... bookend: UInt8
...
>>> # size member automatically filled in
>>> struct2 = Struct2(barray=ByteArray(24), bookend=0x99)
>>> dump(struct2)
+--------+-------------+--------------------------------------------------+-------------------------------------------------+-----------+
| Offset | Access | Value | Memory | Type |
+--------+-------------+--------------------------------------------------+-------------------------------------------------+-----------+
| | x | | | Struct2 |
| 0 | .size | 24 | 18 | UInt8 |
| | .barray | | | ByteArray |
| 1 | [0:16] | [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
| 17 | [16:24] | [0, 0, 0, 0, 0, 0, 0, 0] | 00 00 00 00 00 00 00 00 | |
| 25 | .bookend | 153 | 99 | UInt8 |
+--------+-------------+--------------------------------------------------+-------------------------------------------------+-----------+
>>>
>>> dump(unpack(Struct2, b'\x01\x02\x99'))
+--------+------------+-------+--------+-----------+
| Offset | Access | Value | Memory | Type |
+--------+------------+-------+--------+-----------+
| | x | | | Struct2 |
| 0 | .size | 1 | 01 | UInt8 |
| | .barray | | | ByteArray |
| 1 | [0:1] | [2] | 02 | |
| 2 | .bookend | 153 | 99 | UInt8 |
+--------+------------+-------+--------+-----------+
**********
Properties
**********
The |ByteArray| constructor accepts the same argument variations as :class:`bytearray`:
>>> from plum.bytearray import ByteArray
>>>
>>> # iterable of ints
>>> ByteArray([0, 1, 2])
ByteArray(b'\x00\x01\x02')
>>>
>>> # string
>>> ByteArray('Hello World!', encoding='ascii', errors='strict')
ByteArray(b'Hello World!')
>>>
>>> # bytes or buffer
>>> ByteArray(b'\x00\x01\x02')
ByteArray(b'\x00\x01\x02')
>>>
>>> # integer size
>>> ByteArray(3)
ByteArray(b'\x00\x00\x00')
>>>
>>> # empty
>>> ByteArray()
ByteArray(b'')
|ByteArray| types follow all other :class:`bytearray` method behaviors.
########################################
[plum.float] Tutorial: Using Float Types
########################################
.. include:: ../alias.txt
The |Float| types inherit :class:`float` characteristics as a result of
it being a subclass of it. The |plum| type collection includes |Float| type
variants for packing and unpacking floating point numbers. This tutorial
shows sample usages and properties of |Float| types.
.. contents::
:local:
*******************
Packing / Unpacking
*******************
The :mod:`plum.float` subpackage includes three modules, one for each endian
byte order: :mod:`plum.float.big`, :mod:`plum.float.little`, and :mod:`plum.native`
(byte order follows host architecture). Each module provides a single precision and
double precision float type (``Float32`` and ``Float64`` respectively) for the
particular byte order.
>>> from plum import pack, unpack
>>> from plum.float.little import Float32
>>>
>>> unpack(Float32, b'\x00\x00\x00\x00')
Float32(0.0)
>>>
>>> pack(Float32, 0.0)
b'\x00\x00\x00\x00'
>>>
>>> f = Float32()
>>> f
Float32(0.0)
>>> pack(f)
b'\x00\x00\x00\x00'
**************
Half-Precision
**************
To create half precision floating point types (16 bits in size), subclass |Float| and specify
``2`` for the number of bytes and either ``'big'`` or ``'little'`` for the byte order:
>>> from plum import dump, pack, unpack
>>> from plum.float import Float
>>>
>>> class Float16(Float, nbytes=2, byteorder='little'):
... """Half precision little endian floating point number."""
...
>>> unpack(Float16, b'\x00\x00')
Float16(0.0)
>>>
>>> pack(Float16, 0.0)
b'\x00\x00'
**********
Properties
**********
The float types accept the same argument variations as :class:`float` and share
the same properties:
>>> from plum.float.little import Float32
>>>
>>> # default to 0
>>> Float32()
Float32(0.0)
>>>
>>> # accept string
>>> Float32("10")
Float32(10.0)
>>>
>>> # include all float properties/methods
>>> f = unpack(Float32, b'\x00\x00\x00\x00')
>>> f.as_integer_ratio()
(0, 1)
>>> f.conjugate()
0.0
>>> f.hex()
'0x0.0p+0'
>>> Float32.fromhex(f.hex())
Float32(0.0)
>>> f.imag
0.0
>>> f.is_integer()
True
>>> f.real
0.0
......@@ -5,6 +5,10 @@
.. toctree::
:maxdepth: 2
Arrays <array/index>
Structures <structure/index>
Array <array/index>
ByteArray <bytearray>
Float <float>
Nil <nil>
String <string/index>
Structure <structure/index>
Utilities <utilities/index>
###################################
[plum.nil] Tutorial: Using Nil Type
###################################
.. include:: ../alias.txt
The |Nil| type consumes no memory bytes when unpacked and produces no memory
bytes when packed. Seldom needed, but important when a dynamic structure needs
a placeholder. This tutorial shows a sample usage and the |Nil| type's properties.
.. contents::
:local:
************
Sample Usage
************
One application where |Nil| may become necessary is when a structure member's
datatype depends on another structure member. The :func:`switch_type` function
as shown in the following example demonstrates a use case. The ``data`` member's
type depends on the value of the ``datatype`` member as defined by the mapping
passed to :func:`switch_type`. When ``datatype`` equals ``0``, no memory bytes
are expected for the ``data`` member. When ``datatype`` equals ``1`` or ``2``,
memory bytes for a ``UInt8`` or ``UInt16`` (respectively) are expected for the
``data`` member:
>>> from plum import unpack, dump
>>> from plum.int.little import UInt8, UInt16
>>> from plum.nil import Nil
>>> from plum.structure import Structure, Varies, switch_type
>>>
>>> class Struct1(Structure):
... datatype: UInt8
... data: Varies = switch_type('datatype', {0: Nil, 1: UInt8, 2: UInt16})
...
>>> dump(unpack(Struct1, b'\x00'))
+--------+-------------+-------+--------+---------+
| Offset | Access | Value | Memory | Type |
+--------+-------------+-------+--------+---------+
| | x | | | Struct1 |
| 0 | .datatype | 0 | 00 | UInt8 |
| | .data | nil | | Nil |
+--------+-------------+-------+--------+---------+
>>>
>>> dump(unpack(Struct1, b'\x01\x02'))
+--------+-------------+-------+--------+---------+
| Offset | Access | Value | Memory | Type |
+--------+-------------+-------+--------+---------+
| | x | | | Struct1 |
| 0 | .datatype | 1 | 01 | UInt8 |
| 1 | .data | 2 | 02 | UInt8 |
+--------+-------------+-------+--------+---------+
**********
Properties
**********
During instantiation or packing, the |Nil| type accepts either ``None``,
the singleton ``nil`` object, or nothing:
>>> from plum.nil import Nil, nil
>>>
>>> Nil(None)
Nil()
>>> Nil(nil)
Nil()
>>> Nil()
Nil()
Instantiation of |Nil| always yields the singleton ``nil`` object:
>>> x = Nil()
>>> y = Nil()
>>> x is y
True
>>> x is nil
True
>>> y is nil
True