Commit 4134795d authored by Dan Gass's avatar Dan Gass
Browse files

Merge branch 'separete-namespace' into 'dev_0_1_2'

Resolve "Move BitFlags to separate namespace"

See merge request dangass/plum!9
parent c965f859
......@@ -9,6 +9,11 @@
The |plum| Python package provides classes and utility functions to
efficiently pack and unpack memory bytes and conveniently access the
memory data. The package offers a collection of basic types which may
be further customized and combined to handle complex data structures.
be further customized and combined in any combination to handle
complex data structures. Most significant features include:
- Arbitrary Nesting (e.g. structure of array of structure of bitfields of bitfields)
- Easy to Create Custom Types
- Speed Potential (goal of nearing :mod:`struct` performance)
- Visual Memory Summaries (including partial dumps in exception messages)
- Extensible API
"""
from plum import pack, unpack
from plum.structure import Structure, member
from plum.int.little import UInt8, UInt16
class MyStruct(Structure):
byte: UInt8
word: UInt16 = 0xffff
x = pack(MyStruct, dict(byte=0))
print(repr(x))
x = unpack(MyStruct, b'\x02\x01\x00')
print(repr(x))
from plum import dump
dump(x)
x = unpack(MyStruct, b'\x02\x01')
"""
"""
from enum import IntEnum
from plum import dump, unpack
from plum.int import Int
from plum.int.little import UInt8
from plum.structure import Structure
class Pet(Int, nbytes=1, enum=True):
CAT = 0
DOG = 1
class Family(Structure):
nkids: UInt8
pet: Pet
x = unpack(Family, b'\x02\x01')
dump(x)
"""
"""
from plum import dump, pack, unpack
from plum.bits import BitFields, bitfield
class MyBf(BitFields, nbytes=1):
bflag: bool = bitfield(pos=0, size=1)
iflag: int = bitfield(pos=1, size=1)
nibble: int = bitfield(pos=2, size=4)
reserved: int = bitfield(pos=6, size=2, default=0)
pack(MyBf, dict(bflag=True, iflag=0, nibble=15))
x = unpack(MyBf, b'\x00')
# dump(x)
"""
from plum import pack, pack_and_getdump
from plum.int.little import UInt8, UInt16
from plum.structure import Structure
membytes, dmp = pack_and_getdump(Structure, {'byte': UInt8(1), 'word': UInt16(2)})
print(dmp)
import pytest
from plum_c import Memory
@pytest.fixture
def memcls():
return Memory
class TestUnpack:
def test_simple(self, memcls):
m = Memory(b'\x01\x02\x03')
# TODO: pytest fixture modifies Memory and it loses its Marbles and segfaults on following line
m.junk
......@@ -90,8 +90,8 @@ By adding my name to this signature list and dating, I acknowledge:
--------- --------------------------------------------------------
Date Contributor Name/Email
--------- --------------------------------------------------------
28Feb2019 Daniel Mark Gass (dan.gass@gmail.com)
01Mar2019 Jan Novak (job.jan.novak@gmail.com)
07Mar2019 Daniel Mark Gass (dan.gass@gmail.com)
****************
......
......@@ -45,15 +45,6 @@ What's Next?
- Add feature test cases.
- Add exception (user experience) test cases.
- Support intertwined structure members.
Facilitate creating custom structures where two members are related
(e.g. a size member specifies the dimension of an array member that
follows). Provide mechanism that does not require user to create
full fledged |plum| type implementations. Instead, the user provides
simple implementations for instantiation/unpack where access to
the parent |plum| type instance is given.
- Add types.
- Bool
......
.. |API reference| replace:: :doc:`API reference </reference/index>`
.. |Array| replace:: :class:`~plum.array.Array`
.. |BitFields| replace:: :class:`~plum.bits.BitFields`
.. |BitFields| replace:: :class:`~plum.int.bitfields.BitFields`
.. |BitFlags| replace:: :class:`~plum.int.bitflags.BitFlags`
.. |ExcessMemoryError| replace:: :class:`~plum.ExcessMemoryError`
.. |InsufficientMemoryError| replace:: :class:`~plum.InsufficientMemoryError`
.. |Int| replace:: :class:`~plum.int.Int`
.. |Memory| replace:: :class:`~plum.Memory`
.. |pack()| replace:: :func:`~plum.pack`
.. |PackError| replace:: :class:`~plum.PackError`
.. |plum| replace:: :mod:`plum`
.. |Quick Start| replace:: :doc:`Quick Start </quickstart>`
......@@ -15,4 +17,5 @@
.. |SizeError| replace:: :class:`~plum.SizeError`
.. |Structure| replace:: :class:`~plum.structure.Structure`
.. |Tutorials| replace:: :doc:`Tutorials </tutorials/index>`
.. |unpack| replace:: :func:`~plum.unpack`
.. |UnpackError| replace:: :class:`~plum.UnpackError`
......@@ -25,10 +25,12 @@ more complete usage examples. The following summarizes the
+===========================+======================+=====================================================+
| :mod:`plum.array` | |Array| | list of uniformly typed items |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.bits` | |BitFields| | integer with bitfield accessors |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.int` | |Int| | integer base class |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.int.bitfields` | |BitFields| | integer with bitfield accessors |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.int.bitflags` | |BitFlags| | integer with bit accessors |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.int.big` | SInt8, 16, 32, 64 | big endian signed integers |
| +----------------------+-----------------------------------------------------+
| | UInt8, 16, 32, 64 | big endian unsigned integers |
......@@ -37,11 +39,11 @@ more complete usage examples. The following summarizes the
| +----------------------+-----------------------------------------------------+
| | UInt8, 16, 32, 64 | little endian unsigned integers |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.sequence` | |Sequence| | list of uniquely typed items |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.sizedarray` | |SizedArray| | dictionary with array size and array members |
| :mod:`plum.int.native` | SInt8, 16, 32, 64 | native endian signed integers |
| +----------------------+-----------------------------------------------------+
| | UInt8, 16, 32, 64 | native endian unsigned integers |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.sizedobject` | |SizedObject| | dictionary with item size and item members |
| :mod:`plum.sequence` | |Sequence| | list of uniquely typed items |
+---------------------------+----------------------+-----------------------------------------------------+
| :mod:`plum.structure` | |Structure| | dictionary of named, uniquely typed items |
+---------------------------+----------------------+-----------------------------------------------------+
......
......@@ -24,4 +24,4 @@ Install Steps
At a shell prompt, use `pip <https://pypi.python.org/pypi/pip>`_ to
automatically download and install :doc:`plum </index>`::
python -m pip install --upgrade plum
python -m pip install --upgrade plum-py
......@@ -6,6 +6,9 @@
.. include:: alias.txt
.. contents::
:local:
Basic Types
+++++++++++
......@@ -103,7 +106,7 @@ For example:
>>>
>>> mystruct = unpack(MyStruct, b'\x02\x01\x00')
>>> mystruct
MyStruct({'byte'=2, 'word'=1})
MyStruct({'byte': 2, 'word': 1})
>>>
>>> # access members via attribute or dict lookup
>>> mystruct.byte == 2
......@@ -321,7 +324,7 @@ member in the memory layout. Notice, access to the bitfield members
produces an instance of the Python type from the member definition:
>>> from plum import pack, unpack
>>> from plum.bits import BitFields, bitfield
>>> from plum.int.bitfields import BitFields, bitfield
>>>
>>> class MyBf(BitFields, nbytes=1):
... bflag: bool = bitfield(pos=0, size=1)
......
......@@ -81,13 +81,13 @@ Modules
Module Description
============================== ================================================
:mod:`plum.array` list of uniformly typed items
:mod:`plum.bits` integer with bitfield accessors
:mod:`plum.int` integer base class
:mod:`plum.int.big` big endian integers
:mod:`plum.int.bitfields` integer with bitfield accessors
:mod:`plum.int.bitflags` integer with bit accessors
:mod:`plum.int.little` little endian integers
:mod:`plum.int.native` native endian integers
:mod:`plum.sequence` list of uniquely typed items
:mod:`plum.sizedarray` dictionary with array size and array members
:mod:`plum.sizedobject` dictionary with item size and item members
:mod:`plum.structure` dictionary of named, uniquely typed items
============================== ================================================
......@@ -96,12 +96,11 @@ Modules
:hidden:
array <modules/array.rst>
bits <modules/bits.rst>
int <modules/int.rst>
int.big <modules/int_big.rst>
int.bitfields <modules/int_bitfields.rst>
int.bitflags <modules/int_bitflags.rst>
int.little <modules/int_little.rst>
int.native <modules/int_native.rst>
sequence <modules/sequence.rst>
sizedarray <modules/sizedarray.rst>
sizedobject <modules/sizedobject.rst>
structure <modules/structure.rst>
############################
[plum.bits] Module Reference
############################
#####################################
[plum.int.bitfields] Module Reference
#####################################
.. include:: ../../alias.txt
.. automodule:: plum.bits
.. automodule:: plum.int.bitfields
.. autoclass:: BitFields(value, **kwargs)
......
####################################
[plum.int.bitflags] Module Reference
####################################
.. include:: ../../alias.txt
.. automodule:: plum.int.bitflags
.. autoclass:: BitFlags(value, **kwargs)
.. autoclass:: BitFlagsType
##################################
[plum.sizedarray] Module Reference
##################################
.. include:: ../../alias.txt
.. automodule:: plum.sizedarray
.. autoclass:: SizedArray(value, **kwargs)
.. autoclass:: SizedArrayType
###################################
[plum.sizedobject] Module Reference
###################################
.. include:: ../../alias.txt
.. automodule:: plum.sizedobject
.. autoclass:: SizedObject(value, **kwargs)
.. autoclass:: SizedObjectType
......@@ -6,6 +6,20 @@
.. automodule:: plum.structure
.. autoclass:: Structure(value, **kwargs)
.. autoclass:: StructureType
.. autoclass:: Structure(mapping, **kwargs)
.. autofunction:: limit_size
.. autofunction:: measure_dims
.. autofunction:: measure_size
.. autofunction:: member
.. autofunction:: transfer_dims
.. seealso::
:doc:`Structure Tutorials </tutorials/structure/index>`
......@@ -4,14 +4,20 @@
Versions increment per `semver <http://semver.org/>`_ (except for alpha series).
*****************************************************
Alpha (0.1.X) Versions (only patch number increments)
*****************************************************
**********************************************************
Alpha (0.1.0aX) Versions (only serial number X increments)
**********************************************************
+ 0.1.1 (TBD)
+ 0.1.0a3 (2019-03-07)
- Change str() and repr() formats.
- Remove Tuple type.
- Remove Tuple, SizedArray, SizedObject types.
- Improve tests (use basic conformance for all).
- Add structure tutorials.
- Add structure member protocol.
- Add ``measure_dims()``, ``transfer_dims()``, ``measure_size()``,
and ``limit_size()`` functions for creating member associations
for purposes of sized arrays and sized objects.
- Add BitFlags type.
+ 0.1.0a2 (2019-02-25)
- Stabilize basic types (Array, BitFields, Int, Sequence, Structure)
......
##########################
[plum.array] Tutorials
##########################
.. include:: ../../alias.txt
......@@ -2,10 +2,9 @@
[plum] Tutorials
################
.. contents::
:local:
.. toctree::
:maxdepth: 2
.. currentmodule:: plum
Under Construction!
\ No newline at end of file
Arrays <array/index>
Structures <structure/index>
Utilities <utilities/index>
###################################################
[plum.structure] Tutorial: Access Structure Members
###################################################
.. include:: ../../alias.txt
This tutorial shows how to access structure members of a structure
instance returned by the :func:`unpack` utility or by direct instantiation.
First create a custom structure type and instantiate it:
>>> from plum import unpack
>>> from plum.structure import Structure
>>> from plum.int.little import UInt8, UInt16
>>>
>>> class MyStruct(Structure):
... m1: UInt8
... m2: UInt16
...
>>> mystruct = MyStruct(m1=1, m2=2)
>>> mystruct
MyStruct({'m1': 1, 'm2': 2})
Structure instances follow the same behavior :class:`dict` allowing
access to the members by the normal :class:`dict` methods:
>>> mystruct['m1'] == 1
True
>>> mystruct.get('m2') == 2
True
In addition, structure types support accessing members as an attribute:
>>> mystruct.m1 == 1
True
>>> getattr(mystruct, 'm2') == 2
True
###############################################
[plum.structure] Tutorial: Comparing Structures
###############################################
.. include:: ../../alias.txt
|Structure| types support equality comparisons in a similar manner as
:class:`dict`, but with a couple differences. This tutorial explains
those differences and demonstrates special features of structure types
that relate to comparisons.
.. contents::
:local:
***********
Basic Rules
***********
Structure instances support comparisons against any other :class:`dict`-like
value (any object that supports the :meth:`dict.items` method of iterating
key/value pairs). Similar to :class:`dict` comparisons, the dictionary being
compared against must contain the same members (same names, no more, no less),
and for each member, the member values must be equal. Since structures contain
ordered members, comparisons also require the values compared against be in the
same order:
>>> from plum import unpack
>>> from plum.structure import Structure
>>> from plum.int.little import UInt8
>>>
>>> class Struct1(Structure):
... m1: UInt8
... m2: UInt8
...
>>> struct1 = unpack(Struct1, b'\x01\x02')
>>> struct1
Struct1({'m1': 1, 'm2': 2})
>>>
>>> struct1 == {'m1': 1} # missing m2 member
False
>>> struct1 == {'m1': 1, 'm2': 2, 'm3': 3} # extra m3 member
False
>>> struct1 == {'m2': 2, 'm1': 1} # members out of order
False
>>> struct1 == {'m1': 1, 'm2': 2} # exact match
True
.. tip::
Create an instance of the same structure type to avoid worrying
about specifying the members in the wrong order:
>>> struct1 == Struct1({'m2': 2, 'm1': 1})
True
>>> struct1 == Struct1(m2=2, m1=1)
True
********************
Members With Default
********************
For structure types containing members with default values, the basic rules
remain the same. Each member value must be specified and equal. But, if the
expectation for a member is the same as the default, instantiating the
structure type with the expected values allows the defaulted member to be
left unspecified:
>>> from plum import unpack
>>> from plum.structure import Structure
>>> from plum.int.little import UInt8
>>>
>>> class Struct2(Structure):
... m1: UInt8
... m2: UInt8 = 2
...
>>> struct2 = unpack(Struct2, b'\x01\x02')
>>> struct2
Struct2({'m1': 1, 'm2': 2})
>>>
>>> struct2 == {'m1': 1} # missing m2 member
False
>>> struct2 == Struct2({'m1': 1}) # compares with m2 default
True
***************
Ignored Members
***************
For structure types containing members defined as ignored, the basic rules
remain the same except comparisons pass regardless if the ignored member value
matches or is even present in the expected member values:
>>> from plum import unpack
>>> from plum.structure import Structure, member
>>> from plum.int.little import UInt8
>>>
>>> class Struct3(Structure):
... m1: UInt8
... m2: UInt8 = 2
... m3: UInt8 = member(default=3, ignore=True)
...
>>> struct3 = unpack(Struct3, b'\x01\x02\x03')
>>> struct3
Struct3({'m1': 1, 'm2': 2, 'm3': 3})
>>>
>>> struct3 == {'m1': 1, 'm2': 2, 'm3': 99} # m3 does not match (but ignored)
True
>>> struct3 == {'m1': 1, 'm2': 2} # m3 not in expectations (but ignored)
True
#######################################################
[plum.structure] Tutorial: Create Custom Structure Type
#######################################################
.. include:: ../../alias.txt
This tutorial shows how to create custom structure subclasses
that define the name, type, default value, and significance
for each member within a structure. (The significance of a
member determines whether equality comparisons of the structure
consider the member.) Predefining the structure members facilitates
packing and unpacking the structure's memory bytes and ensures
instantiations include all necessary member values.
.. contents::
:local:
***********
Member Type
***********
To define a custom structure, use a traditional class definition and subclass
|Structure|. Within the namespace of the subclass, use Python's annotation
syntax to specify the member name and member type (place a colon in between).
Normally, type annotations provide just a type hint. Within |Structure| classes,
the member type annotations control how memory bytes are packed and unpacked
when using the structure type:
>>> from plum import pack, unpack
>>> from plum.structure import Structure
>>> from plum.int.little import UInt8, UInt16
>>>
>>> class MyStruct(Structure):
... m1: UInt8
... m2: UInt16
...
>>> pack(MyStruct, {'m1': 1, 'm2': 2})
b'\x01\x02\x00'
>>>
>>> unpack(MyStruct, b'\x01\x02\x00')
MyStruct({'m1': 1, 'm2': 2})
The custom structure class also uses the type annotations to ensure
proper values are provided during instantiation:
>>> MyStruct(m1=1)
Traceback (most recent call last):
...
TypeError: MyStruct() missing 1 required keyword-only argument 'm2'
>>>
>>> MyStruct(m1=1, m2=2, m3=3)
Traceback (most recent call last):
...
TypeError: MyStruct() got 1 unexpected member 'm3'
**************
Member Default
**************
Use an assignment within the member type annotation to specify a default
value for the member. Instantiations and :func:`pack` operations use the
default value for the member when one was not provided:
>>> from plum import pack
>>> from plum.structure import Structure, member
>>> from plum.int.little import UInt8, UInt16
>>>
>>> class MyStruct(Structure):
... m1: UInt8
... m2: UInt16 = 2
...
>>> pack(MyStruct, {'m1': 1})
b'\x01\x02\x00'
>>>
>>> MyStruct(m1=1)
MyStruct({'m1': 1, 'm2': 2})
.. tip::