Commit f989d014 authored by Dan Gass's avatar Dan Gass

add Items, NamedItems

parent 1315311f
......@@ -35,8 +35,9 @@ more complete usage examples. The following summarizes the
:doc:`plum.int.bitfields </tutorials/types/int_bitfields>` integer with bit field accessors
:doc:`plum.int.enum </tutorials/types/int_enum>` integer enumerated constants
:doc:`plum.int.flag </tutorials/types/int_flag>` integer with bit flags
:doc:`plum.items </tutorials/types/items>` collection of uniquely typed items
:doc:`plum.ipv4 </tutorials/types/ipv4address>` IPV4 address
:doc:`plum.nil </tutorials/types/nil>` no bytes
:doc:`plum.str </tutorials/types/str/index>` strings
:doc:`plum.structure </tutorials/types/structure/index>` structure of uniquely typed
:doc:`plum.structure </tutorials/types/structure/index>` predefined structure of uniquely typed members
========================================================== ================================================
......@@ -22,9 +22,10 @@ Plum Types
:mod:`plum.int.enum` integer enumerated constants
:mod:`plum.int.flag` integer with bit flags
:mod:`plum.ipv4` IPV4 address
:mod:`plum.items` collection of uniquely typed items
:mod:`plum.nil` no bytes
:mod:`plum.str` string
:mod:`plum.structure` structure of uniquely typed
:mod:`plum.structure` predefined structure of uniquely typed members
============================== ================================================
.. toctree::
......@@ -47,6 +48,7 @@ Plum Types
ipv4.big <types/ipv4_big.rst>
ipv4.little <types/ipv4_little.rst>
ipv4.native <types/ipv4_native.rst>
items <types/items.rst>
nil <types/nil.rst>
str <types/str.rst>
structure <types/structure.rst>
......
#############################
[plum.items] Module Reference
#############################
.. include:: ../../alias.txt
.. automodule:: plum.items
.. autoclass:: Items
.. autoclass:: NamedItems
.. seealso::
:doc:`plum.items Tutorials </tutorials/types/items>`
......@@ -8,6 +8,14 @@ Versions increment per `semver <http://semver.org/>`_ (except for alpha/beta ver
Beta (0.X.Y) Versions (X increments on backwards incompatible changes)
**********************************************************************
+ 0.3.0 (2020-Mar-TBD)
- Add ``plum.items`` module for collections of typed items for data
packing applications where the item structure is not predefined.
- Remove support from ``plum.structure.Structure`` base class
(it no longer supports accepting typed items and must be
subclassed to instantiate).
+ 0.2.0 (2020-Mar-11)
- Fix ``IpV4Address`` iteration order.
......
......@@ -13,6 +13,7 @@
plum.int.enum <int_enum>
plum.int.flag <int_flag>
plum.ipv4 <ipv4address>
plum.items <items>
plum.nil <nil>
plum.str <str/index>
plum.structure <structure/index>
#######################################
[plum.items] Tutorial: Item Collections
#######################################
.. include:: ../../alias.txt
This tutorial shows how to create typed item collections for packing
bytes. This alleviates the need for creating custom |Structure|
subclasses in applications where data structure varies.
An example situation which shows the benefit is when passing message data
to a communications driver. Passing a :class:`Items` instance facilitates
logging the message showing the individual item values.
.. >>> # function mocks
>>> transmit = bytes
>>> from plum.int.little import UInt8
>>> from plum.items import Items
>>>
>>> def send_message(message):
... msg, dump = message.pack_and_dump()
... print(dump)
... transmit(msg)
...
>>> message = Items([
... UInt8(1), # command
... UInt8(2), # subcommand
... ])
>>> send_message(message)
+--------+--------+-------+-------+-------+
| Offset | Access | Value | Bytes | Type |
+--------+--------+-------+-------+-------+
| | | | | Items |
| 0 | [0] | 1 | 01 | UInt8 |
| 1 | [1] | 2 | 02 | UInt8 |
+--------+--------+-------+-------+-------+
The :class:`Items` class subclasses Python's built-in :class:`list` and inherits
all of the behaviors and features. The alternative :class:`NamedItems` shares
the same purpose but subclasses Python's built-in :class:`dict`. Using
:class:`NamedItems` facilitates meaningful names within the log for each item
in the collection. For example:
>>> from plum.items import NamedItems
>>>
>>> message = NamedItems(command=UInt8(1), subcommand=UInt8(2))
>>> send_message(message)
+--------+-------------------+-------+-------+------------+
| Offset | Access | Value | Bytes | Type |
+--------+-------------------+-------+-------+------------+
| | | | | NamedItems |
| 0 | [0] (.command) | 1 | 01 | UInt8 |
| 1 | [1] (.subcommand) | 2 | 02 | UInt8 |
+--------+-------------------+-------+-------+------------+
The :class:`NamedItems` inherits :class:`dict` behaviors but also supports
retrieving the item by its name as an attribute. For example:
>>> message.command
UInt8(1)
.. important::
Values provided to either :class:`Items` or :class:`NamedItems` must be
|plum| type instances otherwise the packing operation raises a
:class:`PackingError`.
......@@ -9,12 +9,12 @@ instance returned by the :func:`unpack` utility or by direct instantiation.
First create a custom structure type and instantiate it:
>>> from plum.structure import Structure
>>> from plum.structure import Structure, Member
>>> from plum.int.little import UInt8, UInt16
>>>
>>> class MyStruct(Structure):
... m1: UInt8
... m2: UInt16
... m1: int = Member(cls=UInt8)
... m2: int = Member(cls=UInt16)
...
>>> mystruct = MyStruct(m1=1, m2=2)
>>> mystruct
......
......@@ -15,7 +15,6 @@ features of the |Structure| type.
.. toctree::
Using Structure As Is <outofbox>
Create Custom Type <customtype>
Instantiating <instantiating>
Accessing Members <access>
......
################################################
[plum.structure] Tutorial: Using Structure As Is
################################################
.. include:: ../../../alias.txt
This tutorial shows how to create structure instances without creating a custom
|Structure| subclass. Where applicable, this technique simplifies your code,
saves a few hundred bytes, reduces import time, improves code readability, and
facilitates logging.
An example situation which shows the benefit is when passing message data
to a communications driver. Passing a |plum| type instance facilitates
detailed human readable logging of the message. Using structure instances
in particular has the advantage of attaching names to every component in the
logged message:
>>> from plum import pack_and_dump
>>> from plum.int.little import UInt8
>>> from plum.structure import Structure
>>>
>>> # mock a transmit function
>>> transmit = bytes
>>>
>>> def send_message(message):
... msg, dump = pack_and_dump(type(message), message)
... print(dump)
... transmit(msg)
...
>>> message = Structure(command=UInt8(1), subcommand=UInt8(2))
>>> send_message(message)
+--------+-------------------+-------+-------+-----------+
| Offset | Access | Value | Bytes | Type |
+--------+-------------------+-------+-------+-----------+
| | | | | Structure |
| 0 | [0] (.command) | 1 | 01 | UInt8 |
| 1 | [1] (.subcommand) | 2 | 02 | UInt8 |
+--------+-------------------+-------+-------+-----------+
.. important::
Values provided for the structure members must be |plum| type instances
since a |Structure| subclass does not have predefined member types.
While not as beneficial for the byte summary dumps, structure member values
may be passed in without names:
>>> message = Structure(UInt8(1), UInt8(2))
>>> send_message(message)
+--------+--------+-------+-------+-----------+
| Offset | Access | Value | Bytes | Type |
+--------+--------+-------+-------+-----------+
| | | | | Structure |
| 0 | [0] | 1 | 01 | UInt8 |
| 1 | [1] | 2 | 02 | UInt8 |
+--------+--------+-------+-------+-----------+
......@@ -34,7 +34,7 @@ class VersionInfo:
releaselevel = 'final'
major = 0
minor = 2
minor = 3
micro = 0
serial = 0
......
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
"""Collection of uniquely typed items."""
from ._items import Items
from ._nameditems import NamedItems
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
"""Collection of uniquely typed items."""
from plum._plum import Plum, PlumType
class Items(list, Plum, metaclass=PlumType):
"""Collection of uniquely typed items.
:param iterable iterable: items
"""
__nbytes__ = None
@classmethod
def __pack__(cls, buffer, offset, parent, value, dump):
parent = None
if dump:
dump.cls = cls
for i, item in enumerate(value):
subdump = None if dump is None else dump.add_record(access=f'[{i}]')
item_cls = type(item)
if not issubclass(item_cls, Plum):
if subdump:
subdump.value = repr(item)
subdump.cls = item_cls.__name__ + ' (invalid)'
raise TypeError(f'item {i} not a plum type')
offset = item_cls.__pack__(buffer, offset, parent, item, subdump)
return offset
@classmethod
def __unpack__(cls, buffer, offset, parent, dump):
if dump:
dump.cls = cls
raise TypeError(f'{cls.__name__!r} does not support unpacking')
def __repr__(self):
return f'{type(self).__name__}({list.__repr__(self)})'
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
"""Collection of uniquely typed named items."""
from plum._plum import Plum, PlumType
class NamedItems(dict, Plum, metaclass=PlumType):
"""Collection of uniquely typed named items.
:param iterable: items
:type: dict or list of key/values pairs
:param dict kwargs: items
"""
@classmethod
def __pack__(cls, buffer, offset, parent, value, dump):
parent = None
if dump:
dump.cls = cls
for i, (name, item) in enumerate(value.items()):
subdump = None if dump is None else dump.add_record(
access=f'[{i}] (.{name})')
item_cls = type(item)
if not issubclass(item_cls, Plum):
if subdump:
subdump.value = repr(item)
subdump.cls = item_cls.__name__ + ' (invalid)'
raise TypeError(f'item {name!r} not a plum type')
offset = item_cls.__pack__(buffer, offset, parent, item, subdump)
return offset
@classmethod
def __unpack__(cls, buffer, offset, parent, dump):
if dump:
dump.cls = cls
raise TypeError(f'{cls.__name__!r} does not support unpacking')
def __repr__(self):
return f'{type(self).__name__}({dict.__repr__(self)})'
def __getattr__(self, name):
try:
return self[name]
except KeyError:
# for consistent error message, let standard mechanism raise the exception
object.__getattribute__(self, name)
......@@ -3,9 +3,9 @@
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
"""Interpret bytes as a structure with uniquely named and typed members."""
from ._bitfield_member import BitFieldMember
from ._dims_member import DimsMember
from ._member import Member
from ._bitfield_member import BitFieldMember
from ._size_member import SizeMember
from ._structure import Structure
from ._structuretype import StructureType
......
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
"""Special methods for Structure subclasses with defined members."""
from .._plumview import PlumView
def asdict(self):
"""Return structure members in dictionary form.
:returns: structure members
:rtype: dict
"""
names, _types = self.__names_types__
return dict(zip(names, self))
def __pack__(cls, buffer, offset, parent, value, dump):
# pylint: disable=too-many-branches,unused-argument
names, types = cls.__names_types__
if isinstance(value, PlumView):
# read all members at once
value = value.get()
if isinstance(value, dict):
value = cls(**value)
try:
if len(names) != len(value):
raise ValueError(
f'invalid value, '
f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
f'got an iterable of length {len(value)}')
except TypeError:
raise TypeError(
f'invalid value, '
f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
f'got a non-iterable')
if dump:
dump.cls = cls
for i, (name, value_cls) in enumerate(zip(names, types)):
offset = value_cls.__pack__(
buffer, offset, value, value[i], dump.add_record(access=f'[{i}] (.{name})'))
else:
for i, value_cls in enumerate(types):
offset = value_cls.__pack__(buffer, offset, value, value[i], None)
return offset
def __unpack__(cls, buffer, offset, parent, dump):
# pylint: disable=too-many-locals,unused-argument
names, types = cls.__names_types__
self = list.__new__(cls)
append = list.append
if dump:
dump.cls = cls
for i, (name, item_cls) in enumerate(zip(names, types)):
item, offset = item_cls.__unpack__(
buffer, offset, self,
dump.add_record(access=f'[{i}] (.{name})'))
append(self, item)
else:
for item_cls in types:
item, offset = item_cls.__unpack__(buffer, offset, self, dump)
append(self, item)
return self, offset
def __setattr__(self, name, value):
# get the attribute to raise an exception if invalid name
getattr(self, name)
object.__setattr__(self, name, value)
......@@ -4,6 +4,7 @@
"""Structure type."""
from .._plum import Plum
from .._plumview import PlumView
from ._structuretype import StructureType
from ._structureview import StructureView
......@@ -21,75 +22,79 @@ class Structure(list, Plum, metaclass=StructureType):
__nbytes__ = 0
__plum_names__ = []
def __init__(self, *args, **kwargs):
# pylint: disable=super-init-not-called
def asdict(self):
"""Return structure members in dictionary form.
# initializer for anonymous structure (metaclass overrides this
# when creating subclasses with pre-defined members)
list.extend(self, args)
names = [None] * len(args)
if kwargs:
list.extend(self, kwargs.values())
names.extend(kwargs.keys())
object.__setattr__(self, '__plum_names__', names)
:returns: structure members
:rtype: dict
"""
names, _types = self.__names_types__
return dict(zip(names, self))
@classmethod
def __pack__(cls, buffer, offset, parent, value, dump):
# pylint: disable=too-many-branches
# pylint: disable=too-many-branches,unused-argument
names, types = cls.__names_types__
if isinstance(value, PlumView):
# read all members at once
value = value.get()
# implementation for anonymous structure (metaclass overrides this with
# _methods.__pack__ when creating subclasses with pre-defined members)
if isinstance(value, dict):
value = cls(**value)
parent = None
try:
if len(names) != len(value):
raise ValueError(
f'invalid value, '
f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
f'got an iterable of length {len(value)}')
except TypeError:
raise TypeError(
f'invalid value, '
f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
f'got a non-iterable')
if dump:
dump.cls = cls
if isinstance(value, cls):
names = value.__plum_names__
elif isinstance(value, dict):
names = value.keys()
value = value.values()
elif isinstance(value, (list, tuple)):
names = [None] * len(value)
else:
raise TypeError(f'{cls.__name__!r} pack accepts an iterable')
for i, (name, item) in enumerate(zip(names, value)):
if name:
subdump = dump.add_record(access=f'[{i}] (.{name})')
else:
subdump = dump.add_record(access=f'[{i}]')
value_cls = type(item)
if not issubclass(value_cls, Plum):
subdump.value = repr(item)
subdump.cls = value_cls.__name__ + ' (invalid)'
desc = f' ({name})' if name else ''
raise TypeError(
f'anonymous structure member {i}{desc} not a plum type')
offset = value_cls.__pack__(buffer, offset, parent, item, subdump)
for i, (name, value_cls) in enumerate(zip(names, types)):
offset = value_cls.__pack__(
buffer, offset, value, value[i], dump.add_record(access=f'[{i}] (.{name})'))
else:
if isinstance(value, dict):
value = value.values()
elif not isinstance(value, (list, tuple)):
raise TypeError(f'{cls.__name__} pack accepts an iterable item')
for item in value:
item_cls = type(item)
if not issubclass(item_cls, Plum):
raise TypeError('item in anonymous structure not a plum type instance')
offset = item_cls.__pack__(buffer, offset, parent, item, dump)
for i, value_cls in enumerate(types):
offset = value_cls.__pack__(buffer, offset, value, value[i], None)
return offset
@classmethod
def __unpack__(cls, buffer, offset, parent, dump):
# pylint: disable=too-many-locals,unused-argument
names, types = cls.__names_types__
self = list.__new__(cls)
append = list.append
if dump:
dump.cls = cls
return list.__new__(cls), offset
for i, (name, item_cls) in enumerate(zip(names, types)):
item, offset = item_cls.__unpack__(
buffer, offset, self,
dump.add_record(access=f'[{i}] (.{name})'))
append(self, item)
else:
for item_cls in types:
item, offset = item_cls.__unpack__(buffer, offset, self, dump)
append(self, item)
return self, offset
def __setattr__(self, name, value):
# get the attribute to raise an exception if invalid name
getattr(self, name)
object.__setattr__(self, name, value)
def __repr__(self):
lst = [repr(value) if name is None else name + '=' + repr(value)
......@@ -111,45 +116,6 @@ class Structure(list, Plum, metaclass=StructureType):
return StructureView(cls, buffer, offset)
def __getattr__(self, name):
# implementation for anonymous structure (metaclass doesn't bother
# overriding since its harmless)
try:
index = object.__getattribute__(self, '__plum_names__').index(name)
except (AttributeError, ValueError):
# AttributeError -> structure instantiated without names
# ValueError -> name not one used during structure instantiation