add Items, NamedItems

docs/index.rst

@@ -35,8 +35,9 @@ more complete usage examples. The following summarizes the
     :doc:`plum.int.bitfields </tutorials/types/int_bitfields>`  integer with bit field accessors
     :doc:`plum.int.enum </tutorials/types/int_enum>`            integer enumerated constants
     :doc:`plum.int.flag </tutorials/types/int_flag>`            integer with bit flags
+    :doc:`plum.items </tutorials/types/items>`                  collection of uniquely typed items
     :doc:`plum.ipv4 </tutorials/types/ipv4address>`             IPV4 address
     :doc:`plum.nil </tutorials/types/nil>`                      no bytes
     :doc:`plum.str </tutorials/types/str/index>`                strings
-    :doc:`plum.structure </tutorials/types/structure/index>`    structure of uniquely typed
+    :doc:`plum.structure </tutorials/types/structure/index>`    predefined structure of uniquely typed members
     ==========================================================  ================================================

docs/reference/index.rst

@@ -22,9 +22,10 @@ Plum Types
     :mod:`plum.int.enum`            integer enumerated constants
     :mod:`plum.int.flag`            integer with bit flags
     :mod:`plum.ipv4`                IPV4 address
+    :mod:`plum.items`               collection of uniquely typed items
     :mod:`plum.nil`                 no bytes
     :mod:`plum.str`                 string
-    :mod:`plum.structure`           structure of uniquely typed
+    :mod:`plum.structure`           predefined structure of uniquely typed members
     ==============================  ================================================
 
 .. toctree::
@@ -47,6 +48,7 @@ Plum Types
     ipv4.big <types/ipv4_big.rst>
     ipv4.little <types/ipv4_little.rst>
     ipv4.native <types/ipv4_native.rst>
+    items <types/items.rst>
     nil <types/nil.rst>
     str <types/str.rst>
     structure <types/structure.rst>

docs/reference/types/items.rst

+#############################
+[plum.items] Module Reference
+#############################
+
+.. include:: ../../alias.txt
+
+.. automodule:: plum.items
+
+.. autoclass:: Items
+
+.. autoclass:: NamedItems
+
+.. seealso::
+
+    :doc:`plum.items Tutorials </tutorials/types/items>`

docs/release_notes.rst

@@ -8,6 +8,14 @@ Versions increment per `semver <http://semver.org/>`_ (except for alpha/beta ver
 Beta (0.X.Y) Versions (X increments on backwards incompatible changes)
 **********************************************************************
 
++ 0.3.0 (2020-Mar-TBD)
+
+    - Add ``plum.items`` module for collections of typed items for data
+      packing applications where the item structure is not predefined.
+    - Remove support from ``plum.structure.Structure`` base class
+      (it no longer supports accepting typed items and must be
+      subclassed to instantiate).
+
 + 0.2.0 (2020-Mar-11)
 
     - Fix ``IpV4Address`` iteration order.

docs/tutorials/types/index.rst

@@ -13,6 +13,7 @@
     plum.int.enum <int_enum>
     plum.int.flag <int_flag>
     plum.ipv4 <ipv4address>
+    plum.items <items>
     plum.nil <nil>
     plum.str <str/index>
     plum.structure <structure/index>

docs/tutorials/types/items.rst

+#######################################
+[plum.items] Tutorial: Item Collections
+#######################################
+
+.. include:: ../../alias.txt
+
+This tutorial shows how to create typed item collections for packing
+bytes. This alleviates the need for creating custom |Structure|
+subclasses in applications where data structure varies.
+
+An example situation which shows the benefit is when passing message data
+to a communications driver. Passing a :class:`Items` instance facilitates
+logging the message showing the individual item values.
+
+    ..  >>> # function mocks
+        >>> transmit = bytes
+
+    >>> from plum.int.little import UInt8
+    >>> from plum.items import Items
+    >>>
+    >>> def send_message(message):
+    ...     msg, dump = message.pack_and_dump()
+    ...     print(dump)
+    ...     transmit(msg)
+    ...
+    >>> message = Items([
+    ...   UInt8(1), # command
+    ...   UInt8(2), # subcommand
+    ... ])
+    >>> send_message(message)
+    +--------+--------+-------+-------+-------+
+    | Offset | Access | Value | Bytes | Type  |
+    +--------+--------+-------+-------+-------+
+    |        |        |       |       | Items |
+    | 0      | [0]    | 1     | 01    | UInt8 |
+    | 1      | [1]    | 2     | 02    | UInt8 |
+    +--------+--------+-------+-------+-------+
+
+
+The :class:`Items` class subclasses Python's built-in :class:`list` and inherits
+all of the behaviors and features. The alternative :class:`NamedItems` shares
+the same purpose but subclasses Python's built-in :class:`dict`. Using
+:class:`NamedItems` facilitates meaningful names within the log for each item
+in the collection. For example:
+
+    >>> from plum.items import NamedItems
+    >>>
+    >>> message = NamedItems(command=UInt8(1), subcommand=UInt8(2))
+    >>> send_message(message)
+    +--------+-------------------+-------+-------+------------+
+    | Offset | Access            | Value | Bytes | Type       |
+    +--------+-------------------+-------+-------+------------+
+    |        |                   |       |       | NamedItems |
+    | 0      | [0] (.command)    | 1     | 01    | UInt8      |
+    | 1      | [1] (.subcommand) | 2     | 02    | UInt8      |
+    +--------+-------------------+-------+-------+------------+
+
+
+The :class:`NamedItems` inherits :class:`dict` behaviors but also supports
+retrieving the item by its name as an attribute. For example:
+
+    >>> message.command
+    UInt8(1)
+
+
+.. important::
+
+    Values provided to either :class:`Items` or :class:`NamedItems` must be
+    |plum| type instances otherwise the packing operation raises a
+    :class:`PackingError`.

docs/tutorials/types/structure/access.rst

@@ -9,12 +9,12 @@ instance returned by the :func:`unpack` utility or by direct instantiation.
 
 First create a custom structure type and instantiate it:
 
-    >>> from plum.structure import Structure
+    >>> from plum.structure import Structure, Member
     >>> from plum.int.little import UInt8, UInt16
     >>>
     >>> class MyStruct(Structure):
-    ...     m1: UInt8
-    ...     m2: UInt16
+    ...     m1: int = Member(cls=UInt8)
+    ...     m2: int = Member(cls=UInt16)
     ...
     >>> mystruct = MyStruct(m1=1, m2=2)
     >>> mystruct

docs/tutorials/types/structure/index.rst

@@ -15,7 +15,6 @@ features of the |Structure| type.
 
 .. toctree::
 
-    Using Structure As Is <outofbox>
     Create Custom Type <customtype>
     Instantiating <instantiating>
     Accessing Members <access>

docs/tutorials/types/structure/outofbox.rst

-################################################
-[plum.structure] Tutorial: Using Structure As Is
-################################################
-
-.. include:: ../../../alias.txt
-
-This tutorial shows how to create structure instances without creating a custom
-|Structure| subclass. Where applicable, this technique simplifies your code,
-saves a few hundred bytes, reduces import time, improves code readability, and
-facilitates logging.
-
-An example situation which shows the benefit is when passing message data
-to a communications driver. Passing a |plum| type instance facilitates
-detailed human readable logging of the message. Using structure instances
-in particular has the advantage of attaching names to every component in the
-logged message:
-
-    >>> from plum import pack_and_dump
-    >>> from plum.int.little import UInt8
-    >>> from plum.structure import Structure
-    >>>
-    >>> # mock a transmit function
-    >>> transmit = bytes
-    >>>
-    >>> def send_message(message):
-    ...     msg, dump = pack_and_dump(type(message), message)
-    ...     print(dump)
-    ...     transmit(msg)
-    ...
-    >>> message = Structure(command=UInt8(1), subcommand=UInt8(2))
-    >>> send_message(message)
-    +--------+-------------------+-------+-------+-----------+
-    | Offset | Access            | Value | Bytes | Type      |
-    +--------+-------------------+-------+-------+-----------+
-    |        |                   |       |       | Structure |
-    | 0      | [0] (.command)    | 1     | 01    | UInt8     |
-    | 1      | [1] (.subcommand) | 2     | 02    | UInt8     |
-    +--------+-------------------+-------+-------+-----------+
-
-
-.. important::
-
-    Values provided for the structure members must be |plum| type instances
-    since a |Structure| subclass does not have predefined member types.
-
-
-While not as beneficial for the byte summary dumps, structure member values
-may be passed in without names:
-
-    >>> message = Structure(UInt8(1), UInt8(2))
-    >>> send_message(message)
-    +--------+--------+-------+-------+-----------+
-    | Offset | Access | Value | Bytes | Type      |
-    +--------+--------+-------+-------+-----------+
-    |        |        |       |       | Structure |
-    | 0      | [0]    | 1     | 01    | UInt8     |
-    | 1      | [1]    | 2     | 02    | UInt8     |
-    +--------+--------+-------+-------+-----------+

plum-py/plum/__about__.py

@@ -34,7 +34,7 @@ class VersionInfo:
 
     releaselevel = 'final'
     major = 0
-    minor = 2
+    minor = 3
     micro = 0
     serial = 0
 

plum-py/plum/items/__init__.py

+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+"""Collection of uniquely typed items."""
+
+from ._items import Items
+from ._nameditems import NamedItems

plum-py/plum/items/_items.py

+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+"""Collection of uniquely typed items."""
+
+from plum._plum import Plum, PlumType
+
+
+class Items(list, Plum, metaclass=PlumType):
+
+    """Collection of uniquely typed items.
+
+    :param iterable iterable: items
+
+    """
+
+    __nbytes__ = None
+
+    @classmethod
+    def __pack__(cls, buffer, offset, parent, value, dump):
+        parent = None
+
+        if dump:
+            dump.cls = cls
+
+        for i, item in enumerate(value):
+            subdump = None if dump is None else dump.add_record(access=f'[{i}]')
+
+            item_cls = type(item)
+
+            if not issubclass(item_cls, Plum):
+                if subdump:
+                    subdump.value = repr(item)
+                    subdump.cls = item_cls.__name__ + ' (invalid)'
+
+                raise TypeError(f'item {i} not a plum type')
+
+            offset = item_cls.__pack__(buffer, offset, parent, item, subdump)
+
+        return offset
+
+    @classmethod
+    def __unpack__(cls, buffer, offset, parent, dump):
+        if dump:
+            dump.cls = cls
+        raise TypeError(f'{cls.__name__!r} does not support unpacking')
+
+    def __repr__(self):
+        return f'{type(self).__name__}({list.__repr__(self)})'

plum-py/plum/items/_nameditems.py

+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+"""Collection of uniquely typed named items."""
+
+from plum._plum import Plum, PlumType
+
+
+class NamedItems(dict, Plum, metaclass=PlumType):
+
+    """Collection of uniquely typed named items.
+
+    :param iterable: items
+    :type: dict or list of key/values pairs
+    :param dict kwargs: items
+
+    """
+
+    @classmethod
+    def __pack__(cls, buffer, offset, parent, value, dump):
+        parent = None
+
+        if dump:
+            dump.cls = cls
+
+        for i, (name, item) in enumerate(value.items()):
+            subdump = None if dump is None else dump.add_record(
+                access=f'[{i}] (.{name})')
+
+            item_cls = type(item)
+
+            if not issubclass(item_cls, Plum):
+                if subdump:
+                    subdump.value = repr(item)
+                    subdump.cls = item_cls.__name__ + ' (invalid)'
+                raise TypeError(f'item {name!r} not a plum type')
+
+            offset = item_cls.__pack__(buffer, offset, parent, item, subdump)
+
+        return offset
+
+    @classmethod
+    def __unpack__(cls, buffer, offset, parent, dump):
+        if dump:
+            dump.cls = cls
+        raise TypeError(f'{cls.__name__!r} does not support unpacking')
+
+    def __repr__(self):
+        return f'{type(self).__name__}({dict.__repr__(self)})'
+
+    def __getattr__(self, name):
+        try:
+            return self[name]
+        except KeyError:
+            # for consistent error message, let standard mechanism raise the exception
+            object.__getattribute__(self, name)

plum-py/plum/structure/__init__.py

@@ -3,9 +3,9 @@
 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
 """Interpret bytes as a structure with uniquely named and typed members."""
 
+from ._bitfield_member import BitFieldMember
 from ._dims_member import DimsMember
 from ._member import Member
-from ._bitfield_member import BitFieldMember
 from ._size_member import SizeMember
 from ._structure import Structure
 from ._structuretype import StructureType

plum-py/plum/structure/_methods.py

-# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
-# Copyright 2020 Daniel Mark Gass, see __about__.py for license information.
-# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
-"""Special methods for Structure subclasses with defined members."""
-
-from .._plumview import PlumView
-
-
-def asdict(self):
-    """Return structure members in dictionary form.
-
-    :returns: structure members
-    :rtype: dict
-
-    """
-    names, _types = self.__names_types__
-    return dict(zip(names, self))
-
-
-def __pack__(cls, buffer, offset, parent, value, dump):
-    # pylint: disable=too-many-branches,unused-argument
-    names, types = cls.__names_types__
-
-    if isinstance(value, PlumView):
-        # read all members at once
-        value = value.get()
-
-    if isinstance(value, dict):
-        value = cls(**value)
-
-    try:
-        if len(names) != len(value):
-            raise ValueError(
-                f'invalid value, '
-                f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
-                f'got an iterable of length {len(value)}')
-    except TypeError:
-        raise TypeError(
-            f'invalid value, '
-            f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
-            f'got a non-iterable')
-
-    if dump:
-        dump.cls = cls
-
-        for i, (name, value_cls) in enumerate(zip(names, types)):
-            offset = value_cls.__pack__(
-                buffer, offset, value, value[i], dump.add_record(access=f'[{i}] (.{name})'))
-    else:
-        for i, value_cls in enumerate(types):
-            offset = value_cls.__pack__(buffer, offset, value, value[i], None)
-
-    return offset
-
-
-def __unpack__(cls, buffer, offset, parent, dump):
-    # pylint: disable=too-many-locals,unused-argument
-    names, types = cls.__names_types__
-
-    self = list.__new__(cls)
-    append = list.append
-
-    if dump:
-        dump.cls = cls
-
-        for i, (name, item_cls) in enumerate(zip(names, types)):
-            item, offset = item_cls.__unpack__(
-                buffer, offset, self,
-                dump.add_record(access=f'[{i}] (.{name})'))
-            append(self, item)
-    else:
-        for item_cls in types:
-            item, offset = item_cls.__unpack__(buffer, offset, self, dump)
-            append(self, item)
-
-    return self, offset
-
-
-def __setattr__(self, name, value):
-    # get the attribute to raise an exception if invalid name
-    getattr(self, name)
-    object.__setattr__(self, name, value)

plum-py/plum/structure/_structure.py

@@ -4,6 +4,7 @@
 """Structure type."""
 
 from .._plum import Plum
+from .._plumview import PlumView
 from ._structuretype import StructureType
 from ._structureview import StructureView
 
@@ -21,75 +22,79 @@ class Structure(list, Plum, metaclass=StructureType):
     __nbytes__ = 0
     __plum_names__ = []
 
-    def __init__(self, *args, **kwargs):
-        # pylint: disable=super-init-not-called
+    def asdict(self):
+        """Return structure members in dictionary form.
 
-        # initializer for anonymous structure (metaclass overrides this
-        # when creating subclasses with pre-defined members)
-        list.extend(self, args)
-        names = [None] * len(args)
-        if kwargs:
-            list.extend(self, kwargs.values())
-            names.extend(kwargs.keys())
-        object.__setattr__(self, '__plum_names__', names)
+        :returns: structure members
+        :rtype: dict
+
+        """
+        names, _types = self.__names_types__
+        return dict(zip(names, self))
 
     @classmethod
     def __pack__(cls, buffer, offset, parent, value, dump):
-        # pylint: disable=too-many-branches
+        # pylint: disable=too-many-branches,unused-argument
+        names, types = cls.__names_types__
+
+        if isinstance(value, PlumView):
+            # read all members at once
+            value = value.get()
 
-        # implementation for anonymous structure (metaclass overrides this with
-        # _methods.__pack__ when creating subclasses with pre-defined members)
+        if isinstance(value, dict):
+            value = cls(**value)
 
-        parent = None
+        try:
+            if len(names) != len(value):
+                raise ValueError(
+                    f'invalid value, '
+                    f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
+                    f'got an iterable of length {len(value)}')
+        except TypeError:
+            raise TypeError(
+                f'invalid value, '
+                f'{cls.__name__!r} pack expects an iterable of length {len(names)}, '
+                f'got a non-iterable')
 
         if dump:
             dump.cls = cls
 
-            if isinstance(value, cls):
-                names = value.__plum_names__
-            elif isinstance(value, dict):
-                names = value.keys()
-                value = value.values()
-            elif isinstance(value, (list, tuple)):
-                names = [None] * len(value)
-            else:
-                raise TypeError(f'{cls.__name__!r} pack accepts an iterable')
-
-            for i, (name, item) in enumerate(zip(names, value)):
-                if name:
-                    subdump = dump.add_record(access=f'[{i}] (.{name})')
-                else:
-                    subdump = dump.add_record(access=f'[{i}]')
-
-                value_cls = type(item)
-                if not issubclass(value_cls, Plum):
-                    subdump.value = repr(item)
-                    subdump.cls = value_cls.__name__ + ' (invalid)'
-                    desc = f' ({name})' if name else ''
-                    raise TypeError(
-                        f'anonymous structure member {i}{desc} not a plum type')
-
-                offset = value_cls.__pack__(buffer, offset, parent, item, subdump)
+            for i, (name, value_cls) in enumerate(zip(names, types)):
+                offset = value_cls.__pack__(
+                    buffer, offset, value, value[i], dump.add_record(access=f'[{i}] (.{name})'))
         else:
-            if isinstance(value, dict):
-                value = value.values()
-            elif not isinstance(value, (list, tuple)):
-                raise TypeError(f'{cls.__name__} pack accepts an iterable item')
-
-            for item in value:
-                item_cls = type(item)
-                if not issubclass(item_cls, Plum):
-                    raise TypeError('item in anonymous structure not a plum type instance')
-
-                offset = item_cls.__pack__(buffer, offset, parent, item, dump)
+            for i, value_cls in enumerate(types):
+                offset = value_cls.__pack__(buffer, offset, value, value[i], None)
 
         return offset
 
     @classmethod
     def __unpack__(cls, buffer, offset, parent, dump):
+        # pylint: disable=too-many-locals,unused-argument
+        names, types = cls.__names_types__
+
+        self = list.__new__(cls)
+        append = list.append
+
         if dump:
             dump.cls = cls
-        return list.__new__(cls), offset
+
+            for i, (name, item_cls) in enumerate(zip(names, types)):
+                item, offset = item_cls.__unpack__(
+                    buffer, offset, self,
+                    dump.add_record(access=f'[{i}] (.{name})'))
+                append(self, item)
+        else:
+            for item_cls in types:
+                item, offset = item_cls.__unpack__(buffer, offset, self, dump)
+                append(self, item)
+
+        return self, offset
+
+    def __setattr__(self, name, value):
+        # get the attribute to raise an exception if invalid name
+        getattr(self, name)
+        object.__setattr__(self, name, value)
 
     def __repr__(self):
         lst = [repr(value) if name is None else name + '=' + repr(value)
@@ -111,45 +116,6 @@ class Structure(list, Plum, metaclass=StructureType):
 
         return StructureView(cls, buffer, offset)
 
-    def __getattr__(self, name):
-        # implementation for anonymous structure (metaclass doesn't bother
-        # overriding since its harmless)
-        try:
-            index = object.__getattribute__(self, '__plum_names__').index(name)
-        except (AttributeError, ValueError):
-            # AttributeError -> structure instantiated without names
-            # ValueError -> name not one used during structure instantiation