Commit d65b34d9 authored by Dan Gass's avatar Dan Gass
Browse files

Improve variable structure member support.

parent 326d5456
[run]
branch = True
omit =
*/tests/*
*/test_*.py
[report]
fail_under = 80
......@@ -2,14 +2,12 @@ cd # Compiled python modules.
*.pyc
# Setuptools distribution folder.
/dist/
/build/
*/dist/
*/build/
# Python egg metadata, regenerated from source files by setuptools.
/*.egg-info
/*.egg
/plum_c/*.egg-info
/plum_c/*.egg
**/*.egg-info
**/*.egg
# PyCharm IDE
.idea/*
......
stages:
- test
- test_plum-py
- test_plum-boost
- lint
- build
before_script:
- pip install tox
py36:
stage: test
py36_plum-py:
stage: test_plum-py
image: python:3.6
coverage: '/Total coverage: (\d+\.\d+)%/'
before_script:
- pip install tox
- cd plum-py
script:
- tox -vv -e py36
artifacts:
paths:
- .tox/py36/tmp
- plum-py/.tox/py36/tmp
when: always
py37:
stage: test
py37_plum-py:
stage: test_plum-py
image: python:3.7
coverage: '/Total coverage: (\d+\.\d+)%/'
before_script:
- pip install tox
- cd plum-py
script:
- tox -vv -e py37
artifacts:
paths:
- .tox/py37/tmp
- plum-py/.tox/py37/tmp
when: always
make_docs:
stage: test
py36_plum-boost:
stage: test_plum-boost
image: python:3.6
coverage: '/Total coverage: (\d+\.\d+)%/'
before_script:
- pip install tox
- cd plum-py
script:
- tox -vv -e docs
- tox -vv -e boost
artifacts:
paths:
- .tox/docs/tmp
- plum-py/.tox/boost/tmp
when: always
pylint:
stage: test
py37_plum-boost:
stage: test_plum-boost
image: python:3.7
coverage: '/Total coverage: (\d+\.\d+)%/'
before_script:
- pip install tox
- cd plum-py
script:
- tox -vv -e boost
artifacts:
paths:
- plum-py/.tox/boost/tmp
when: always
pylint_plum-py:
stage: lint
image: python:3.6
before_script:
- pip install tox
- cd plum-py
script:
- tox -vv -e pylint
build:
pylint_plum-boost:
stage: lint
image: python:3.6
before_script:
- pip install tox
- cd plum-boost
script:
- tox -vv -e pylint
pylint_plum-benchmark:
stage: lint
image: python:3.6
before_script:
- pip install tox
- cd plum-benchmark
script:
- tox -vv -e pylint
pylint_plum-tests:
stage: lint
image: python:3.6
before_script:
- pip install tox
- cd plum-tests
script:
- tox -vv -e pylint
build_plum-py:
stage: build
image: python:3.6
before_script:
- pip install tox
- cd plum-py
script:
- tox -vv -e build
artifacts:
paths:
- plum-py/.tox/build/tmp
when: always
build_plum-boost:
stage: build
image: python:3.6
before_script:
- pip install tox
- cd plum-boost
script:
- tox -vv -e build
artifacts:
paths:
- .tox/build/tmp
- plum-boost/.tox/dist
when: always
build_plum-benchmark:
stage: build
image: python:3.6
before_script:
- pip install tox
- cd plum-benchmark
script:
- tox -vv -e build
artifacts:
paths:
- plum-benchmark/.tox/build/tmp
when: always
build_plum-tests:
stage: build
image: python:3.6
before_script:
- pip install tox
- cd plum-tests
script:
- tox -vv -e build
artifacts:
paths:
- plum-tests/.tox/build/tmp
when: always
make_docs:
stage: build
image: python:3.6
before_script:
- pip install tox
- cd plum-py
script:
- tox -vv -e docs
artifacts:
paths:
- plum-py/.tox/docs/tmp
when: always
......@@ -3,7 +3,7 @@
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code.
extension-pkg-whitelist=
extension-pkg-whitelist=plum_boost
# Add files or directories to the blacklist. They should be base names, not
# paths.
......@@ -228,7 +228,7 @@ const-naming-style=UPPER_CASE
# Regular expression matching correct constant names. Overrides const-naming-
# style.
#const-rgx=
const-rgx=(([A-Z_][A-Z_0-9]*)|([a-z_][a-z_0-9]*))
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
......@@ -485,7 +485,7 @@ defining-attr-methods=__init__,
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,
_fields,
__fields__,
_replace,
_source,
_make
......@@ -494,13 +494,13 @@ exclude-protected=_asdict,
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=cls
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# Maximum number of arguments for function / method.
max-args=5
max-args=6
# Maximum number of attributes for a class (see R0902).
max-attributes=7
......@@ -527,7 +527,7 @@ max-returns=6
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
min-public-methods=0
[IMPORTS]
......
......@@ -92,7 +92,7 @@ By adding my name to this signature list and dating, I acknowledge:
--------- --------------------------------------------------------
01Mar2019 Jan Novak (job.jan.novak@gmail.com)
16Mar2019 Tyler N. Thieding (python@thieding.com)
09Apr2019 Daniel Mark Gass (dan.gass@gmail.com)
29Apr2019 Daniel Mark Gass (dan.gass@gmail.com)
****************
......
......@@ -75,51 +75,12 @@ What's Next?
- make accelerators for more types (prioritizing the |Structure| type)
- further optimize "C" and Python implementations
- Support "views" of memory, where a "view" associates a |plum| type to a memory
proxy at a particular memory offset.
- Develop memory proxy interface such that a "view" follows a certain protocol
to get/set memory such that memory may be local (e.g. a large file) or remote
(e.g. another process or even memory in a networked device).
>>> memory = LocalMemory(bytearray([0, 1, 99, 3, 4]))
>>> memory
LocalMemory([0, 1, 99, 3, 4])
- For the core |plum| types, develop associated "proxy" types that are joined at the hip
such that a "view" for a given |plum| type (e.g. UInt8) instantiates the
associated proxy type (e.g. UInt8Proxy) with the user provided memory interface
instance and byte offset into that memory.
>>> x = view(UInt8, memory, offset=2)
>>> type(x)
UInt8Proxy
>>> x.__memory__
LocalMemory([0, 1, 99, 3, 4])
>>> x.__offset__
2
- When utilized (e.g. a ``__repr__`` or ``__eq__`` invocation), the proxy instance
retrieves the bytes from ``__memory__`` at the recorded ``__offset__``, unpacks
the bytes using the associated |plum| type (e.g. UInt8), and uses the unpacked
value to complete the operation.
>>> x # invoke proxy __repr__
UInt8Proxy(99)
>>>
>>> x == 99 # invoke proxy __eq__
True
- Support interpretting the memory in the form of the original |plum| type:
>>> getvalue(x)
UInt8(99)
- Support altering the memory:
>>> setvalue(x, 55)
>>> memory
LocalMemory([0, 1, 55, 3, 4])
- Expand support for "views" of memory.
- Develop additional memory view interfaces such that memory may be local (e.g. a large file)
or remote (e.g. another process or even memory in a networked device).
- For the remaining core |plum| types, develop associated "view" types.
- Use cases:
- interpret portions of large memory image with low overhead
......
@ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=.
set BUILDDIR=_build
set SPHINXPROJ=MarryDoc
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
:end
popd
@ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=.
set BUILDDIR=_build
set SPHINXPROJ=MarryDoc
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
:end
popd
File mode changed from 100755 to 100644
......@@ -351,7 +351,7 @@ values that are powers of two:
>>> class Status16(Flag, nbytes=2):
... CARRY = 1
... OVERFLOW = 4
... PARITY = 8
... PARITY = 8
...
>>> status = Status16.CARRY
>>> dump(status)
......@@ -510,3 +510,48 @@ unpacked item and a dump object summarizing the unpacked item.
| 1 | [1] | 2 | 02 | UInt8 |
| 2 | [2] | 3 | 03 | UInt8 |
+--------+--------+-------+--------+-------+
Memory Views
++++++++++++
The |plum| package provides a :class:`LocalMemory` interface with a get/set bytes protocol. For
example::
>>> from plum.memory import LocalMemory
>>> memory = LocalMemory(bytearray([0, 1, 99, 3, 4]))
>>> memory
LocalMemory([0, 1, 99, 3, 4])
>>>
>>> # Get bytes in the memory interface.
>>> memory.get_bytes(4)
bytearray(b'\x00\x01c\x03')
>>> memory.get_bytes(2, offset=3)
bytearray(b'\x03\x04')
>>>
>>> # Set bytes in the memory interface.
>>> memory.set_bytes(bytearray([5, 6]), offset=3)
>>> memory
LocalMemory([0, 1, 99, 5, 6])
Leverage the :func:`view` function to instantiate a view type to the memory bytes in the specified
memory interface. This view type is joined at the hip such that a "view" for a given |plum| type
(e.g. UInt8) instantiates the associated view type (e.g. IntView) with the user provided memory
interface instance and byte offset into that memory. For example::
>>> from plum import view
>>> from plum.int.big import UInt8
>>> x = view(UInt8, memory, offset=2)
>>> x
IntView(99)
>>> x == 99
True
Performing operations on the view type also updates the base memory. For example::
>>> x -= 92
>>> x
IntView(7)
>>> memory
LocalMemory([0, 1, 7, 5, 6])
Observe that the :class:`LocalMemory` instance's bytes reflect the change to the integer view.
......@@ -17,15 +17,19 @@ Utility Functions
:func:`calcsize` Get size of packed item (in bytes).
:func:`dump` Print packed memory summary.
:func:`exhaust` Verify all memory bytes consumed.
:func:`getbytes` Get memory bytes.
:func:`getdump` Get packed memory summary.
:func:`getvalue` Unpack item from memory bytes of a plum view.
:func:`pack` Pack items and return memory bytes.
:func:`pack_and_getdump` Pack items and return memory bytes and summary.
:func:`pack_into` Pack items into memory bytes.
:func:`pack_into` Pack items into memory bytes and return memory summary.
:func:`setvalue` Pack value into memory bytes of a plum view.
:func:`unpack` Unpack item from memory bytes.
:func:`unpack_from` Unpack item from binary file stream.
:func:`unpack_and_getdump` Unpack item from memory bytes and get memory summary.
:func:`unpack_from_and_getdump` Unpack item from binary file stream and get memory summary.
:func:`view` Create plum view of memory bytes.
=============================== =======================================================================
.. toctree::
......@@ -34,15 +38,19 @@ Utility Functions
calcsize() <utilities/calcsize.rst>
dump() <utilities/dump.rst>
exhaust() <utilities/exhaust.rst>
getbytes() <utilities/getbytes.rst>
getdump() <utilities/getdump.rst>
getvalue() <utilities/getvalue.rst>
pack() <utilities/pack.rst>
pack_and_getdump() <utilities/pack_and_getdump.rst>
pack_into() <utilities/pack_into.rst>
pack_into_and_getdump() <utilities/pack_into_and_getdump.rst>
setvalue() <utilities/setvalue.rst>
unpack() <utilities/unpack.rst>
unpack_from() <utilities/unpack_from.rst>
unpack_and_getdump() <utilities/unpack_and_getdump.rst>
unpack_from_and_getdump() <utilities/unpack_from_and_getdump.rst>
view() <utilities/view.rst>
**********
......@@ -92,6 +100,7 @@ Modules
:mod:`plum.int.flag.native` native endian integer with bit flags
:mod:`plum.int.little` little endian integers
:mod:`plum.int.native` native endian integers
:mod:`plum.memory` memory bytes interface
:mod:`plum.nil` no memory bytes
:mod:`plum.str` string
:mod:`plum.structure` structure of uniquely typed items
......@@ -120,6 +129,7 @@ Modules
int.flag <modules/int_flag_native.rst>
int.little <modules/int_little.rst>
int.native <modules/int_native.rst>
memory <modules/memory.rst>
nil <modules/nil.rst>
str <modules/str.rst>
structure <modules/structure.rst>
##############################
[plum.memory] Module Reference
##############################
.. include:: ../../alias.txt
.. automodule:: plum.memory
.. autoclass:: MemoryInterface
:members:
.. autoclass:: LocalMemory
......@@ -10,19 +10,19 @@
.. autoclass:: Structure(mapping, **kwargs)
.. autoclass:: Varies
.. autofunction:: dims_of
.. autofunction:: limit_size
.. autofunction:: dims_via
.. autofunction:: measure_dims
.. autofunction:: member
.. autofunction:: measure_size
.. autofunction:: size_of
.. autofunction:: member
.. autofunction:: size_via
.. autofunction:: switch_type
.. autofunction:: type_of
.. autofunction:: transfer_dims
.. autofunction:: type_via
.. seealso::
......
####################################
[plum] Utility Reference: getbytes()
####################################
.. include:: ../../alias.txt
.. currentmodule:: plum
.. autofunction:: getbytes
####################################
[plum] Utility Reference: getvalue()
####################################
.. include:: ../../alias.txt
.. currentmodule:: plum
.. autofunction:: getvalue
####################################
[plum] Utility Reference: setvalue()
####################################
.. include:: ../../alias.txt
.. currentmodule:: plum
.. autofunction:: setvalue
################################
[plum] Utility Reference: view()
################################
.. include:: ../../alias.txt
.. currentmodule:: plum
.. autofunction:: view
......@@ -8,6 +8,21 @@ Versions increment per `semver <http://semver.org/>`_ (except for alpha series).
Alpha (0.1.0aX) Versions (only serial number X increments)
**********************************************************
+ 0.1.0a7 (2019-Apr-30)
- Improve variably dimensioned structure member support.
- Change `measure_dims()` to `dims_of()`.
- Change `transfer_dims()` to `dims_via()`.