Commit 6f7a5ddf authored by Martin Holicky's avatar Martin Holicky
Browse files

Major updates for CAN, adding support for python-can.

parent af606a89
[submodule "matoha-message"]
path = matoha-message
[submodule "mcl/matoha_message"]
path = mcl/matoha_message
url = https://gitlab.com/matohascience/matoha-message.git
MIT License
Copyright 2019 Matoha Instrumentation Ltd.
Copyright 2021 Matoha Instrumentation Ltd.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
......
# Matoha Communication Library
# Matoha Communication Library - Python
This library provides interfaces for communication with our instruments.
This library provides the Python interface for communicating with our devices.
## Python
## Protocols and interfaces
See python/README.md
#### The protocol
## JavaScript
We use [Protocol Buffers](https://en.wikipedia.org/wiki/Protocol_Buffers) to transport data in an efficient manner.
Work in progress - don't use yet.
Our PB message definition which can be easily implemented across various programming languages can found in the
[matoha-message](https://gitlab.com/matohascience/matoha-message) repository. Please have a look into this repository
for the documentation of the protocol and the commands.
Currently supported interfaces are below. More will be added in the future; if you have specific requirements please
contact us.
#### CAN bus
[CAN bus](https://en.wikipedia.org/wiki/CAN_bus) is a widely use network protocol for communication across many nodes.
Requiring only two wires, it supports relatively high speed (250-1000 kbits/s) communications, with many nodes connected
to the same two physical wires. One drawback is that the packet payload size is restricted to 8 bytes (ignoring the CAN-FD
mode which is not supported by our spectrometers as it is covered by 3rd-party patents).
In order to send larger than 8-byte messages across the CAN bus, our spectrometers implement the
[ISO-TP](https://en.wikipedia.org/wiki/ISO_15765-2) transport layer. ISO-TP splits our transmission into 8-bytes chunks
and adds headers ensuring the whole message is received in the correct order. There are many libraries available that
implement ISO-TP, for instance the [Python library](https://github.com/pylessard/python-can-isotp) or the
[Arduino library](https://github.com/altelch/iso-tp). The Python library linked is used internally by Matoha for
this package.
#### Other buses and interfaces
It should be technically feasible to support the following protocols:
- Bluetooth LE GATT
- WiFi HTTP server or client
- WiFi Websockets client (the device would be the client)
- UART (RS232)
- I2C
- Google/AWS IOT systems
Please do not hesitate to contact us to discuss your requirements.
## Basic usage
### Installation via pip
Coming soon, for now please used the steps below.
### Installation for development
- Clone the repository, *including the submodules*.
```
git clone --recurse-submodules git@gitlab.com:matohascience/mcl.git
cd mcl
```
- (Optional but recommended) Create a virtual environment
```
python3 -m venv venv && source venv/bin/activate
```
- Install Python dependencies
```
pip3 install -r mcl/requirements.txt
pip3 install -r mcl/matoha_message/requirements.txt
```
- Compile Matoha Message
```
cd mcl/matoha_message
./compile_protobufs.sh
cd ../../
```
### Running examples
Currently, two CAN interfaces are supported GS-CAN and python-can. Please choose the relevant example and open the file.
You may need to modify the configuration to match your setup (see the instructions inside the example). Finally,
you should be able to run the example, e.g.
`python3 examples/pythoncan_comms.py`
## Developer instructions
### Building the wheel
To create a distributable .whl, please run
```bash
pip3 install setuptools wheel twine
python3 setup.py sdist bdist_wheel
```
The latest build shall be found in the top folder as a .whl.
TODO: automatic builds / PIP integration.
## License
MIT license - see the LICENSE file.
\ No newline at end of file
(c) 2021 Matoha Instrumentation Ltd, licensed under the open-source MIT license.
See the LICENSE file for further details.
\ No newline at end of file
# *Communications example using a GS-CAN adapter*
#
# Warning - as we had stability issues with GSCAN on Mac, we have internally switched to python-can. If GS-CAN works for
# you - feel free to continue using it
#
# This example assumes a GS-CAN (Geschwister Schneider, candleLight) USB to CAN adapter is connected to your
# computer.
#
# The driver used is https://pypi.org/project/gs-usb/ , available via pip under the MIT License
#
# (c) 2021 Matoha Instrumentation Ltd, Licensed under the MIT license
from gs_usb.gs_usb import GsUsb
from gs_usb.gs_usb_frame import GsUsbFrame
import isotp
import time
# This is to allow relative import
import sys
sys.path.append('../')
sys.path.append('.')
try:
from ..can_device import CANDevice
except:
from can_device import CANDevice
import os
sys.path.append(os.path.realpath('.'))
from mcl.matoha_can import MatohaCAN
import time
# Adjust this to your configuration, numbers below are Matoha default
can_bitrate = 250000
......@@ -20,7 +30,11 @@ matoha_can_tx_offset = 1
matoha_can_rx_offset = 0
can_adapter = None
# Set this to true to print all the frames coming in/out
print_traffic = False
# Set this to true to print device logs as they come in
print_logs = True
......@@ -53,7 +67,7 @@ def rx_function():
if can_adapter.read(frame, 1):
if frame.can_id != matoha_can_id+matoha_can_rx_offset and frame.can_id != matoha_can_id+matoha_can_tx_offset:
# Not our traffic, ignore
# print(f"RX {frame} --- drop")
print(f"RX {frame} --- drop")
return
if print_traffic:
......@@ -90,7 +104,7 @@ def shutdown_function():
if __name__ == '__main__':
matoha_dev = CANDevice(can_tx_address=matoha_can_id + matoha_can_tx_offset,
matoha_dev = MatohaCAN(can_tx_address=matoha_can_id + matoha_can_tx_offset,
can_rx_address=matoha_can_id + matoha_can_rx_offset,
setup_function=setup_function,
shutdown_function=shutdown_function,
......@@ -101,16 +115,23 @@ if __name__ == '__main__':
start = time.time()
things_to_do = [matoha_dev.start_measuring, matoha_dev.stop_measuring, matoha_dev.single_measurement,
matoha_dev.request_status, matoha_dev.sleep, matoha_dev.wake_up]
# The functions below will be called in periodic intervals
things_to_do = [
matoha_dev.start_measuring, matoha_dev.stop_measuring, matoha_dev.single_measurement,
matoha_dev.request_status,
matoha_dev.sleep,
matoha_dev.wake_up,
matoha_dev.request_status
]
while 1:
time.sleep(0.2)
time.sleep(0.1)
if matoha_dev.available():
msg = matoha_dev.receive()
if "log" in msg:
if print_logs:
print(f"Device log: {msg['log']}")
print(f"Device log: {msg['log']}", end="")
else:
print(msg)
......
# *Communications example using the python-can library*
#
# This example should support any python-can adapter.
#
# The driver used is https://python-can.readthedocs.io/en/master/ , available via pip under the LGPL License
#
# (c) 2021 Matoha Instrumentation Ltd, Licensed under the MIT license
import can
import isotp
import time
import sys
import os
sys.path.append(os.path.realpath('.'))
from mcl.matoha_can import MatohaCAN
# Adjust this to your configuration, numbers below are Matoha default
can_bitrate = 250000
matoha_can_id = 0x400
matoha_can_tx_offset = 1
matoha_can_rx_offset = 0
# Adjust specific to your CAN adapter
# The settings below are for USBTin, which we use internally - but other adapters will, of course, be supported.
# For bustype and chanel values see https://python-can.readthedocs.io/en/master/interfaces.html
bustype = "slcan"
channel = "/dev/tty.usbmodemA02103A41"
can_adapter = None
# Set this to true to print all the frames coming in/out
print_traffic = False
# Set this to true to print device logs as they come in
print_logs = True
def setup_function():
"""
Function to call when setting up the bus
:return: None
"""
global can_adapter
can_adapter = can.Bus(channel=channel, bustype=bustype, bitrate=can_bitrate)
def rx_function():
"""
Custom receive function to receive data over a GS CAN adapter from a Matoha device
:return: isotp.CanMessage object with the received frame
"""
global can_adapter
frame = can_adapter.recv(0.001)
if frame:
if frame.arbitration_id != matoha_can_id+matoha_can_rx_offset:
# Not our traffic, ignore
#print(f"RX {frame} --- drop")
return
elif print_traffic:
print(f"RX {frame}")
return isotp.CanMessage(arbitration_id=frame.arbitration_id, data=frame.data, extended_id=frame.is_extended_id,
is_fd=frame.is_fd, dlc=frame.dlc)
def tx_function(msg):
"""
Custom receive function to transmit data over a GS CAN adapter to a Matoha device
:param msg: ISOTP message object
:return: None
"""
global can_adapter
msg_obj = can.Message(data=msg.data, arbitration_id=msg.arbitration_id)
can_adapter.send(msg_obj)
if print_traffic:
print(f"TX {msg_obj}")
def shutdown_function():
"""
Custom shutdown function to gracefully terminate our GS CAN adapter comms.
:return: None
"""
global can_adapter
can_adapter.shutdown()
if __name__ == '__main__':
matoha_dev = MatohaCAN(can_tx_address=matoha_can_id + matoha_can_tx_offset,
can_rx_address=matoha_can_id + matoha_can_rx_offset,
setup_function=setup_function,
shutdown_function=shutdown_function,
rx_function=rx_function,
tx_function=tx_function
)
matoha_dev.start_comms()
start = time.time()
# The functions below will be called in periodic intervals
things_to_do = [
matoha_dev.start_measuring,
matoha_dev.request_status,
matoha_dev.stop_measuring,
matoha_dev.single_measurement,
matoha_dev.request_status,
matoha_dev.sleep,
matoha_dev.request_status,
matoha_dev.wake_up,
matoha_dev.request_status
]
while 1:
time.sleep(0.1)
if matoha_dev.available():
msg = matoha_dev.receive()
if "log" in msg:
if print_logs:
print(f"Device log: {msg['log']}", end="")
else:
print(msg)
# Optionally terminate when nothing else to do
#if len(things_to_do) == 0:
# break
# Every 10 seconds send a command
if (time.time()-start) > 10 and len(things_to_do) > 0:
thing_to_do = things_to_do.pop(0)
print(f"Executing: {thing_to_do}")
thing_to_do()
start = time.time()
matoha_dev.terminate_comms()
/*
Matoha Communication Library
Copyright 2019 Matoha Instrumentation Ltd.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/*
This is work in progress and very messy - don't use (yet)
*/
let state = "UNCALIBRATED";
let attributes = {};
let previous_attributes = {};
materials_names =[];
materials_ids = [];
materials_content = [];
$(document).ready(function(){
setState("UNCALIBRATED");
// This code adds scrolling between bookmarks/anchors
let anchorlinks = document.querySelectorAll('a[href^="#"]');
for (let item of anchorlinks) {
item.addEventListener('click', (e)=> {
e.preventDefault();
let hashval = item.getAttribute('href');
let validate = item.getAttribute('data-validate');
if(validate === "sample-id-strict"){
if(!checkID(true)){
return;
}
}
else if(validate === "sample-id"){
checkID(false);
}
let target = document.querySelector(hashval);
target.scrollIntoView({
behavior: 'instant',
block: 'start'
});
//history.pushState(null, null, hashval);
});
}
const SOCKET_URI = "ws://"+window.location.hostname+":5003";
websocket = new WebSocket(SOCKET_URI);
websocket.onopen = function(evt) { onSocketOpen(evt) };
websocket.onclose = function(evt) { onSocketClose(evt) };
websocket.onmessage = function(evt) { onSocketMessage(evt) };
websocket.onerror = function(evt) { onSocketError(evt) };
disableScroll();
cancel_button.css("visibility", "hidden");
});
function setState(new_state){
state = new_state;
setButtons(state);
if(state === "MEASURING"){
loader.css("visibility", "visible");
setMessage("");
} else{
loader.css("visibility", "hidden");
}
}
function performMeasurement() {
websocket.send("SINGLE_MEASUREMENT")
}
function measureColour() {
websocket.send("MEASURE_COLOUR")
}
function calibrate() {
websocket.send("CALIBRATE")
}
function performContinuousMeasurement() {
websocket.send("START_MEASUREMENT");
}
function stopContinuousMeasurement() {
websocket.send("STOP_MEASUREMENT");
}
function saveMeasurement() {
let sample_data = JSON.stringify(attributes);
console.log("Sample data: "+ sample_data);
websocket.send("SAVE_MEASUREMENT:" + sample_data);
previous_attributes = attributes;
clearSampleInfo();
disableScroll();
hideX();
}
function onSocketOpen(evt) {
console.log("Matoha Device running.");
}
function onSocketClose(evt) {
console.warn("Error: can't connect to the core. If problems persist please restart the app.");
}
function onSocketMessage(evt) {
console.log(evt);
const message = JSON.parse(evt.data);
switch (message.type){
case "DEVICE_STATUS":
$("#device-msg").html(message.payload);
break;
case "MESSAGE":
setMessage(message.payload,'result');
const save_match = /Sample #([0-9]+)/;
if(message.payload.match(save_match) !== null){
let sample_id = message.payload.match(save_match)[1];
previous_attributes["sample_id"] = sample_id;
$("#previous-sample-btn").html("Previous sample (#"+sample_id+")");
}
break;
case "DEBUG_MESSAGE":
$("#debug-msg").html(message.payload);
break;
case "ERROR":
setMessage(message.payload,'failure');
break;
case "COLOUR":
$("#colour-message").html(message.payload);
break;
case "STATE":
setState(message.payload);
break;
case "GRAPH":
if(message.payload.includes("svg")) {
$(".graph-image").html("Sample fingerprint: " + message.payload);
} else {
$(".graph-image").html(message.payload);
}
break;
default:
console.log("Unknown message type. " + message.type);
}
}
function onSocketError(evt) {
setMessage("WebSocket Communication Error.",'failure')
}
Subproject commit 217b16666795f9e281396be93a2f37ff7970efbb
import isotp
import logging
import time
import threading
from message_parser import MatohaMessage
import matoha_message_pb2
from .parsed_message import ParsedMessage
from .matoha_message import matoha_message_pb2
class CANDevice:
class MatohaCAN:
"""
Main driver for communicating with Matoha devices over CAN with ISO-TP transport layer.
"""
......@@ -61,24 +60,44 @@ class CANDevice:
:param error: CAN error
:return: None
"""
logging.warning(f"CAN Error! {error}")
print(f"CAN Error! {error}")
def thread_task(self):
"""
Periodically executed task checking for exit flags and incoming data
:return: None
"""
while not self.exit_requested:
self.layer.process() # Non-blocking
time.sleep(self.layer.sleep_time()) # Variable sleep time based on state machine state
def terminate_comms(self):
"""
Terminate the communications with the device / CAN-USB adapter.
:return: None
"""
self.stop_comms()
if self.shutdown_function:
self.shutdown_function()
def available(self):
"""
Check if new messages have been sent by the device
:return: True if message(s) are available
"""
return self.layer.available()
def receive(self):
"""
Receive the new messages, nicely formatted by Matoha codes.
:return: parsed dictionary, see the matoha-message repo documentation
"""
data = self.layer.recv()
parsed = MatohaMessage(data)
parsed = ParsedMessage(data)
return parsed.to_dict()
def _send_command(self, command):
......@@ -87,22 +106,57 @@ class CANDevice:
self.layer.send(msg.SerializeToString())
def start_measuring(self):
"""
Enable the measurement auto-trigger.
:return: None
"""
self._send_command(matoha_message_pb2.MatohaMessage.command_t.START_MEASURING)
def stop_measuring(self):
"""
Disable the measurement auto-trigger.
:return: None
"""
self._send_command(matoha_message_pb2.MatohaMessage.command_t.STOP_MEASURING)
def single_measurement(self):
"""
Force a measurement, regardless of the auto-trigger. We recommend stopping the measurement before (stop_measuring())
:return: None
"""
self._send_command(matoha_message_pb2.MatohaMessage.command_t.SINGLE_MEASUREMENT)
def restart_device(self):
"""
Restart the device.
:return: None
"""
self._send_command(matoha_message_pb2.MatohaMessage.command_t.RESTART)
def request_status(self):
"""
Request the device status.
:return: None
"""
self._send_command(matoha_message_pb2.MatohaMessage.command_t.SEND_STATUS)
def sleep(self):
"""