... | ... | @@ -33,7 +33,7 @@ typedef struct { |
|
|
} can_message_s;
|
|
|
```
|
|
|
Now we need to send this data message to the cloud. A popular choice is using [BSD Sockets](https://en.wikipedia.org/wiki/Berkeley_sockets). After establishing connection with the cloud and obtaining the socket file descriptor `sockfd` we can use [`send` Socket API](https://pubs.opengroup.org/onlinepubs/009695399/functions/send.html) to send our data.
|
|
|
```
|
|
|
```c
|
|
|
/**
|
|
|
* @param sockfd Specifies the socket file descriptor.
|
|
|
* @param buf Points to the buffer containing the message to send.
|
... | ... | @@ -65,23 +65,26 @@ Sibros repository Integrates Nanopb library in [Bazel](https://www.bazel.build/) |
|
|
To use Nanopb, protocol compiler needs to be installed.
|
|
|
Install `Python3` and `Python3-Pip` package:
|
|
|
```
|
|
|
apt-get install python3 python3-pip
|
|
|
$ apt-get install python3 python3-pip
|
|
|
```
|
|
|
Install `protobuf` and `grpcio-tools` packages need by Nanopb:
|
|
|
```
|
|
|
pip3 install protobuf grpcio-tools
|
|
|
$ pip3 install protobuf grpcio-tools
|
|
|
```
|
|
|
|
|
|
Protocol Buffers messages are defined in a `proto file` as follows:
|
|
|
```
|
|
|
```c
|
|
|
//foo.proto
|
|
|
|
|
|
message Foo {
|
|
|
required int id = 1;
|
|
|
}
|
|
|
```
|
|
|
This `.proto` file is used to generate the Nanopb headers(.h) and source files(.c) using the python script provided by Nanopb.
|
|
|
```
|
|
|
python3 ../../generator/nanopb_generator.py foo.proto
|
|
|
$ git clone https://github.com/nanopb/nanopb.git
|
|
|
$ cd nanopb
|
|
|
$ python3 generator/nanopb_generator.py /path/to/foo.proto
|
|
|
```
|
|
|
|
|
|
# Implementation and Examples
|
... | ... | @@ -92,24 +95,69 @@ Download the example files from our [public repository](https://gitlab.com/sibro |
|
|
|
|
|
## Example - 1
|
|
|
|
|
|
The source files for this example can be found [here](https://gitlab.com/sibros_public/sibros-open-source/nanopb-examples/-/tree/master/ex1)
|
|
|
|
|
|
### Proto Files
|
|
|
Let's say you want to transmit a Can message in a serialized protobuf format. Then we need to define the can message and its contents inside a proto file as follows:
|
|
|
Let's say you want to transmit a CAN message in a serialized protobuf format. Then we need to define the CAN message and its contents inside a proto file as follows:
|
|
|
|
|
|
```
|
|
|
//can.proto
|
|
|
message can_message {
|
|
|
required int64 timestamp_ms = 1;
|
|
|
```c
|
|
|
// ex1/ex1.proto
|
|
|
message proto_can_message {
|
|
|
required int32 message_id = 1; // Standard 11-bit
|
|
|
|
|
|
required int32 bus_id = 2; // The bus number which received the CAN message
|
|
|
required int32 message_id = 3; // Standard 11-bit
|
|
|
|
|
|
required int64 timestamp_ms = 3;
|
|
|
required int32 data_byte = 4; // Usually 8 bytes but modified for simplified example
|
|
|
}
|
|
|
```
|
|
|
In this file a message to be transmitted is defined using `message` keyword, followed by message name. There are 3 members inside the message with int64 and int32 data types respectively. The member of the message has to be initialized if `required` keyword is used. Alternatively `option` keyword can also be used. The `required int64 timestamp_ms = 1` the number after equality operator is the tag of the message member, which are used to match fields when serializing and deserializing the data.
|
|
|
For more information about `proto` message, visit the [documentation.](https://developers.google.com/protocol-buffers/docs/overview).
|
|
|
In the `proto` file a message to be transmitted is defined using `message` keyword, followed by message name (`proto_can_message` in the exampe above). The proto message can have multiple data members such as message_id, bus_id, timestamp_ms, data_byte as above.
|
|
|
|
|
|
If `required` keyword is used before tahe declaration of the message's member, the member has to be initialized. Alternatively `option` keyword can also be used which makes initializing the data member optional.
|
|
|
|
|
|
The the number after equality operator is the tag of the message member which are used to match fields when serializing and deserializing the data. For Eg. the tag number for `timestamp_ms` is 1.
|
|
|
|
|
|
For more information about `proto` message, visit [Google's proto file documentation.](https://developers.google.com/protocol-buffers/docs/overview)
|
|
|
|
|
|
### Compiling proto files and generation of C header(.h) file
|
|
|
|
|
|
On running the python script to generate header and source files from proto file, we get structure declared in (.h) file:
|
|
|
For Bazel users, we create an instance (or target) of `cc_nanopb_library` rule, which when executed will compile the proto file and generate the (.h) file.
|
|
|
|
|
|
```python
|
|
|
# ex1/BUILD
|
|
|
|
|
|
load("@rules_proto//proto:defs.bzl", "proto_library")
|
|
|
load("//embedded/infrastructure/bazel/rules:nanopb.bzl", "cc_nanopb_library")
|
|
|
|
|
|
proto_library(
|
|
|
name = "ex1_proto",
|
|
|
srcs = ["ex1.proto"],
|
|
|
)
|
|
|
|
|
|
cc_nanopb_library(
|
|
|
name = "ex1_c_proto",
|
|
|
deps = [
|
|
|
":ex1_proto",
|
|
|
],
|
|
|
)
|
|
|
```
|
|
|
|
|
|
Building the target we obtain the Nanopb generated file.
|
|
|
```
|
|
|
$ bazel build //shared/temp_work/nanopb-examples/ex1:ex1_c_proto
|
|
|
|
|
|
bazel-bin/shared/temp_work/nanopb-examples/ex1/ex1.npb.h
|
|
|
```
|
|
|
|
|
|
Non-Bazel users, can generate the header (.h) file using the python script provided by Nanopb:
|
|
|
|
|
|
```
|
|
|
$ cd nanopb
|
|
|
$ python3 generator/nanopb_generator.py /path/to/nanopb-examples/ex1/ex1.proto
|
|
|
```
|
|
|
|
|
|
The Nanopb generated header (.h) file will create an proto message equivalent C strucutre as follows:
|
|
|
```c
|
|
|
/* Struct definitions */
|
|
|
typedef struct _can_message {
|
|
|
int64_t timestamp_ms;
|
... | ... | @@ -120,11 +168,13 @@ typedef struct _can_message { |
|
|
} can_message;
|
|
|
```
|
|
|
|
|
|
Refer to the Nanopb generated header [here]().
|
|
|
|
|
|
### Encoding:
|
|
|
|
|
|
Now we can use this generated structure to populate our data to be serialized as follows:
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
proto_msg->timestamp_ms = can_msg->timestamp_ms;
|
|
|
proto_msg->bus_id = can_msg->bus_id;
|
|
|
proto_msg->message_id = can_msg->message_id;
|
... | ... | @@ -133,12 +183,12 @@ Now we can use this generated structure to populate our data to be serialized as |
|
|
|
|
|
We then create an output stream for writing the data to the buffer:
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
pb_ostream_t stream = pb_ostream_from_buffer(encoded_packet->buffer, encoded_packet->encoded_packet_size);
|
|
|
```
|
|
|
|
|
|
The Protobuf message is then encoded to a serialized format using Nanopb's `pb_encode()` API.
|
|
|
```
|
|
|
```c
|
|
|
if (!pb_encode(&stream, proto_can_message_fields, proto_msg)) {
|
|
|
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
|
|
|
return 1;
|
... | ... | @@ -149,15 +199,15 @@ if (!pb_encode(&stream, proto_can_message_fields, proto_msg)) { |
|
|
|
|
|
To verify integrity of encoded data, we decode the encoded packet. First we create a stream that reads from the encoded buffer.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
pb_istream_t stream = pb_istream_from_buffer(packet->buffer, packet->bytes_written);
|
|
|
```
|
|
|
The we decode the encoded message by using Nanopb's decoding API such that proto message ie `proto_can_message* proto_msg` is re-populated.
|
|
|
```
|
|
|
```c
|
|
|
pb_decode(&stream, proto_can_message_fields, proto_msg)
|
|
|
```
|
|
|
Finally we re-populate the can message using the newly populated `proto_msg` ie. Copy decoded data to can message
|
|
|
```
|
|
|
```c
|
|
|
can_msg->message_id = proto_msg->message_id;
|
|
|
can_msg->bus_id = proto_msg->bus_id;
|
|
|
can_msg->timestamp_ms = proto_msg->timestamp_ms;
|
... | ... | @@ -168,7 +218,7 @@ Finally we re-populate the can message using the newly populated `proto_msg` ie. |
|
|
|
|
|
A CAN message usually has a 8 byte data field (if using CAN-FD the data field is 64 bytes). Thus we modify the can message in the `proto` file such that the CAN proto message can encode 8 byte of data at once.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
//can.proto
|
|
|
message can_message {
|
|
|
required int64 timestamp_ms = 1;
|
... | ... | @@ -180,7 +230,7 @@ message can_message { |
|
|
|
|
|
After generating the (.h) file again the message is modified as follows:
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
/* Struct definitions */
|
|
|
typedef struct _proto_can_message {
|
|
|
int64_t timestamp_ms;
|
... | ... | @@ -201,7 +251,7 @@ Thus, members of a message are generated as `pb_callback_t` datatypes for variab |
|
|
For more information on callbacks in Nanopb visit [here](https://jpa.kapsi.fi/nanopb/docs/concepts.html#field-callbacks).
|
|
|
|
|
|
The callback structure is declared as follows:
|
|
|
```
|
|
|
```c
|
|
|
typedef struct _pb_callback_t pb_callback_t;
|
|
|
struct _pb_callback_t {
|
|
|
union {
|
... | ... | @@ -221,13 +271,13 @@ If the function pointer is `NULL`, the corresponding message field will be skipp |
|
|
|
|
|
Let's consider we would like to encode a CAN message as follows:
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
can_message_s can_msgs_to_encode = {.message_id = 111U, .timestamp_ms = 5U, .dlc = 8U, .bus_id = 2};
|
|
|
memset(can_msgs_to_encode.data, 0x0E, 8);
|
|
|
```
|
|
|
Here, the data field of CAN message has 8 bytes, and each byte is set to the value `0x0E`. The CAN message is sent to the encode function as follows:
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
size_t encode_packet(proto_can_message *proto_msg, serial_data_packet_s *packet, can_message_s *can_msg)
|
|
|
```
|
|
|
|
... | ... | @@ -237,7 +287,7 @@ This is required since the data field of CAN message has variable amount of byte |
|
|
* `proto_msg->data_byte.funcs.encode = &callback_encode_can_bytes`
|
|
|
Will hold the function pointer for the callback funtion.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
/* Fill the proto packet */
|
|
|
proto_msg->timestamp_ms = can_msg->timestamp_ms;
|
|
|
proto_msg->bus_id = can_msg->bus_id;
|
... | ... | @@ -249,7 +299,7 @@ Will hold the function pointer for the callback funtion. |
|
|
```
|
|
|
We can then the encode the data using Nanopb's `pb_encode` function. The `pb_encode` will internally call the callback registered above to encode all the data bytes inside the CAN message.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
if (!pb_encode(&stream, proto_can_message_fields, proto_msg)) {
|
|
|
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
|
|
|
return 1;
|
... | ... | @@ -258,13 +308,13 @@ We can then the encode the data using Nanopb's `pb_encode` function. The `pb_enc |
|
|
|
|
|
A callback function is created to encode the variable amount of bytes in the CAN message.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
bool callback_encode_can_bytes(pb_ostream_t *ostream, const pb_field_t *field, void *const *arg) {}
|
|
|
```
|
|
|
|
|
|
We deference the `arg` pointer which points to the array of CAN messages, get the count of CAN messages inside the array and encode all bytes at once using Nanopb's `pb_encode_string` [API](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb-encode-string).
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
const can_message_s *const can_messages = (const can_message_s *)(*arg);
|
|
|
const uint8_t *const can_byte = (const uint8_t *)can_messages->data;
|
|
|
const size_t can_message_byte_count = can_messages->dlc;
|
... | ... | @@ -283,7 +333,7 @@ Decoding the packet is similar to the first example, however we make use Callbac |
|
|
|
|
|
The `callback_t` structure members are assigned the pointer to callback function and CAN message data structure which will hold the decoded CAN message. The Nanopb's `pb_decode` API is then called.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
bool decode_packet(proto_can_message *proto_msg, serial_data_packet_s *packet, can_message_s *can_msg) {
|
|
|
....
|
|
|
proto_msg->data_byte.arg = can_msg;
|
... | ... | @@ -295,7 +345,7 @@ bool decode_packet(proto_can_message *proto_msg, serial_data_packet_s *packet, c |
|
|
```
|
|
|
The Callback to decode multiple CAN bytes reads all the bytes at once using `pb_read` [API.](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb-read) `pb_byte_t pb_bytes` is essentially a byte array. `istream->bytes_left` argument to `pb_read` API informs the Nanopb how many bytes are left to be read.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
static bool callback_decode_can_bytes(pb_istream_t *istream, const pb_field_t *field, void **arg)
|
|
|
...
|
|
|
// Read 8 bytes
|
... | ... | @@ -325,7 +375,7 @@ message proto_message{ |
|
|
```
|
|
|
After generating the `.c` and `.h` files we will now have have two `pb_callback_t` structures inside the messages.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
/* Struct definitions */
|
|
|
typedef struct _proto_can_message {
|
|
|
...
|
... | ... | @@ -347,14 +397,14 @@ So we need to have 2 callback functions: |
|
|
### Encoding
|
|
|
|
|
|
We start by passing `N` number of CAN messages to `encode_packet` function. The `can_msg_count` variable passes the number of CAN messages in the `can_msg` array.
|
|
|
```
|
|
|
```c
|
|
|
size_t encode_packet(proto_message *proto_pkt, serial_data_packet_s *packet, can_message_s *can_msg,
|
|
|
size_t can_msg_count)
|
|
|
```
|
|
|
|
|
|
We register the array of CAN messages to be encoded to the `arg` member of `pb_callback_t` structure along with the `callback_encode_can_messages` callback function. We then pass the entire `proto_pkt` to the Nanopb's `pb_encode` API for encoding.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
proto_pkt->can_msgs.arg = (void *)(&encode_can_messages_callback_parameter);
|
|
|
proto_pkt->can_msgs.funcs.encode = &callback_encode_can_messages;
|
|
|
|
... | ... | @@ -371,7 +421,7 @@ bool callback_encode_can_messages(pb_ostream_t *ostream, const pb_field_t *field |
|
|
```
|
|
|
We then deference the array of CAN messages passed into the `void *const *arg` argument and iteratively encode individual CAN messages. Since the `pb_callback_t can_msgs` is a member field of `proto_message` we use the Nanopb's `pb_encode_submessage()` API to encode individual CAN messages.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
bool callback_encode_can_messages(pb_ostream_t *ostream, const pb_field_t *field, void *const *arg) {
|
|
|
bool is_success = true;
|
|
|
|
... | ... | @@ -409,7 +459,7 @@ bool callback_encode_can_messages(pb_ostream_t *ostream, const pb_field_t *field |
|
|
|
|
|
Nanopb's `pb_encode_submessage()` function will internally call `callback_encode_can_bytes` callback function to encoded individual CAN data bytes
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
static bool callback_encode_can_bytes(pb_ostream_t *ostream, const pb_field_t *field, void *const *arg) {
|
|
|
bool is_success = true;
|
|
|
|
... | ... | @@ -437,12 +487,12 @@ Thus subsequently all the CAN messages are encoded. |
|
|
|
|
|
Now to decode the Array of CAN messages we reverse the encoding process. We pass an empty array of CAN message which will hold the decoded data from the `serial_data_packet_s *packet` data structure.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
bool decode_packet(proto_message *proto_msg, serial_data_packet_s *packet, can_message_s *decoded_can_messages)
|
|
|
```
|
|
|
To decode the array of messages we register the callback `decode_callback_can_messages` to the `pb_callback_t` structure `can_msgs` and also pass the empty CAN message array to store the decoded CAN data. We then call Nanopb's `pb_decode()` API.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
proto_msg->can_msgs.arg = (void *)decoded_can_messages;
|
|
|
proto_msg->can_msgs.funcs.decode = &decode_callback_can_messages;
|
|
|
|
... | ... | @@ -454,7 +504,7 @@ To decode the array of messages we register the callback `decode_callback_can_me |
|
|
```
|
|
|
The `decode_callback_can_messages` callback is invoked for EACH repeated message. It again uses Nanopb's `pb_decode()` function to decode CAN message for a single message. It internally call the `callback_decode_can_bytes` callback as illustrated in the second example to encoded the 8 bytes of CAN message's data field.
|
|
|
|
|
|
```
|
|
|
```c
|
|
|
static bool decode_callback_can_messages(pb_istream_t *istream, const pb_field_t *field, void **arg) {
|
|
|
bool is_success = true;
|
|
|
|
... | ... | |