... | ... | @@ -12,7 +12,7 @@ |
|
|
Protocol buffer can serialize data from variety of languages such as Java, Python, Objective-C C++, Dart, Go, Ruby, and C# along with running on any platform.
|
|
|
|
|
|
## Nanopb
|
|
|
[Nanopb](https://jpa.kapsi.fi/nanopb/) provides a C based library for encoding and decoding messages in Google's Protocol Buffers format with suitable for microcontrollers with less RAM and code space availability. Google's Protocol Buffer Tool can generate data structures for C++ and not for C, thus making it less suitable for Microcontrollers.
|
|
|
Google's Protocol Buffer Tool can generate data structures for C++ and not for C, thus making it less suitable for microcontrollers with limited RAM and code space availability. [Nanopb](https://jpa.kapsi.fi/nanopb/) provides a C based library for encoding and decoding messages in Google's Protocol Buffers format.
|
|
|
|
|
|
## Why do we need Protocol Buffers (or Nanopb)?
|
|
|
Consider the scenario where we need to transmit [CAN message](https://en.wikipedia.org/wiki/CAN_bus) frames from vehicle to the cloud for data logging and analysis. The data of a CAN frame can be held in the following C Data structure:
|
... | ... | @@ -43,7 +43,7 @@ Now we need to send this data message to the cloud. A popular choice is using [B |
|
|
*/
|
|
|
ssize_t send(int sockfd, const void *buf, size_t len, int flags);
|
|
|
```
|
|
|
Since the `send` API requires an array of bytes to be transferred, we need a way to convert our data in the CAN based C strucutre above to serialized data bytes. This is where Protocol Buffers (or Nanopb) comes into the play. Protocol Buffers will serialize the CAN data strucuture as an array of bytes in an efficient manner which can then be transfered to the cloud server using Socket `send` API.
|
|
|
Since the `send` API requires an array of bytes to be transferred, we need a way to convert our data to CAN based C structure above to serialized data bytes. This is where Protocol Buffers (or Nanopb) comes into play. Protocol Buffers will serialize the CAN data structure as an array of bytes in an efficient manner which can then be transferred to the cloud server using Socket `send` API.
|
|
|
|
|
|
# Getting started
|
|
|
|
... | ... | @@ -95,10 +95,11 @@ Download the example files from our [public repository](https://gitlab.com/sibro |
|
|
|
|
|
## Example - 1
|
|
|
|
|
|
In this example, a single CAN message with a single byte of data is serialized to protobuf format using Nanopb.
|
|
|
The source files for this example can be found [here](https://gitlab.com/sibros_public/sibros-open-source/nanopb-examples/-/tree/master/ex1)
|
|
|
|
|
|
### Proto Files
|
|
|
Let's say you want to transmit a CAN message in a serialized protobuf format. Then we need to define the CAN message and its contents inside a proto file as follows:
|
|
|
Proto files are used to structure the protocol buffer data using protocol buffer language. We need to define the CAN message and its contents inside a proto file as follows:
|
|
|
|
|
|
```c
|
|
|
// ex1/ex1.proto
|
... | ... | @@ -108,20 +109,20 @@ message proto_can_message { |
|
|
required int32 bus_id = 2; // The bus number which received the CAN message
|
|
|
|
|
|
required int64 timestamp_ms = 3;
|
|
|
required int32 data_byte = 4; // Usually 8 bytes but modified for simplified example
|
|
|
required int32 data_byte = 4; // only 1 byte of data (CAN frame data bytes are usually 8 but modified here for simplification)
|
|
|
}
|
|
|
```
|
|
|
In the `proto` file a message to be transmitted is defined using `message` keyword, followed by message name (`proto_can_message` in the exampe above). The proto message can have multiple data members such as message_id, bus_id, timestamp_ms, data_byte as above.
|
|
|
In the `proto` file a message to be transmitted is defined using the `message` keyword, followed by the message name (`proto_can_message` in the exampe above). The proto message can have multiple data members such as message_id, bus_id, timestamp_ms, data_byte as above.
|
|
|
|
|
|
If `required` keyword is used before tahe declaration of the message's member, the member has to be initialized. Alternatively `option` keyword can also be used which makes initializing the data member optional.
|
|
|
If `required` keyword is used before the declaration of the message's member, the member has to be initialized. Alternatively `option` keyword can also be used which makes initializing the data member optional.
|
|
|
|
|
|
The the number after equality operator is the tag of the message member which are used to match fields when serializing and deserializing the data. For Eg. the tag number for `timestamp_ms` is 1.
|
|
|
The number after equality operator is the tag of the message member which are used to match fields when serializing and deserializing the data. For Eg. the tag number for `timestamp_ms` is 1.
|
|
|
|
|
|
For more information about `proto` message, visit [Google's proto file documentation.](https://developers.google.com/protocol-buffers/docs/overview)
|
|
|
|
|
|
### Compiling proto files and generation of C header(.h) file
|
|
|
|
|
|
For Bazel users, we create an instance (or target) of `cc_nanopb_library` rule, which when executed will compile the proto file and generate the (.h) file.
|
|
|
For **Bazel users**, we create an instance (or target) of `cc_nanopb_library` rule, which when executed will compile the proto file and generate the (.h) file.
|
|
|
|
|
|
```python
|
|
|
# ex1/BUILD
|
... | ... | @@ -149,7 +150,7 @@ $ bazel build //shared/temp_work/nanopb-examples/ex1:ex1_c_proto |
|
|
bazel-bin/shared/temp_work/nanopb-examples/ex1/ex1.npb.h
|
|
|
```
|
|
|
|
|
|
Non-Bazel users, can generate the header (.h) file using the python script provided by Nanopb:
|
|
|
**Non-Bazel** users, can generate the header (.h) file using the python script provided by Nanopb:
|
|
|
|
|
|
```
|
|
|
$ cd nanopb
|
... | ... | @@ -159,76 +160,153 @@ $ python3 generator/nanopb_generator.py /path/to/nanopb-examples/ex1/ex1.proto |
|
|
The Nanopb generated header (.h) file will create an proto message equivalent C strucutre as follows:
|
|
|
```c
|
|
|
/* Struct definitions */
|
|
|
typedef struct _can_message {
|
|
|
int64_t timestamp_ms;
|
|
|
int32_t bus_id;
|
|
|
typedef struct _proto_can_message {
|
|
|
int32_t message_id;
|
|
|
int32_t bus_id;
|
|
|
int64_t timestamp_ms;
|
|
|
int32_t data_byte;
|
|
|
/* @@protoc_insertion_point(struct:can_message) */
|
|
|
} can_message;
|
|
|
/* @@protoc_insertion_point(struct:proto_can_message) */
|
|
|
} proto_can_message;
|
|
|
```
|
|
|
|
|
|
Refer to the Nanopb generated header [here]().
|
|
|
Refer to the complete Nanopb generated header [here]().
|
|
|
|
|
|
### Encoding data using Nanopb
|
|
|
|
|
|
### Encoding:
|
|
|
We start with creating our `.c` and `.h` files for defining encoding data operation by creating `encode_packet` function. We have to include the Nanopb headers and the generated header files.
|
|
|
|
|
|
```c
|
|
|
// ex1/ex1.c
|
|
|
|
|
|
/* Main Module Header */
|
|
|
#include "ex1.h"
|
|
|
|
|
|
/* Standard Includes */
|
|
|
#include <stdio.h>
|
|
|
|
|
|
/* Nanopb Includes */
|
|
|
#include "pb_decode.h"
|
|
|
#include "pb_encode.h"
|
|
|
|
|
|
/**
|
|
|
* @param proto_msg Generated proto_can_message
|
|
|
* @param packet Data structure to hold encoded data bytes
|
|
|
* @param can_msg CAN message data structure to encode
|
|
|
*
|
|
|
* @return number of bytes encoded, or 0 upon error.
|
|
|
*/
|
|
|
size_t encode_packet(proto_can_message* proto_msg, serial_data_packet_s* packet, can_message_s* can_msg);
|
|
|
```
|
|
|
**Note:** `serial_data_packet_s* packet` holds the memory buffer for Nanopb encoded data bytes:
|
|
|
```c
|
|
|
typedef struct encoded_packet {
|
|
|
uint8_t* buffer;
|
|
|
uint16_t max_buffer_size;
|
|
|
size_t bytes_written;
|
|
|
} serial_data_packet_s;
|
|
|
```
|
|
|
|
|
|
Now we can use this generated structure to populate our data to be serialized as follows:
|
|
|
Now we can use the generated structure to populate our data to be serialized as follows:
|
|
|
|
|
|
```c
|
|
|
size_t encode_packet(proto_can_message* proto_msg, serial_data_packet_s* packet, can_message_s* can_msg) {
|
|
|
|
|
|
/* Fill the proto packet */
|
|
|
proto_msg->timestamp_ms = can_msg->timestamp_ms;
|
|
|
proto_msg->bus_id = can_msg->bus_id;
|
|
|
proto_msg->message_id = can_msg->message_id;
|
|
|
proto_msg->data_byte = can_msg->data[0];
|
|
|
proto_msg->data_byte = can_msg->data;
|
|
|
|
|
|
}
|
|
|
```
|
|
|
|
|
|
We then create an output stream for writing the data to the buffer:
|
|
|
|
|
|
```c
|
|
|
size_t encode_packet(proto_can_message* proto_msg, serial_data_packet_s* packet, can_message_s* can_msg) {
|
|
|
/* Fill the proto packet */
|
|
|
...
|
|
|
|
|
|
pb_ostream_t stream = pb_ostream_from_buffer(encoded_packet->buffer, encoded_packet->encoded_packet_size);
|
|
|
|
|
|
}
|
|
|
```
|
|
|
`pb_ostream_t pb_ostream_from_buffer(pb_byte_t *buf, size_t bufsize)` constructs an output stream for writing into a memory buffer. It uses an internal callback that stores the pointer in stream state field. `pb_ostream_t stream` will hold the output stream buffer.
|
|
|
|
|
|
The Protobuf message is then encoded to a serialized format using Nanopb's `pb_encode()` API.
|
|
|
The Protobuf message is then encoded to a serialized format using Nanopb's `pb_encode()` API. The number of bytes encoded is obtained from `stream.bytes_written` data member which is then returned.
|
|
|
```c
|
|
|
if (!pb_encode(&stream, proto_can_message_fields, proto_msg)) {
|
|
|
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
|
|
|
return 1;
|
|
|
size_t encode_packet(proto_can_message* proto_msg, serial_data_packet_s* packet, can_message_s* can_msg) {
|
|
|
size_t number_bytes_encoded = 0;
|
|
|
|
|
|
/* Fill the proto packet */
|
|
|
...
|
|
|
|
|
|
pb_ostream_t stream = pb_ostream_from_buffer(packet->buffer, packet->max_buffer_size);
|
|
|
|
|
|
if (pb_encode(&stream, proto_can_message_fields, proto_msg)) {
|
|
|
printf("Encoding Success!!!\n");
|
|
|
|
|
|
number_bytes_encoded = stream.bytes_written;
|
|
|
packet->bytes_written = stream.bytes_written;
|
|
|
}
|
|
|
|
|
|
return number_bytes_encoded;
|
|
|
}
|
|
|
```
|
|
|
|
|
|
`pb_encode()` function encodes the contents of a structure as a protocol buffers message and writes it to output stream. Visit [here](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb_encode) for more reference.
|
|
|
|
|
|
`proto_can_message_fields` argument to the `pb_encode()` function is an auto-generated Struct field which will encoding specification for Nanopb. Visit [here](https://github.com/nanopb/nanopb/blob/master/pb.h#L315) for more reference. Also visit the auto-generated Nanopb [header file]() for reference.
|
|
|
|
|
|
At this point data bytes are encoded in a serialized format and stored in the `serial_data_packet_s` memory structure.
|
|
|
|
|
|
Refer the [source files](https://gitlab.com/sibros_public/sibros-open-source/nanopb-examples/-/tree/master/ex1) for more context.
|
|
|
|
|
|
### Decoding
|
|
|
|
|
|
To verify integrity of encoded data, we decode the encoded packet. First we create a stream that reads from the encoded buffer.
|
|
|
To verify the integrity of encoded data, we decode the encoded packet. First, we create a stream that reads from the encoded buffer.
|
|
|
|
|
|
```c
|
|
|
pb_istream_t stream = pb_istream_from_buffer(packet->buffer, packet->bytes_written);
|
|
|
```
|
|
|
The we decode the encoded message by using Nanopb's decoding API such that proto message ie `proto_can_message* proto_msg` is re-populated.
|
|
|
Visit [here](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb_istream_from_buffer) for more reference on `pb_istream_from_buffer`.
|
|
|
|
|
|
Then we decode the encoded message by using Nanopb's decoding API such that proto message ie, `proto_can_message* proto_msg` is re-populated.
|
|
|
```c
|
|
|
pb_decode(&stream, proto_can_message_fields, proto_msg)
|
|
|
```
|
|
|
Finally we re-populate the can message using the newly populated `proto_msg` ie. Copy decoded data to can message
|
|
|
Visit [here](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb_decode) for more reference on `pb_decode`.
|
|
|
|
|
|
Finally, we re-populate the CAN message using the newly populated `proto_msg` ie. Copy decoded data to CAN message.
|
|
|
```c
|
|
|
can_msg->message_id = proto_msg->message_id;
|
|
|
can_msg->bus_id = proto_msg->bus_id;
|
|
|
can_msg->timestamp_ms = proto_msg->timestamp_ms;
|
|
|
can_msg->data = proto_msg->data_byte;
|
|
|
```
|
|
|
Refer the [source files](https://gitlab.com/sibros_public/sibros-open-source/nanopb-examples/-/tree/master/ex1) for more context.
|
|
|
|
|
|
## Example - 2 : Using Callbacks for Nanopb
|
|
|
|
|
|
A CAN message usually has a 8 byte data field (if using CAN-FD the data field is 64 bytes). Thus we modify the can message in the `proto` file such that the CAN proto message can encode 8 byte of data at once.
|
|
|
The source files for this example can be found [here](https://gitlab.com/sibros_public/sibros-open-source/nanopb-examples/-/tree/master/ex2)
|
|
|
|
|
|
A CAN message usually has a 8 byte data field (if using CAN-FD the data field is 64 bytes). Thus we modify the CAN message in the `proto` file such that the CAN proto message can encode 8 bytes of data at once.
|
|
|
|
|
|
```c
|
|
|
//can.proto
|
|
|
message can_message {
|
|
|
required int64 timestamp_ms = 1;
|
|
|
required int32 bus_id = 2; // The bus number which received the CAN message
|
|
|
required int32 message_id = 3; // Standard 11-bit
|
|
|
bytes data_byte = 4; // Usually 8 bytes but modified for simplified example
|
|
|
// ex2/ex2.proto
|
|
|
message proto_can_message {
|
|
|
required int32 message_id = 1; // Standard 11-bit
|
|
|
required int32 bus_id = 2; // The bus number which received the CAN message
|
|
|
required int64 timestamp_ms = 3;
|
|
|
required bytes data_byte = 4; // Varaible number of bytes
|
|
|
}
|
|
|
```
|
|
|
|
|
|
After generating the (.h) file again the message is modified as follows:
|
|
|
**Note:** The `int32` data type for data_byte member has changed to `bytes` data type, which is used to allocate variable-length storage for message fields. Refer the [Scalar Value Types](https://developers.google.com/protocol-buffers/docs/overview#scalar) section for more context.
|
|
|
|
|
|
### Compiling proto files and generation of C header(.h) file
|
|
|
|
|
|
On compiling the `.proto` file again and generating the Nanopb header (`ex2.npb.h`) file the Proto message C structure `proto_can_message` is modified as follows:
|
|
|
|
|
|
```c
|
|
|
/* Struct definitions */
|
... | ... | @@ -241,16 +319,16 @@ typedef struct _proto_can_message { |
|
|
} proto_can_message;
|
|
|
```
|
|
|
|
|
|
Note the data type of `data_byte` is changed from to `int32_t` to `pb_callback_t` type. Nanopb callbacks are explained next.
|
|
|
|
|
|
### Callbacks
|
|
|
|
|
|
Callbacks are used when the members of a message have variable length and storage is not statically allocated for it.
|
|
|
For example if a message contains a string member such as `string name` instead of generating `char *name` Nanopb generates the variable `name` of `pb_callback_t` datatype. This allows the user to allocated the variable `name` with any number of chars using a custom call back function.
|
|
|
For example if a proto message (in a `.proto` file) contains a string member such as `string name`, instead of generating `char *name` Nanopb generates the variable `name` of `pb_callback_t` (ie. '`pb_callback_t name`') datatype. This allows the user to allocate variable `name` with any number of chars using a custom callback function.
|
|
|
|
|
|
Thus, members of a message are generated as `pb_callback_t` datatypes for variable length arrays/strings and repeated member messages(This is demonstrated below).
|
|
|
Thus, members of a Proto message are generated as `pb_callback_t` datatypes for variable-length arrays/strings and repeated member messages(This is demonstrated below).
|
|
|
|
|
|
For more information on callbacks in Nanopb visit [here](https://jpa.kapsi.fi/nanopb/docs/concepts.html#field-callbacks).
|
|
|
|
|
|
The callback structure is declared as follows:
|
|
|
The **pb_callback_t structure** is defined as follows:
|
|
|
```c
|
|
|
typedef struct _pb_callback_t pb_callback_t;
|
|
|
struct _pb_callback_t {
|
... | ... | @@ -263,41 +341,42 @@ struct _pb_callback_t { |
|
|
};
|
|
|
```
|
|
|
The `pb_callback_t` structure consists of two members:
|
|
|
* Function pointer: This points to callback function for processing variable length member of a message.
|
|
|
* `void` pointer `arg`: It is used to pass a pointer to the data structure which is processed by the callback function.
|
|
|
If the function pointer is `NULL`, the corresponding message field will be skipped.
|
|
|
* **Union of Function pointers**: This member holds a callback function for processing variable length member of a message.
|
|
|
* **`void *arg`**: It is used to pass a pointer to the data structure which is processed by the callback function. If the function pointer is `NULL`, the corresponding message field will be skipped.
|
|
|
|
|
|
For more information on callbacks in Nanopb visit [here](https://jpa.kapsi.fi/nanopb/docs/concepts.html#field-callbacks).
|
|
|
|
|
|
### Encoding a callback message
|
|
|
|
|
|
Let's consider we would like to encode a CAN message as follows:
|
|
|
Let's encode a single CAN message instantiated as follows:
|
|
|
|
|
|
```c
|
|
|
can_message_s can_msgs_to_encode = {.message_id = 111U, .timestamp_ms = 5U, .dlc = 8U, .bus_id = 2};
|
|
|
memset(can_msgs_to_encode.data, 0x0E, 8);
|
|
|
memset(can_msgs_to_encode.data, 0x0E, sizeof(can_msgs_to_encode.data));
|
|
|
```
|
|
|
Here, the data field of CAN message has 8 bytes, and each byte is set to the value `0x0E`. The CAN message is sent to the encode function as follows:
|
|
|
Here, the data field of CAN message has 8 bytes, and each byte is set to the value `0x0E`.
|
|
|
|
|
|
```c
|
|
|
size_t encode_packet(proto_can_message *proto_msg, serial_data_packet_s *packet, can_message_s *can_msg)
|
|
|
```
|
|
|
Inside the `encode_packet` function `proto_can_message *proto_msg` (ie. The generated Proto C Structure) is populated as before from the `can_message_s *can_msg`. Since `data_bytes` member is of `pb_callback_t` it is populated as follows:
|
|
|
|
|
|
Since the Nanopb's `pb_encode` function requires the data to be encoded in `proto` message format, the `can_message_s *can_msg` to encode is populated into the `proto_can_message *proto_msg`. The Callback structure of the `data_bytes` member is populated as follows:
|
|
|
* `proto_msg->data_byte.arg = can_msg`
|
|
|
This is required since the data field of CAN message has variable amount of bytes which is handled by the callback function by passing it as an argument.
|
|
|
* `proto_msg->data_byte.funcs.encode = &callback_encode_can_bytes`
|
|
|
Will hold the function pointer for the callback funtion.
|
|
|
* `proto_msg->data_byte.arg = can_msg`: The `can_msg` can now be passed as an argument to callback function and it's variable amount of data bytes (8 in this case) can be encoded.
|
|
|
* `proto_msg->data_byte.funcs.encode = &callback_encode_can_bytes` : Will hold the function pointer for the callback funtion. The callback function definition is described below.
|
|
|
|
|
|
```c
|
|
|
/* Fill the proto packet */
|
|
|
size_t encode_packet(proto_can_message *proto_msg, serial_data_packet_s *packet, can_message_s *can_msg) {
|
|
|
|
|
|
/* Fill the proto packet */
|
|
|
proto_msg->timestamp_ms = can_msg->timestamp_ms;
|
|
|
proto_msg->bus_id = can_msg->bus_id;
|
|
|
proto_msg->message_id = can_msg->message_id;
|
|
|
|
|
|
// Callback structure
|
|
|
// `pb_callback_t` member
|
|
|
proto_msg->data_byte.arg = can_msg;
|
|
|
proto_msg->data_byte.funcs.encode = &callback_encode_can_bytes;
|
|
|
|
|
|
/* Create a stream that will write to our buffer. */
|
|
|
pb_ostream_t stream = pb_ostream_from_buffer(packet->buffer, packet->max_buffer_size);
|
|
|
}
|
|
|
```
|
|
|
We can then the encode the data using Nanopb's `pb_encode` function. The `pb_encode` will internally call the callback registered above to encode all the data bytes inside the CAN message.
|
|
|
We then call Nanopb's `pb_encode()` function. The `pb_encode()` will internally call the callback registered above to encode all the data bytes inside the CAN message.
|
|
|
|
|
|
```c
|
|
|
if (!pb_encode(&stream, proto_can_message_fields, proto_msg)) {
|
... | ... | @@ -306,36 +385,47 @@ We can then the encode the data using Nanopb's `pb_encode` function. The `pb_enc |
|
|
}
|
|
|
```
|
|
|
|
|
|
A callback function is created to encode the variable amount of bytes in the CAN message.
|
|
|
The strucutre of the callback function is as follows:
|
|
|
|
|
|
```c
|
|
|
bool callback_encode_can_bytes(pb_ostream_t *ostream, const pb_field_t *field, void *const *arg) {}
|
|
|
```
|
|
|
|
|
|
We deference the `arg` pointer which points to the array of CAN messages, get the count of CAN messages inside the array and encode all bytes at once using Nanopb's `pb_encode_string` [API](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb-encode-string).
|
|
|
static bool callback_encode_can_bytes(pb_ostream_t *ostream, const pb_field_t *field, void *const *arg) {
|
|
|
bool is_success = true;
|
|
|
|
|
|
```c
|
|
|
const can_message_s *const can_messages = (const can_message_s *)(*arg);
|
|
|
const uint8_t *const can_byte = (const uint8_t *)can_messages->data;
|
|
|
const size_t can_message_byte_count = can_messages->dlc;
|
|
|
if ((NULL == arg) || (NULL == *arg)) {
|
|
|
is_success = false;
|
|
|
} else {
|
|
|
const can_message_s *const can_message = (const can_message_s *)(*arg);
|
|
|
const uint8_t *const can_data_bytes = (const uint8_t *)can_message->data;
|
|
|
const size_t can_message_byte_count = can_message->dlc;
|
|
|
|
|
|
if (!pb_encode_tag_for_field(ostream, field)) {
|
|
|
is_success = false;
|
|
|
}
|
|
|
if (!pb_encode_string(ostream, can_byte, can_message_byte_count)) {
|
|
|
if (!pb_encode_string(ostream, can_data_bytes, can_message_byte_count)) {
|
|
|
is_success = false;
|
|
|
}
|
|
|
}
|
|
|
|
|
|
return is_success;
|
|
|
}
|
|
|
```
|
|
|
|
|
|
We deference the `arg` pointer which points to the array of CAN messages, get the count of CAN messages inside the array and encode all bytes at once using Nanopb's [`pb_encode_string` API](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb-encode-string).
|
|
|
`pb_encode_tag_for_field` starts a field in the Protocol Buffers binary format. More information [here](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb_encode_tag_for_field).
|
|
|
|
|
|
### Decoding
|
|
|
|
|
|
Decoding the packet is similar to the first example, however we make use Callback structure to decoded multiple CAN bytes.
|
|
|
Decoding the packet is similar to the first example however, we make use of callback structure to decoded multiple CAN bytes.
|
|
|
|
|
|
The `callback_t` structure members are assigned the pointer to callback function and CAN message data structure which will hold the decoded CAN message. The Nanopb's `pb_decode` API is then called.
|
|
|
|
|
|
```c
|
|
|
bool decode_packet(proto_can_message *proto_msg, serial_data_packet_s *packet, can_message_s *can_msg) {
|
|
|
....
|
|
|
|
|
|
/* Create a stream that reads from the buffer. */
|
|
|
pb_istream_t stream = pb_istream_from_buffer(packet->buffer, packet->bytes_written);
|
|
|
|
|
|
proto_msg->data_byte.arg = can_msg;
|
|
|
proto_msg->data_byte.funcs.decode = &callback_decode_can_bytes;
|
|
|
|
... | ... | @@ -343,56 +433,136 @@ bool decode_packet(proto_can_message *proto_msg, serial_data_packet_s *packet, c |
|
|
....
|
|
|
}
|
|
|
```
|
|
|
The Callback to decode multiple CAN bytes reads all the bytes at once using `pb_read` [API.](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb-read) `pb_byte_t pb_bytes` is essentially a byte array. `istream->bytes_left` argument to `pb_read` API informs the Nanopb how many bytes are left to be read.
|
|
|
The Callback to decode multiple CAN bytes reads all the bytes at once using [`pb_read` API.](https://jpa.kapsi.fi/nanopb/docs/reference.html#pb-read) Here, `pb_byte_t pb_bytes` is essentially a byte array. `istream->bytes_left` argument to `pb_read` API informs the Nanopb how many bytes are left to be read.
|
|
|
|
|
|
```c
|
|
|
static bool callback_decode_can_bytes(pb_istream_t *istream, const pb_field_t *field, void **arg)
|
|
|
...
|
|
|
// Read 8 bytes
|
|
|
pb_byte_t pb_bytes[8 + 1] = {0};
|
|
|
static bool callback_decode_can_bytes(pb_istream_t *istream, const pb_field_t *field, void **arg) {
|
|
|
bool is_success = true;
|
|
|
|
|
|
pb_read(istream, pb_bytes, istream->bytes_left)
|
|
|
|
|
|
// Copy decoded stream to CAN message data bytes
|
|
|
memcpy(can_messages->data, pb_bytes, sizeof(can_messages->data));
|
|
|
can_message_s *can_message = (can_message_s *)(*arg);
|
|
|
|
|
|
// Read 8 bytes
|
|
|
pb_byte_t pb_bytes[8 + 1] = {0};
|
|
|
|
|
|
// With `pb_read` NanoPB API, we can directly read the bytes for string or byte types
|
|
|
if (!pb_read(istream, pb_bytes, istream->bytes_left)) {
|
|
|
is_success = false;
|
|
|
printf("pb_read() failed while reading CAN data bytes");
|
|
|
}
|
|
|
// Copy decoded stream to CAN message data bytes
|
|
|
memcpy(can_message->data, pb_bytes, sizeof(can_message->data));
|
|
|
|
|
|
return is_success;
|
|
|
}
|
|
|
```
|
|
|
|
|
|
## Example - 3 : Nested Callback Structures
|
|
|
|
|
|
Consider we have the following `.proto` file:
|
|
|
```
|
|
|
The source files for this example can be found [here](https://gitlab.com/sibros_public/sibros-open-source/nanopb-examples/-/tree/master/ex3)
|
|
|
|
|
|
Consider a scenario when we need to send multiple CAN frames and some other information (for eg. software version information) to the cloud in the form of encoded data bytes.
|
|
|
|
|
|
For this we need to define the following `.proto` file:
|
|
|
|
|
|
```c
|
|
|
// ex3/ex3.proto
|
|
|
|
|
|
syntax = "proto2";
|
|
|
|
|
|
message proto_can_message {
|
|
|
....
|
|
|
required bytes data_byte = 4; // Usually 8 bytes but modified for simplified example
|
|
|
required int32 message_id = 1; // Standard 11-bit
|
|
|
required int32 bus_id = 2; // The bus number which received the CAN message
|
|
|
required int64 timestamp_ms = 3;
|
|
|
required bytes data_byte = 4; // Usually 8 data bytes
|
|
|
}
|
|
|
|
|
|
....
|
|
|
message version_info{
|
|
|
required int32 major_version = 1;
|
|
|
required int32 minor_version = 2;
|
|
|
}
|
|
|
|
|
|
message proto_message{
|
|
|
....
|
|
|
required string sw_version = 1;
|
|
|
required version_info release_version = 2;
|
|
|
repeated proto_can_message can_msgs = 3;
|
|
|
}
|
|
|
```
|
|
|
After generating the `.c` and `.h` files we will now have have two `pb_callback_t` structures inside the messages.
|
|
|
* The `.proto` file defines `proto_can_message` message as before.
|
|
|
* `version_info` message is used to encode the release version information in integer data format.
|
|
|
* `proto_message` wraps `version_info` message, multiple `can_msgs` and `sw_version` (ie. software version) in string format.
|
|
|
* `proto_can_message can_msgs` member of `proto_message` has been declared as `repeated`. This allows us to encoded muliple CAN frames within a single proto message.
|
|
|
|
|
|
### `.options` file
|
|
|
|
|
|
Using generator options, we can set maximum sizes for fields in order to allocate them statically. The preferred way to do this is to create an `.options` file with the same name as your `.proto` file:
|
|
|
|
|
|
```c
|
|
|
proto_message.sw_version type:FT_STATIC
|
|
|
proto_message.sw_version max_size:6 //Also includes space for `NULL` character
|
|
|
|
|
|
// Prevents mangling of names for global enums
|
|
|
* long_names:false
|
|
|
```
|
|
|
|
|
|
In the `.options` file above we fix the size of `proto_message.sw_version` message member in the `.proto` file to 6 characters.
|
|
|
|
|
|
For more information on Proto `.options` file in Nanopb visit [here](https://jpa.kapsi.fi/nanopb/docs/reference.html#proto-file-options).
|
|
|
|
|
|
### Compiling proto files and generation of C header(.h) file
|
|
|
|
|
|
For **Bazel users**, we modify the BUILD file from the example-1 by adding `ex3.options` to the `data` attribute of `proto_library` rule.
|
|
|
|
|
|
```python
|
|
|
# ex3/BUILD
|
|
|
|
|
|
proto_library(
|
|
|
name = "ex3_proto",
|
|
|
srcs = ["ex3.proto"],
|
|
|
data = ["ex3.options"],
|
|
|
)
|
|
|
|
|
|
cc_nanopb_library(
|
|
|
name = "ex3_c_proto",
|
|
|
deps = [
|
|
|
":ex3_proto",
|
|
|
],
|
|
|
)
|
|
|
```
|
|
|
|
|
|
Building the Nanopb target we generate the (ex3.npb.h) file.
|
|
|
|
|
|
```
|
|
|
$ bazel build //shared/temp_work/nanopb-examples/ex3:ex3_c_proto
|
|
|
|
|
|
bazel-bin/shared/temp_work/nanopb-examples/ex3/ex3.npb.h
|
|
|
```
|
|
|
|
|
|
```c
|
|
|
/* Struct definitions */
|
|
|
typedef struct _proto_can_message {
|
|
|
...
|
|
|
int32_t message_id;
|
|
|
int32_t bus_id;
|
|
|
int64_t timestamp_ms;
|
|
|
pb_callback_t data_byte;
|
|
|
/* @@protoc_insertion_point(struct:proto_can_message) */
|
|
|
} proto_can_message;
|
|
|
|
|
|
....
|
|
|
typedef struct _version_info {
|
|
|
int32_t major_version;
|
|
|
int32_t minor_version;
|
|
|
/* @@protoc_insertion_point(struct:version_info) */
|
|
|
} version_info;
|
|
|
|
|
|
typedef struct _proto_message {
|
|
|
....
|
|
|
char sw_version[6];
|
|
|
version_info release_version;
|
|
|
pb_callback_t can_msgs;
|
|
|
/* @@protoc_insertion_point(struct:proto_message) */
|
|
|
} proto_message;
|
|
|
```
|
|
|
So we need to have 2 callback functions:
|
|
|
* First callback function `callback_encode_can_messages` will encode multiple (variable amount of) CAN messages
|
|
|
* Second callback function `callback_encode_can_bytes` will encode multiple data bytes of the CAN message.
|
|
|
* `sw_version` member is also defined as a char array of 6 bytes as defined in `.option` file.
|
|
|
|
|
|
### Encoding
|
|
|
|
... | ... | @@ -457,7 +627,7 @@ bool callback_encode_can_messages(pb_ostream_t *ostream, const pb_field_t *field |
|
|
}
|
|
|
```
|
|
|
|
|
|
Nanopb's `pb_encode_submessage()` function will internally call `callback_encode_can_bytes` callback function to encoded individual CAN data bytes
|
|
|
Nanopb's `pb_encode_submessage()` function will internally call `callback_encode_can_bytes` callback function to encoded individual CAN data bytes.
|
|
|
|
|
|
```c
|
|
|
static bool callback_encode_can_bytes(pb_ostream_t *ostream, const pb_field_t *field, void *const *arg) {
|
... | ... | @@ -502,7 +672,7 @@ To decode the array of messages we register the callback `decode_callback_can_me |
|
|
return 0;
|
|
|
}
|
|
|
```
|
|
|
The `decode_callback_can_messages` callback is invoked for EACH repeated message. It again uses Nanopb's `pb_decode()` function to decode CAN message for a single message. It internally call the `callback_decode_can_bytes` callback as illustrated in the second example to encoded the 8 bytes of CAN message's data field.
|
|
|
The `decode_callback_can_messages` callback is invoked for EACH repeated message. It again uses Nanopb's `pb_decode()` function to decode CAN message for a single message.
|
|
|
|
|
|
```c
|
|
|
static bool decode_callback_can_messages(pb_istream_t *istream, const pb_field_t *field, void **arg) {
|
... | ... | @@ -537,6 +707,34 @@ static bool decode_callback_can_messages(pb_istream_t *istream, const pb_field_t |
|
|
}
|
|
|
```
|
|
|
|
|
|
`decode_callback_can_messages()` function internally call the `callback_decode_can_bytes` callback as illustrated in the second example to encoded the 8 bytes of CAN message's data field.
|
|
|
|
|
|
```c
|
|
|
/**
|
|
|
* With NanoPB API, we can directly read the bytes for string or byte types
|
|
|
*/
|
|
|
static bool callback_decode_can_bytes(pb_istream_t *istream, const pb_field_t *field, void **arg) {
|
|
|
bool is_success = true;
|
|
|
|
|
|
if ((NULL == arg) || (NULL == *arg)) {
|
|
|
is_success = false;
|
|
|
} else {
|
|
|
can_message_s *can_messages = (can_message_s *)(*arg);
|
|
|
|
|
|
// Read 8 bytes
|
|
|
pb_byte_t pb_bytes[8 + 1] = {0};
|
|
|
|
|
|
if (!pb_read(istream, pb_bytes, istream->bytes_left)) {
|
|
|
is_success = false;
|
|
|
printf("pb_read() failed while reading CAN data bytes");
|
|
|
}
|
|
|
// Copy decoded stream to CAN message data bytes
|
|
|
memcpy(can_messages->data, pb_bytes, sizeof(can_messages->data));
|
|
|
}
|
|
|
|
|
|
return is_success;
|
|
|
}
|
|
|
```
|
|
|
|
|
|
# External Resources
|
|
|
* [Protocol Buffer Introduction](https://developers.google.com/protocol-buffers/)
|
... | ... | |