Protobuf: feature of adding missing fields with default values
Description including problem, use cases, benefits, and/or goals
New Feature Description
Make Protobuf fields that are not serialized on the wire (missing in capture files) to be displayed with default values. In 'proto2', default values can be explicitly declared. In 'proto3', if a field is set to its default, the value will not be serialized on the wire.
The default value will be displayed according to following situations:
- Explicitly-declared default values in 'proto2', for example:
optional int32 result_per_page = 3 [default = 10]; // default value is 10- For bools, the default value is false.
- For enums, the default value is the first defined enum value, which must be 0 in 'proto3' (but allowed to be other in 'proto2').
- For numeric types, the default value is zero. There are no default values for fields 'repeated' or 'bytes' and 'string' without default value declared. If the missing field is 'required' in a 'proto2' file, an expert warning item will be added to the tree.
Which fields will be displayed is controlled by the new 'add_default_value' option (preference):
- [none] "None" -- do not display any missing fields.
- [decl] "Only Explicitly-Declared (proto2)" -- only missing fields of situation (1) will be displayed.
- [enbl] "Explicitly-Declared, ENUM and BOOL" -- missing fields of situantions (1, 2 and 3) will be displayed.
- [all] "All" -- missing fields of all situations (1, 2, 3, and 4) will be displayed.
Example
Put the file test_default_value.proto in one of the 'Protobuf search paths' in Protobuf protocol preferences, and make this path 'load all files' option on. The content of test_default_value.proto is:
// Test default values of Protobuf fields
syntax = "proto2";
package wireshark.protobuf.test;
message TestDefaultValueMessage {
enum EnumFoo {
ENUM_FOO_V_FIRST = 1;
ENUM_FOO_V_SECOND = 0x2;
ENUM_FOO_V_THIRD = 3;
ENUM_FOO_V_FOURTH = - 4;
}
// The format of field name is:
// <type> "With" ( "Value" | "DefaultValue" | "NoValue" ) [ "_" <correct_value_in_wireshark> ]
// The "DefaultValue" fields should be wrapped with generated mark ("[" and "]") of Wireshark tree item.
// The "NoValue" fields should not appear in Wireshark.
// The default value is overridden to 8 at running time.
required int32 int32WithValue_8 = 1 [ default = 2 ];
// The default value is overridden to ENUM_FOO_V_THIRD at running time.
optional EnumFoo enumFooWithValue_Third = 2 [ default = ENUM_FOO_V_SECOND ];
// default values of bool
optional bool boolWithDefaultValue_False = 11;
optional bool boolWithDefaultValue_True = 12 [ default = true ];
// default values of enum
optional EnumFoo enumFooWithDefaultValue_First = 21;
optional EnumFoo enumFooWithDefaultValue_Second = 22 [ default = ENUM_FOO_V_SECOND ];
optional EnumFoo enumFooWithDefaultValue_Fouth = 23 [ default = ENUM_FOO_V_FOURTH ];
// default values of integer number
optional int32 int32WithDefaultValue_0 = 31;
optional int64 int64WithDefaultValue_Negative1152921504606846976 = 32 [ default = - 1152921504606846976 ];
optional uint32 uint32WithDefaultValue_11 = 33 [ default = 11 ];
optional uint64 uint64WithDefaultValue_1152921504606846976 = 34 [ default = 1152921504606846976 ]; // equals to 2^60
optional sint32 sint32WithDefaultValue_Negative12 = 35 [ default = -12 ];
optional sint64 sint64WithDefaultValue_0 = 36; // default value is zero
optional fixed64 fixed64WithDefaultValue_1152921504606846976 = 37 [ default = 1152921504606846976 ];
optional sfixed32 sfixed32WithDefaultValue_Negative31 = 38 [ default = -0X1f ]; // -21
// default values of float and double
optional float floatWithDefaultValue_0point23 = 41 [ default = 0.23 ];
optional double doubleWithDefaultValue_Negative0point12345678 = 42 [ default = -0.12345678 ];
// default values of string and bytes
optional string stringWithNoValue = 51; // default value must not appear because not declared
optional string stringWithDefaultValue_SymbolPi = 52 [ default = "The symbol \'\xF0\x9D\x9B\x91\' is mathematical bold small Pi."];
optional bytes bytesWithNoValue = 53; // default value must not appear because not declared
// '\'nnn is octal value of a byte, '\x'nn is hex value of a byte
optional bytes bytesWithDefaultValue_1F2F890D0A00004B = 54 [ default = "\x1F\x2F\211\r\n\000\x0\x4B" ];
// others
repeated int32 repeatedFieldWithNoValue = 81; // should not appear
required int32 missingRequiredField = 82; // for testing mising required field
}You can open this capture protobuf_test_default_value.pcapng. Make sure to add UDP port 8128 and wireshark.protobuf.test.TestDefaultValueMessage to your 'Protobuf UDP Message Types' table in preferences.
The default detail of this package is ('add_default_value' option value is [none] "None"):
...
User Datagram Protocol, Src Port: 52545, Dst Port: 8128
Protocol Buffers (Message: wireshark.protobuf.test.TestDefaultValueMessage)
int32WithValue_8: 8
enumFooWithValue_Third: ENUM_FOO_V_THIRD (3)That means only two fields are actually transmitted on the wire.
If you change the 'add_default_value' option to [decl] "Only Explicitly-Declared (proto2)", the detail will be:
...
User Datagram Protocol, Src Port: 52545, Dst Port: 8128
Protocol Buffers (Message: wireshark.protobuf.test.TestDefaultValueMessage)
int32WithValue_8: 8
enumFooWithValue_Third: ENUM_FOO_V_THIRD (3)
[boolWithDefaultValue_True: True]
[enumFooWithDefaultValue_Second: ENUM_FOO_V_SECOND (2)]
[enumFooWithDefaultValue_Fouth: ENUM_FOO_V_FOURTH (-4)]
[int64WithDefaultValue_Negative1152921504606846976: -1152921504606846976]
[uint32WithDefaultValue_11: 11]
[uint64WithDefaultValue_1152921504606846976: 1152921504606846976]
[sint32WithDefaultValue_Negative12: -12]
[fixed64WithDefaultValue_1152921504606846976: 1152921504606846976]
[sfixed32WithDefaultValue_Negative31: -31]
[floatWithDefaultValue_0point23: 0.23]
[doubleWithDefaultValue_Negative0point12345678: -0.12345678]
[stringWithDefaultValue_SymbolPi: The symbol '𝛑' is mathematical bold small Pi.]
[bytesWithDefaultValue_1F2F890D0A00004B: 1f2f890d0a00004b]
[Expert Info (Warning/Protocol): missing required field 'missingRequiredField']The added missing fields are wrapped in '[' and ']'. The missing 'required' field named 'missingRequiredField' will also be detected by Wireshark, and a expert warning item will be added to the tree.
If you change the 'add_default_value' option to [enbl] "Explicitly-Declared, ENUM and BOOL", the detail will be:
...
User Datagram Protocol, Src Port: 52545, Dst Port: 8128
Protocol Buffers (Message: wireshark.protobuf.test.TestDefaultValueMessage)
int32WithValue_8: 8
enumFooWithValue_Third: ENUM_FOO_V_THIRD (3)
[boolWithDefaultValue_False: False] -- the default value of bools is false
[boolWithDefaultValue_True: True]
[enumFooWithDefaultValue_First: ENUM_FOO_V_FIRST (1)] -- the default value is first enum value
[enumFooWithDefaultValue_Second: ENUM_FOO_V_SECOND (2)]
[enumFooWithDefaultValue_Fouth: ENUM_FOO_V_FOURTH (-4)]
[int64WithDefaultValue_Negative1152921504606846976: -1152921504606846976]
[uint32WithDefaultValue_11: 11]
[uint64WithDefaultValue_1152921504606846976: 1152921504606846976]
[sint32WithDefaultValue_Negative12: -12]
[fixed64WithDefaultValue_1152921504606846976: 1152921504606846976]
[sfixed32WithDefaultValue_Negative31: -31]
[floatWithDefaultValue_0point23: 0.23]
[doubleWithDefaultValue_Negative0point12345678: -0.12345678]
[stringWithDefaultValue_SymbolPi: The symbol '𝛑' is mathematical bold small Pi.]
[bytesWithDefaultValue_1F2F890D0A00004B: 1f2f890d0a00004b]
[Expert Info (Warning/Protocol): missing required field 'missingRequiredField']If you change the 'add_default_value' option to [all] "All", the detail will be:
...
User Datagram Protocol, Src Port: 52545, Dst Port: 8128
Protocol Buffers (Message: wireshark.protobuf.test.TestDefaultValueMessage)
int32WithValue_8: 8
enumFooWithValue_Third: ENUM_FOO_V_THIRD (3)
[boolWithDefaultValue_False: False]
[boolWithDefaultValue_True: True]
[enumFooWithDefaultValue_First: ENUM_FOO_V_FIRST (1)]
[enumFooWithDefaultValue_Second: ENUM_FOO_V_SECOND (2)]
[enumFooWithDefaultValue_Fouth: ENUM_FOO_V_FOURTH (-4)]
[int32WithDefaultValue_0: 0] -- the default value of numeric type is 0
[int64WithDefaultValue_Negative1152921504606846976: -1152921504606846976]
[uint32WithDefaultValue_11: 11]
[uint64WithDefaultValue_1152921504606846976: 1152921504606846976]
[sint32WithDefaultValue_Negative12: -12]
[sint64WithDefaultValue_0: 0]
[fixed64WithDefaultValue_1152921504606846976: 1152921504606846976]
[sfixed32WithDefaultValue_Negative31: -31]
[floatWithDefaultValue_0point23: 0.23]
[doubleWithDefaultValue_Negative0point12345678: -0.12345678]
[stringWithDefaultValue_SymbolPi: The symbol '𝛑' is mathematical bold small Pi.]
[bytesWithDefaultValue_1F2F890D0A00004B: 1f2f890d0a00004b]
[Expert Info (Warning/Protocol): missing required field 'missingRequiredField']Links / references / protocol specifications
Sample capture
- test_default_value.proto - the .proto file related to the sample capture
- protobuf_test_default_value.pcapng - sample capture of testing adding missing fileds with default values