Skip to content

responseClass loses proto Map values

We are calling new responseClass(update) in execStreamMethod (server/ta2/client.ts). When the proto response contains a Map, the Map (key, value)s are lost and I cannot find a way to retrieve them.

Here is a simple reproduction:

// in execStreamMethod
if (update['exposed_outputs']) {
  console.log('before', update['exposed_outputs']);
}
const convertedUpdate = new responseClass(update); // this is where we convert to responseClass
if (convertedUpdate['exposed_outputs']) {
  console.log('after', convertedUpdate['exposed_outputs']);
}

Here is the output:

[Node] before { 'outputs.0':
[Node]    { value: 'csv_uri',
[Node]      error: null,
[Node]      raw: null,
[Node]      dataset_uri: '',
[Node]      csv_uri: 'file:///out/predict1.csv',
[Node]      pickle_uri: '',
[Node]      pickle_blob: <Buffer >,
[Node]      plasma_id: <Buffer > } }
[Node] after Map {
[Node]   field:
[Node]    T {
[Node]      builder:
[Node]       Builder {
[Node]         ns: [T],
[Node]         ptr: [T],
[Node]         resolved: false,
[Node]         result: null,
[Node]         files: [Object],
[Node]         importRoot: null,
[Node]         options: [Object] },
[Node]      parent:
[Node]       T {
[Node]         builder: [Builder],
[Node]         parent: [T],
[Node]         name: 'GetProduceSolutionResultsResponse',
[Node]         className: 'Message',
[Node]         children: [Array],
[Node]         options: {},
[Node]         syntax: 'proto3',
[Node]         extensions: undefined,
[Node]         clazz: [Function],
[Node]         isGroup: false,
[Node]         _fields: [Array],
[Node]         _fieldsById: [Object],
[Node]         _fieldsByName: [Object],
[Node]         _oneofsByName: {} },
[Node]      name: 'exposed_outputs',
[Node]      className: 'Message.Field',
[Node]      required: false,
[Node]      repeated: false,
[Node]      map: true,
[Node]      keyType: { name: 'string', wireType: 2, defaultValue: '' },
[Node]      type: { name: 'message', wireType: 2, defaultValue: null },
[Node]      resolvedType:
[Node]       T {
[Node]         builder: [Builder],
[Node]         parent: [T],
[Node]         name: 'Value',
[Node]         className: 'Message',
[Node]         children: [Array],
[Node]         options: {},
[Node]         syntax: 'proto3',
[Node]         extensions: undefined,
[Node]         clazz: [Function],
[Node]         isGroup: false,
[Node]         _fields: [Array],
[Node]         _fieldsById: [Object],
[Node]         _fieldsByName: [Object],
[Node]         _oneofsByName: [Object] },
[Node]      id: 3,
[Node]      options: {},
[Node]      defaultValue: null,
[Node]      oneof: null,
[Node]      syntax: 'proto3',
[Node]      originalName: 'exposed_outputs',
[Node]      element:
[Node]       Element {
[Node]         type: [Object],
[Node]         resolvedType: [T],
[Node]         isMapKey: false,
[Node]         syntax: 'proto3',
[Node]         name: 'exposed_outputs' },
[Node]      keyElement:
[Node]       Element {
[Node]         type: [Object],
[Node]         resolvedType: undefined,
[Node]         isMapKey: true,
[Node]         syntax: 'proto3',
[Node]         name: 'exposed_outputs' } },
[Node]   keyElem:
[Node]    Element {
[Node]      type: { name: 'string', wireType: 2, defaultValue: '' },
[Node]      resolvedType: null,
[Node]      isMapKey: true,
[Node]      syntax: 'proto3',
[Node]      name: undefined },
[Node]   valueElem:
[Node]    Element {
[Node]      type: { name: 'message', wireType: 2, defaultValue: null },
[Node]      resolvedType:
[Node]       T {
[Node]         builder: [Builder],
[Node]         parent: [T],
[Node]         name: 'Value',
[Node]         className: 'Message',
[Node]         children: [Array],
[Node]         options: {},
[Node]         syntax: 'proto3',
[Node]         extensions: undefined,
[Node]         clazz: [Function],
[Node]         isGroup: false,
[Node]         _fields: [Array],
[Node]         _fieldsById: [Object],
[Node]         _fieldsByName: [Object],
[Node]         _oneofsByName: [Object] },
[Node]      isMapKey: false,
[Node]      syntax: 'proto3',
[Node]      name: undefined },
[Node]   map: { 'outputs.0': { key: 'outputs.0',
[Node]      value:
[Node]       { value: 'plasma_id',
[Node]         error: null,
[Node]         raw: null,
[Node]         dataset_uri: null,
[Node]         csv_uri: null,
[Node]         pickle_uri: null,
[Node]         pickle_blob: null,
[Node]         plasma_id: [ByteBuffer] } } } } }

It can be seen that after the conversion the values in exposed_outputs['outputs.0'] are lost. csv_uri should contain a csv path but it is now null. Am I missing something?

My get-around is:

const originalUpdate = cloneDeep(update);
update = new responseClass(update);

// This is a hack that prevents grpc Map being converted to a proto Map class instance that loses its orignal (key, values).
for (const key in update) {
  const value = update[key];
  if (value.field && value.keyElem && value.valueElem && value.map) {
    update[key] = originalUpdate[key];
  }
}

This hack replaces the proto Map instance by the original object.

Edited by Bowen Yu