Encoding/decoding of attributes in binary format
sriggi avatar
sriggi
Dear all,

I would like to ask help/suggestions about how to proper encoding/decoding attributes in binary format. I have some structured data defined in particular with Google Protobuf.
I'm assuming a DevStringArray attribute (attr_raw_data_read) in which I store:
[0]: the data buffer
[1]: the buffer size (stringified)

Apparently things works on the server side, while I cannot parse data on the client side.
Here's what I'm doing on the server/client.

== SERVER ==
attr_raw_data_read[0]= CORBA::string_alloc(buffer_size);
attr_raw_data_read[0]= buffer_data;
attr_raw_data_read[1]= Tango::string_dup(buffer_size_string);

//Check encoding
DataModelPB::RawData* parsed_data= new DataModelPB::RawData;
bool isValidParsing= parsed_data->ParseFromArray(
   attr_raw_data_read[0],
   buffer_size
);
//OK check passed!
//Send event to clients
push_change_event(
   "raw_data", attr_raw_data_read, 
   tv,Tango::ATTR_VALID,
   2,0,false
);

== CLIENT (event callback) ==


void EventCallBack::push_event(Tango::EventData* event){
  Tango::DevVarStringArray* attr= 0;
  try { 
    DeviceAttribute device_attr= *(event->attr_value);
    device_attr >> attr;
    Tango::DevString buffer_size_string= (*attr)[1];
    int buffer_size = atoi(buffer_size_string);
    Tango::DevString buffer_data= (*attr_list)[0];
    
    //Parse data
    DataModelPB::RawData* parsed_data= new DataModelPB::RawData;
    bool isValidParsing= parsed_data->ParseFromArray(
       buffer_data,buffer_size
    );
    //check failed!!
    
  }//try
  catch (…) {
    //…
  }
}

For sure I'm doing something wrong. Probably somewhere in the client or within TANGO core (operators to extract data from attribute) the string is not fully read (i.e. truncated due to binary char).
Another question is: is this the suggested way to encode binary attributes or am I supposed to use other types (i.e. DevEncoded)?

I thank you all for your help.

Cheers,

Simone
****************************************************************
Simone Riggi
INAF, Osservatorio Astrofisico di Catania
Via S. Sofia 78
95123, Catania - Italy
phone: +39 095 7332 extension 282
e-mail: simone.riggi@gmail.com,
sriggi@oact.inaf.it
skype: simone.riggi
****************************************************************
rbourtembourg avatar
Reynald
Hi Simone,

I guess your problem is that there is a 0x00 which is interpreted as a '\0' (end of string) at some point in the protocol buffer you are sending.

I would suggest to try with DevVarCharArray rather than with DevVarStringArray type.
DevEncoded type could fit as well your need.

Hoping this helps,

Reynald
Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.
rbourtembourg avatar
Reynald
Dear Simone,

I hope you are not using Tango just as a way to transport binary data.
Please note that by doing so, you would lose many of the benefits provided by Tango.
Tango provides you some ways for the clients to discover the interface (list of attributes and commands) of your devices automatically.
It can provides you tools to configure the polling of some attributes, to configure a poll ring depth (ie the possibility to retrieve the last X values of a given attribute), to configure on which criteria events for a given attributes are sent to clients and to configure how and when a given attribute is archived, and more…

I would discourage you to use Tango just to transmit binary data unless you really need to, because this would make your code more complex and you would probably need to re-implement some features which are already there.

It would make your development more complex and you would lose the benefits of Pogo, the tango device server code generator, including the automatic documentation generation, for instance.

Tango is already encoding and decoding data for you and provides you a way to automatically discover the interface.
With protocol buffers, the client must know your protobuf interface in order to be able to decode the data (you add a dependency to a .proto file).

Hoping this helps,

Reynald
Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.
sriggi avatar
sriggi
Dear Raynald,

many thanks for your help. I will try the DevCharArray and I will let you know.

Let me explain briefly what I'm doing. I'm re-designing with TANGO an imaging computing system for production purposes I implemented time ago.
There's a device server acting as a gateway for incoming data produced by an acquisition system. Data are received, pre-processed and packed to be sent to other device servers performing imaging computation tasks.
Data are structured, e.g. they contain muon track hits, plus some reconstructed variables needed for the imaging tasks, etc.
From what I understand there is no possibility to have structured types as arguments (looking forwards for pipes in TANGO 9…), this is the reason why I'm employing some serialization schema to pack these data. The alternatives I see are keeping all the data variables as distinct arguments (and collect all argument events on clients) or packing as I'm currently doing (in the old software I was using a json encoding for example).

Hope that this clarifies a bit my aims.

Thank you again for your support,

Cheers,

Simone
****************************************************************
Simone Riggi
INAF, Osservatorio Astrofisico di Catania
Via S. Sofia 78
95123, Catania - Italy
phone: +39 095 7332 extension 282
e-mail: simone.riggi@gmail.com,
sriggi@oact.inaf.it
skype: simone.riggi
****************************************************************
 
Register or login to create to post a reply.