Subscribe events from device with many attributes

I'm trying to get events from device with really many attributes.
(About 600 attributes). Below code is what I tried with wild chars.

	try 
	{ 
		// create a connection to a TANGO device 
		device = new DeviceProxy(device_name); 
  
		// Ping the device 
		device->ping(); 


		double_callback = new DoubleEventCallBack;

		const string attr_name("*");  // this causes exception

		event_id = device->subscribe_event(attr_name, Tango::CHANGE_EVENT, double_callback);
		std::cerr << "event_client() id = " << event_id << endl;
		// The callback methods are executed by the Tango event reception thread.
		// The main thread is not concerned of event reception. 

		return 1;
	} 
	catch (DevFailed &e) 
	{ 
		Except::print_exception(e);
		return -1;
	}

And exception show that I cannot do this. (See attached picture.
)

It says that * attribute not found.
I was expecting that receiving events from all attributes available under the device.

Is it impossible?

Hello,

As far as I know, wildcards are not supported by the subscribe_event method.
When you look at the API documentation ( http://www.esrf.eu/computing/cs/tango/tango_doc/kernel_doc/cpp_doc/classTango_1_1DeviceProxy.html#a5fef157c74708fa54acde839749b9449 ), you can see that it says:

Parameters
[in] att_name The attribute name

There is no mention of wildcards support here.
If you want to subscribe to events for all the attributes of a specific device, you will need to:
- get the list of all the attributes names of the device using Tango::DeviceProxy::get_attribute_list() method.
- Iterate through the returned vector and subscribe for each attribute individually.

Hoping this helps,

Reynald
Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.
There's another function with less argument like this

//——————————————————————————————————————-
//
// method :
// 		DeviceProxy::subscribe_event
//
// description :
//		Subscribe to a device event- Add the statless flag for stateless event subscription.
//
//——————————————————————————————————————-

int DeviceProxy::subscribe_event (EventType event,CallBack *callback,bool stateless)
{
	if (version < MIN_IDL_DEV_INTR)
	{
		stringstream ss;
		ss << "Device " << dev_name() << " does not support device interface change event\n";
		ss << "Available since Tango release 9 AND for device inheriting from IDL release 5 (Device_5Impl)";

		Tango::Except::throw_exception(API_NotSupportedFeature,ss.str(),"DeviceProxy::subscribe_event()");
	}

    ApiUtil *api_ptr = ApiUtil::instance();
  	if (api_ptr->get_zmq_event_consumer() == NULL)
	{
		api_ptr->create_zmq_event_consumer();
	}

    int ret;
	ret = api_ptr->get_zmq_event_consumer()->subscribe_event(this,event,callback,stateless);

	return ret;
}

May I ask what is this function used for, then?

I could change my test code without any error, but it won't get any eventssmile

	try 
	{ 
		// create a connection to a TANGO device 
		device = new DeviceProxy(device_name); 
  
		// Ping the device 
		device->ping(); 


		double_callback = new DoubleEventCallBack;

		if (attribute_name.length() == 1 && attribute_name[0] == '*')
		{
			event_id = device->subscribe_event(Tango::INTERFACE_CHANGE_EVENT, double_callback);
		}
		else
		{
			const string attr_name(attribute_name);
			event_id = device->subscribe_event(attr_name, Tango::CHANGE_EVENT, double_callback);
		}

		std::cerr << "event_client() id = " << event_id << endl;
		// The callback methods are executed by the Tango event reception thread.
		// The main thread is not concerned of event reception. 

		return 1;
	} 
	catch (DevFailed &e) 
	{ 
		Except::print_exception(e);
		return -1;
	}
Hi,

As explained in the documentation (http://www.esrf.eu/computing/cs/tango/tango_doc/kernel_doc/cpp_doc/classTango_1_1DeviceProxy.html#ac33ae96ec8f69bdd5a16020fd799293c), DeviceProxy::subscribe_event (EventType event,CallBack *callback,bool stateless) method is available only for the device interface change events, which will send events … only if your device interface changes(!), when new dynamic commands or dynamic attributes are created.
You are probably not interested in that in your case.

There are several types of events in TANGO. You need to understand the differences between these different types of events and subscribe to the ones you are interested in.

I advise you to try to receive events from one attribute first, to get familiar with Tango events and because you seemed to be interested in attribute related events.
Please be aware that some configuration might be needed.
If you are not pushing events manually from your device server code, you will have to activate the polling on the attributes you want to receive events from and depending on the event type you are interested in, you might need to configure the events parameters for these attributes from jive because the events will be sent only if some conditions are met, which is the main interest of this feature actually.
Please have a look at the section 4.6 from the TANGO Controls System Handbook.

Hoping this helps,

Reynald
Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.
Hi,

what events are you expecting from the 600? Is your device server pushing CHANGE events or have you configured the polling and events to have the TANGO library send them? 600 attributes sounds like a lot (too much?). Even if it works and interface with 600 different attributes will be difficult to handle in the gui tools and for the operators. Are you sure some attributes cannot be grouped together to reduce the overall number. I would try to get them under 100 unless you have a special use case and you really need 600 and your operators know how to manage this. If you could tell us more about your use case we could maybe give you some tips.

Your need to subscribe to events from many attributes using wildcards like "*" is not possible today. In order to do this in code today you have to introspect the interface to get the list of attributes for your device and then subscribe to each one in a loop. It might be a good idea to add a feature request for subscribing to a group of attributes.

Cheers

Andy
Thanks Reynald and Andy for your feedback.

Tango system needs to get data from 3500 devices.
And each device has from 350 to 1500 attributes. (So I assume that average 600 attributes)
The total number of the attributes is over 2,100,000.

Regarding the GUI, this size can be handled without much trouble, I assume.
Of course operators cannot manage this just using jive toolssmile

Even though making them under 100 per each device (as Andy's suggestion), it comes to 350,000 attributes.
As the manual says polling is inefficient under this circumstances.

I'm thinking push subscribe instead of pull, because the events are not triggered so often.
So I configured the polling and event and let the TANGO library handle them.


As Reynald suggested, I could loop each attributes for one device and found it works.
But repeating them over 3500 times really takes too much timesmile
Also I'm afraid TANGO could handle the full capacity, finally.


Below is what I tested with only one device;


// global variables
DeviceProxy *device = NULL;
DoubleEventCallBack *double_callback = NULL;
deque<int> event_ids;


// forward declarations
int handleExit();
bool subscribeEvent(const string &attr_name);


int main(int argc, char* argv[])
{
	string device_name;
	string attribute_name;

	if (argc == 3)
	{
		device_name = argv[1];
		attribute_name = argv[2];
	}
	else
	{
		cerr << "Usage : TestEvents.exe [device_name] [attribute_name]" << endl;
		cerr << "Example :" << endl;
		cerr << "   Get one attribute" << endl;
		cerr << "        TestEvents.exe sys/database/2 one_attr" << endl;
		cerr << "   Get all attributes" << endl;
		cerr << "        TestEvents.exe sys/database/2 *" << endl;

		// use default names
		device_name = "sys/database/2";
		attribute_name = "*";
		cerr << "Connect to device " << device_name << " …" << endl;
	}

	try 
	{ 
		// create a connection to a TANGO device 
		device = new DeviceProxy(device_name); 
  
		// Ping the device 
		device->ping(); 


		double_callback = new DoubleEventCallBack;

		if (attribute_name.length() == 1 && attribute_name[0] == '*')
		{
			vector<string> *attribute_name_list = device->get_attribute_list();

#if DO_NOT_USING_THIS_IT_IS_TOO_SLOW
			/* get the attribute configuration */
			Tango::AttributeInfoList *attr_config_list;
			attr_config_list = device->get_attribute_config(*attribute_name_list);
			
			/* loop over all attributes */
			for (int i=0; i<attribute_name_list->size(); i++)
			{
				Tango::AttributeInfo ai = attr_config_list->at(i);

				// skip write attribute
				if (ai.writable != READ)
					continue;

				subscribeEvent(ai.name);
			}
#else
			/* loop over all attributes */
			for (int i=0; i<attribute_name_list->size(); i++)
			{
				Tango::AttributeInfo ai = device->get_attribute_config(attribute_name_list->at(i));

				// skip write attribute
				if (ai.writable != READ)
					continue;

				subscribeEvent(ai.name);
			}
#endif
		}
		else
		{
			subscribeEvent(attribute_name);
		}

		// The callback methods are executed by the Tango event reception thread.
		// The main thread is not concerned of event reception. 

		// Sleep some time to check events
		sleep(10000);

		return handleExit();
	} 
	catch (DevFailed &e) 
	{ 
		Except::print_exception(e);
		return -1;
	}

	return 0;
}

bool subscribeEvent(const string &attr_name)
{
	assert(device != NULL);

	try 
	{
		int event_id = device->subscribe_event(attr_name, Tango::CHANGE_EVENT, double_callback);
		//std::cerr << "event_client() id = " << event_id << endl;
		event_ids.push_back(event_id);
		return true;
	}
	catch (DevFailed &e) 
	{ 
		Except::print_exception(e);
		return false;
	}

	return false;
}

int handleExit()
{
	if (device != NULL)
		return 0;
	
	try 
	{
		while (!event_ids.empty())
		{
			int event_id = event_ids.back();
			device->unsubscribe_event(event_id);
			event_ids.pop_back();
		}

		delete double_callback;
		delete device;

		return 1;
	}
	catch (DevFailed &e) 
	{ 
		Except::print_exception(e);
		return -1;
	}

	return 0;
}

While testing events, I found that the device name and attribute name is coming with small chars only.smile
Any option to get the names as they are. (case sensitive)
Hi,

now that I understand your use case a but better. Tango should be able to scale to the number of attributes you mention because it is based on a point to point protocol between servers and clients. If you need more attributes and the server cannot handle the load then simply distribute the attributes to more servers and more hosts. You are right the polling thread is probably not optimised for 600 attributes but would need to be tested to be sure. It depends on how the attribute data is read. If the values are already in memory then 600 is not an issue. Version 9 of Tango has optimised the polling a bit more by reading all attributes polled with the same period in one call using read_attributes().

If you want to avoid the large number of device attributes to import by clients you could group multiple attributes into one and return the data in an array. This will enable you to reduce the number of attributes. This is a common design pattern if you have attributes which are related.

You are right the name returned is normalised to small characters. This is because Tango does not distinguish between small and big letters in device names. Therefore all device and attribute names are converted to small letters in the code to optimise comparisons.

Hope that helps.

Andy
Andy

If you want to avoid the large number of device attributes to import by clients you could group multiple attributes into one and return the data in an array.

When you say "goup", you mean Spectrum attribute?

Not the "Tango::Group" that something like


g1 = new Tango::Group("many_devices_all_together");
g1->add("my/device/01");
g1->add("my/device/03");
g1->add("my/device/02");


By group I mean simply putting data into an array (Spectrum or Image data type) and not the Groups supported by the api you mentioned.

The Groups are convenient for accessing many devices with one call either reading/writing their attributes or sending the same command to many devices. For example we have 224 beam position monitors which we access using the Groups in the api.

But as you have noticed these Groups do not support subscribing to events from hundreds of attributes in one call. Reducing the number of attributes would help solve this problem. Note it is only necessary to solve if you have a client(s) which will subscribe to all the attributes. If each client subscribes to only a few attributes then it is not an issue.

Andy
 
Register or login to create to post a reply.