Event system problem - TANGO Controls

# 7 years ago
daneos	Hello, I have set up a test system with Tango 9.2.2 and I'm facing a following problem: When I create DeviceProxy with just the device path, everything works fine. `In [2]: PyTango.DeviceProxy("sys/tg_test/1").subscribe_event("double_scalar", PyTango.EventType.CHANGE_EVENT, PyTango.utils.EventCallBack()) 2016-11-25 12:11:51.822173 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 254.091776645 2016-11-25 12:11:54.240473 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 255.025814753 2016-11-25 12:11:56.240080 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 255.376374226 2016-11-25 12:11:58.240392 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 255.649143713` But if I add also database address I start getting EventTimeout errors. In [2]: PyTango.DeviceProxy("localhost:10000/sys/tg_test/1").subscribe_event("double_scalar", PyTango.EventType.CHANGE_EVENT, PyTango.utils.EventCallBack()) 2016-11-25 12:13:10.428639 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 214.699918216 2016-11-25 12:13:22.403601 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [API_EventTimeout] Event channel is not responding anymore, maybe the server or event system is down 2016-11-25 12:13:22.404218 SYS/TG_TEST/1 DOUBLE_SCALAR IDL5_CHANGE [ATTR_VALID] 198.949672514 2016-11-25 12:13:32.413914 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [API_EventTimeout] Event channel is not responding anymore, maybe the server or event system is down 2016-11-25 12:13:32.414499 SYS/TG_TEST/1 DOUBLE_SCALAR IDL5_CHANGE [ATTR_VALID] 187.226889937 2016-11-25 12:13:42.432173 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [API_EventTimeout] Event channel is not responding anymore, maybe the server or event system is down It's a problem, since for example Taurus works that way. I have Tango 9.2.2, omniORB 4.1.7 and ZeroMQ 4.1.4. Thank you for help!

# 7 years ago

Hello,
I have set up a test system with Tango 9.2.2 and I'm facing a following problem:

When I create DeviceProxy with just the device path, everything works fine.


In [2]: PyTango.DeviceProxy("sys/tg_test/1").subscribe_event("double_scalar", PyTango.EventType.CHANGE_EVENT, PyTango.utils.EventCallBack())
2016-11-25 12:11:51.822173 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 254.091776645
2016-11-25 12:11:54.240473 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 255.025814753
2016-11-25 12:11:56.240080 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 255.376374226
2016-11-25 12:11:58.240392 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 255.649143713

But if I add also database address I start getting EventTimeout errors.


In [2]: PyTango.DeviceProxy("localhost:10000/sys/tg_test/1").subscribe_event("double_scalar", PyTango.EventType.CHANGE_EVENT, PyTango.utils.EventCallBack())
2016-11-25 12:13:10.428639 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [ATTR_VALID] 214.699918216
2016-11-25 12:13:22.403601 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [API_EventTimeout] Event channel is not responding anymore, maybe the server or event system is down
2016-11-25 12:13:22.404218 SYS/TG_TEST/1 DOUBLE_SCALAR IDL5_CHANGE [ATTR_VALID] 198.949672514
2016-11-25 12:13:32.413914 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [API_EventTimeout] Event channel is not responding anymore, maybe the server or event system is down
2016-11-25 12:13:32.414499 SYS/TG_TEST/1 DOUBLE_SCALAR IDL5_CHANGE [ATTR_VALID] 187.226889937
2016-11-25 12:13:42.432173 SYS/TG_TEST/1 DOUBLE_SCALAR CHANGE [API_EventTimeout] Event channel is not responding anymore, maybe the server or event system is down

It's a problem, since for example Taurus works that way.
I have Tango 9.2.2, omniORB 4.1.7 and ZeroMQ 4.1.4.

Thank you for help!

# 7 years ago
cfalcon	Hi Tangoers, I can not reproduce the error with a similar configuration omniORB 4.2.0 libtango9 9.2.2 python-pytango 9.2.0 zmq 4.1.2 In [1]: import PyTango In [2]: id = PyTango.DeviceProxy("controls04:10000/sys/tg_test/1").subscribe_event("short_scalar", PyTango.EventType.CHANGE_EVENT, PyTango.utils.EventCallBack()) 2016-11-25 12:50:14.462563 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 30 In [3]: 2016-11-25 12:50:16.273153 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 118 2016-11-25 12:50:18.372217 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 154 2016-11-25 12:50:20.172216 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 22 2016-11-25 12:50:22.273077 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 143 2016-11-25 12:50:24.373237 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 159

# 7 years ago

cfalcon

Hi Tangoers,

I can not reproduce the error with a similar configuration

omniORB 4.2.0
libtango9 9.2.2
python-pytango 9.2.0
zmq 4.1.2


In [1]: import PyTango

In [2]: id = PyTango.DeviceProxy("controls04:10000/sys/tg_test/1").subscribe_event("short_scalar", PyTango.EventType.CHANGE_EVENT, PyTango.utils.EventCallBack())
2016-11-25 12:50:14.462563 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 30

In [3]: 2016-11-25 12:50:16.273153 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 118
2016-11-25 12:50:18.372217 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 154
2016-11-25 12:50:20.172216 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 22
2016-11-25 12:50:22.273077 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 143
2016-11-25 12:50:24.373237 SYS/TG_TEST/1 SHORT_SCALAR CHANGE [ATTR_VALID] 159

# 7 years ago
Reynald	Hi, I managed to reproduce your problem. If you set TANGO_HOST environment variable to localhost:<port_number>, and do not use the Tango Fully Qualified Domain Name (FQDN) in your client when you subscribe to events, the events work as expected. If you use a FQDN starting with localhost:<port_number>, you get the problem. I think if you use your real machine name in the FQDN, it should work. It seems like when you set TANGO_HOST to localhost:<port_number> and do not use the FQDN, TANGO is replacing localhost with the real machine name at the time of the subscription. It looks like this step is not done when localhost is used in the FQDN. This is a problem because the device server will send events with a name containing the real machine name, not localhost. Then, there is a mismatch between what the client is expecting and what the server is sending, so the heartbeat mechanism does not work as expected in this specific case. Try using your machine name in the FQDN and please avoid using localhost in the FQDN until we fix this problem. Hoping this helps. Reynald Rosenberg's Law: Software is easy to make, except when you want it to do something new. Corollary: The only software that's worth making is software that does something new.

# 7 years ago

Reynald

Hi,

I managed to reproduce your problem.
If you set TANGO_HOST environment variable to localhost:<port_number>, and do not use the Tango Fully Qualified Domain Name (FQDN) in your client when you subscribe to events, the events work as expected.
If you use a FQDN starting with localhost:<port_number>, you get the problem.
I think if you use your real machine name in the FQDN, it should work.
It seems like when you set TANGO_HOST to localhost:<port_number> and do not use the FQDN, TANGO is replacing localhost with the real machine name at the time of the subscription. It looks like this step is not done when localhost is used in the FQDN.

This is a problem because the device server will send events with a name containing the real machine name, not localhost.
Then, there is a mismatch between what the client is expecting and what the server is sending, so the heartbeat mechanism does not work as expected in this specific case.

Try using your machine name in the FQDN and please avoid using localhost in the FQDN until we fix this problem.
Hoping this helps.

Reynald

Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.

# 7 years ago
daneos	Hi, Indeed it seems that using machine FQDN resolves the problem. I have run couple of tests and it seems to be working. Thank you very much!

# 7 years ago
Reynald	FYI, I created the following issue on GitHub to keep track of this problem: https://github.com/tango-controls/cppTango/issues/315 Cheers Rosenberg's Law: Software is easy to make, except when you want it to do something new. Corollary: The only software that's worth making is software that does something new.