Tango Feature Request 4: Defining a standard Tango REST API

# 8 years ago
johfor	Hi all, I'm looking at the proposal for REST API posted by Ingvord. First of all, nice work, overall it looks great! I have a few comments off the top of my head. Regarding the CamelCase vs. snake_case question, I think it would just cause confusion to automatically try and convert between them. However in order to be consistent with Tango, the API should definitely be case insensitive for all Tango names. What case should be returned by the API I suppose must remain undefined (i.e. determined by the server). I suppose the "/{domain}/{family}/{member}…" paths should always be prefixed by something (e.g. "/devices/{domain}/… otherwise there could be some confusion (hypothetically there could be a domain called "devices"…). But maybe that was implied. Would it make sense to also add the server tree (like Jive) in a similar way, e.g. "/servers/MyServer/1/MyClass/some/device/name/..? I guess it's not as useful from a pure user perspective, but in order to allow full database configuration abilities it might be practical. But perhaps in that case, we should separate the "device" and "server" API specifications so that one does not have to wait for the other. I have a feeling that an API that allows e.g. creating servers and so on would get complicated… I agree that the PATCH operation is probably more confusing than useful. I do like the ability to read/write several attributes in one call. In the response for reading an attribute, how about using "value" instead of "argout"? Anyway we also need to be able to get the write value ("w_value"). These are the member names used in the Tango DeviceAttribute class so it seems consistent to keep them. I assume the choice to go with "argout" was made so that attribute and command results are consistent? I'm not sure what's more important here; consistency across the API, or with Tango… I'm playing around with a simple python implementation, with the intention to see if I run into any corner cases or other questions. I'll be back in that case :)

# 8 years ago

Hi all,

I'm looking at the proposal for REST API posted by Ingvord. First of all, nice work, overall it looks great! I have a few comments off the top of my head.

Regarding the CamelCase vs. snake_case question, I think it would just cause confusion to automatically try and convert between them. However in order to be consistent with Tango, the API should definitely be case insensitive for all Tango names. What case should be returned by the API I suppose must remain undefined (i.e. determined by the server).
I suppose the "/{domain}/{family}/{member}…" paths should always be prefixed by something (e.g. "/devices/{domain}/… otherwise there could be some confusion (hypothetically there could be a domain called "devices"…). But maybe that was implied.
Would it make sense to also add the server tree (like Jive) in a similar way, e.g. "/servers/MyServer/1/MyClass/some/device/name/..? I guess it's not as useful from a pure user perspective, but in order to allow full database configuration abilities it might be practical. But perhaps in that case, we should separate the "device" and "server" API specifications so that one does not have to wait for the other. I have a feeling that an API that allows e.g. creating servers and so on would get complicated…
I agree that the PATCH operation is probably more confusing than useful. I do like the ability to read/write several attributes in one call.
In the response for reading an attribute, how about using "value" instead of "argout"? Anyway we also need to be able to get the write value ("w_value"). These are the member names used in the Tango DeviceAttribute class so it seems consistent to keep them. I assume the choice to go with "argout" was made so that attribute and command results are consistent? I'm not sure what's more important here; consistency across the API, or with Tango…

I'm playing around with a simple python implementation, with the intention to see if I run into any corner cases or other questions. I'll be back in that case :)

# 8 years ago
Ingvord	Hi Johfor, Thank you very much for the input! johfor Would it make sense to also add the server tree (like Jive) in a similar way, e.g. "/servers/MyServer/1/MyClass/some/device/name/..? I guess it's not as useful from a pure user perspective, but in order to allow full database configuration abilities it might be practical. But perhaps in that case, we should separate the "device" and "server" API specifications so that one does not have to wait for the other. I have a feeling that an API that allows e.g. creating servers and so on would get complicated… I tend to move this to a particular client implementation, i.e. Feature Request #6 - Tango Web Application (aka Jive in browser via REST API). As you correctly mentioned it does not give a lot to the end users, but increases API and implementations significantly. BTW what do mean under `full database configuration abilities`? From my vision one has them just operating on the databaseds, am I wrong? For instance, `PUT http://host/tango_host_10000/devices/sys/databaseds/2/DbRenameServer?input=[TangoTest/test,TangoTest/development]` Anyway, we can add it later. johfor In the response for reading an attribute, how about using "value" instead of "argout"? Anyway we also need to be able to get the write value ("w_value"). These are the member names used in the Tango DeviceAttribute class so it seems consistent to keep them. I assume the choice to go with "argout" was made so that attribute and command results are consistent? I'm not sure what's more important here; consistency across the API, or with Tango Agree with value. What for do you need w_value? In my philosophy GET or PUT to device's attribute must always return the latest actual value of the attribute (PUT sometimes may return different value from what user has sent if there is a race). johfor I'm playing around with a simple python implementation, with the intention to see if I run into any corner cases or other questions. I'll be back in that case :) This is awesome. In the best case we can announce at least two implementations on the next Tango meeting. Keep me informed. Edited 8 years ago

# 8 years ago

Ingvord

Hi Johfor,

Thank you very much for the input!

johfor
Would it make sense to also add the server tree (like Jive) in a similar way, e.g. "/servers/MyServer/1/MyClass/some/device/name/..? I guess it's not as useful from a pure user perspective, but in order to allow full database configuration abilities it might be practical. But perhaps in that case, we should separate the "device" and "server" API specifications so that one does not have to wait for the other. I have a feeling that an API that allows e.g. creating servers and so on would get complicated…

I tend to move this to a particular client implementation, i.e. Feature Request #6 - Tango Web Application (aka Jive in browser via REST API). As you correctly mentioned it does not give a lot to the end users, but increases API and implementations significantly.

BTW what do mean under `full database configuration abilities`? From my vision one has them just operating on the databaseds, am I wrong? For instance,

PUT http://host/tango_host_10000/devices/sys/databaseds/2/DbRenameServer?input=[TangoTest/test,TangoTest/development]

Anyway, we can add it later.

johfor
In the response for reading an attribute, how about using "value" instead of "argout"? Anyway we also need to be able to get the write value ("w_value"). These are the member names used in the Tango DeviceAttribute class so it seems consistent to keep them. I assume the choice to go with "argout" was made so that attribute and command results are consistent? I'm not sure what's more important here; consistency across the API, or with Tango

Agree with value.

What for do you need w_value? In my philosophy GET or PUT to device's attribute must always return the latest actual value of the attribute (PUT sometimes may return different value from what user has sent if there is a race).

johfor
I'm playing around with a simple python implementation, with the intention to see if I run into any corner cases or other questions. I'll be back in that case :)

This is awesome. In the best case we can announce at least two implementations on the next Tango meeting. Keep me informed.

Edited 8 years ago

# 8 years ago
Ingvord	Hi All, Here is a new version of Tango REST API specification - Release Candidate 1 aka RC1. There will be some minor changes but as for now I want to freeze the specification to start implement it. Hopefully this one will make it to Executive Committee next Tango meeting, i.e. no major issues will occur.

# 8 years ago
Andy	Hi, I agree the w_value will be useful to have. Because the set+read values are closely related. Otherwise you have to cache the write value but you might have concurrency problems ensuring they refer to the same value in time. Is there a way the w_value can be supported? You are right about the database being a device which can use the same api. The same is true for the admin device in each server. Good news that coding on the first implementation is soon going to start. Kind regards Andy

# 8 years ago

Andy

Hi,

I agree the w_value will be useful to have. Because the set+read values are closely related. Otherwise you have to cache the write value but you might have concurrency problems ensuring they refer to the same value in time. Is there a way the w_value can be supported?

You are right about the database being a device which can use the same api. The same is true for the admin device in each server.

Good news that coding on the first implementation is soon going to start.

Kind regards

Andy

# 8 years ago
Ingvord	I do not know about set+read values. While dig in into it, could you please give me an example how it is used?

# 8 years ago
johfor	Regarding DB operations: I guess you are right, all operations are available through the database device commands. But this is sometimes pretty cumbersome, in part due to the limitations in the TANGO command arguments (expressing everything through lists of strings). So a more "friendly" API would probably be useful, e.g. where you can put many properties in one call, etc, but it can certainly wait. Perhaps it'd make sense to call the API under discussion the "client" API, and later perhaps define a "database" API? This split would be in line with the PyTango API at least. Regarding read/write values: For example, since a TANGO device is (usually) backed by some hardware which may have any kind of behaviour, such as being slow to react to changes, it can be useful to be able to tell if the last written value differs from the read value from the hardware. Also, if the set value is changed by someone else, it's good to be able to tell that it's not just the read value that is changing, which could be the case if the hardware is overloaded, etc. Anyway, would there be an issue with just adding a "w_value" field to the results for writeable attributes? I'm looking at the proposed RC1 API and I think it looks pretty complete, awesome job! I'll be back if/when I have some concrete comments.

# 8 years ago

johfor

Regarding DB operations: I guess you are right, all operations are available through the database device commands. But this is sometimes pretty cumbersome, in part due to the limitations in the TANGO command arguments (expressing everything through lists of strings). So a more "friendly" API would probably be useful, e.g. where you can put many properties in one call, etc, but it can certainly wait. Perhaps it'd make sense to call the API under discussion the "client" API, and later perhaps define a "database" API? This split would be in line with the PyTango API at least.

Regarding read/write values: For example, since a TANGO device is (usually) backed by some hardware which may have any kind of behaviour, such as being slow to react to changes, it can be useful to be able to tell if the last written value differs from the read value from the hardware. Also, if the set value is changed by someone else, it's good to be able to tell that it's not just the read value that is changing, which could be the case if the hardware is overloaded, etc. Anyway, would there be an issue with just adding a "w_value" field to the results for writeable attributes?

I'm looking at the proposed RC1 API and I think it looks pretty complete, awesome job! I'll be back if/when I have some concrete comments.

# 8 years ago
johfor	How about moving hyperlinks out of the "_meta" field into a separate "_links" field? I've seen this used elsewhere and it would make it easier to distinguish what's what. Also, what's the reason for the prefixing of "_" to the keys in these fields? There should be no risk of name collision there. Including the <prefix> in all links seems unnecessary, it should be enough to assume the API entry point. At least it could be optional. For consistency, perhaps the DevCommandInfo stuff (in_type, …) about commands could be put in an "info" field just like for attributes? I'm a bit unclear on how the filters work; do they apply recursively to all fields? All in all it looks great, but I haven't gotten to trying to implement it yet.

# 8 years ago

johfor

How about moving hyperlinks out of the "_meta" field into a separate "_links" field? I've seen this used elsewhere and it would make it easier to distinguish what's what. Also, what's the reason for the prefixing of "_" to the keys in these fields? There should be no risk of name collision there.
Including the <prefix> in all links seems unnecessary, it should be enough to assume the API entry point. At least it could be optional.
For consistency, perhaps the DevCommandInfo stuff (in_type, …) about commands could be put in an "info" field just like for attributes?
I'm a bit unclear on how the filters work; do they apply recursively to all fields?

All in all it looks great, but I haven't gotten to trying to implement it yet.

# 8 years ago
Ingvord	johfor How about moving hyperlinks out of the "_meta" field into a separate "_links" field? I've seen this used elsewhere and it would make it easier to distinguish what's what. Also, what's the reason for the prefixing of "_" to the keys in these fields? There should be no risk of name collision there. _meta refers to the implementation related stuff. Eventually it contains only links at present. So, yes, maybe we can rename it to _links. Prefix is to stress the fact that these info is not related to Tango. johfor Including the <prefix> in all links seems unnecessary, it should be enough to assume the API entry point. At least it could be optional. Yes, it is implementation's choice to put links relative to the entry point as well as to put an absolute path. Client should work fine in both cases. johfor For consistency, perhaps the DevCommandInfo stuff (in_type, …) about commands could be put in an "info" field just like for attributes? Ok. johfor I'm a bit unclear on how the filters work; do they apply recursively to all fields? Yes. Filter scans the whole response and keeps (or removes) specified fields only.

# 8 years ago

Ingvord

johfor
How about moving hyperlinks out of the "_meta" field into a separate "_links" field? I've seen this used elsewhere and it would make it easier to distinguish what's what. Also, what's the reason for the prefixing of "_" to the keys in these fields? There should be no risk of name collision there.

_meta refers to the implementation related stuff. Eventually it contains only links at present. So, yes, maybe we can rename it to _links.

Prefix is to stress the fact that these info is not related to Tango.

johfor
Including the <prefix> in all links seems unnecessary, it should be enough to assume the API entry point. At least it could be optional.

Yes, it is implementation's choice to put links relative to the entry point as well as to put an absolute path. Client should work fine in both cases.

johfor
For consistency, perhaps the DevCommandInfo stuff (in_type, …) about commands could be put in an "info" field just like for attributes?

Ok.

johfor
I'm a bit unclear on how the filters work; do they apply recursively to all fields?

Yes. Filter scans the whole response and keeps (or removes) specified fields only.

# 8 years ago
Manu	Just my two cents: In the restfull API, I have noticed the followed missing features compared to the C++ API. This does not mean that they have to be implemented but in case you did not know they exist: It is not possible to ping a device (ping() method on a DeviceProxy) It is not possible to retrieve the device black-box The source parameter is not handled meaning you rely on the default one which is CACHE_DEVICE Is it possible to do a "set_attribute_config()"? What about the history related calls (read_attribute_history() and command_inout_history()) What about the write_read_attribute() and write_read_attributes() calls (if it follows a RESTfull api) What about the new pipe related calls (too new may be) For my understanding, if I want to read several attributes in one call, what do I have to give to the GET verb? Regards Emmanuel

# 8 years ago

Manu

Just my two cents:

In the restfull API, I have noticed the followed missing features compared to the C++ API.
This does not mean that they have to be implemented but in case you did not know they exist:

It is not possible to ping a device (ping() method on a DeviceProxy)
It is not possible to retrieve the device black-box
The source parameter is not handled meaning you rely on the default one which is CACHE_DEVICE
Is it possible to do a "set_attribute_config()"?
What about the history related calls (read_attribute_history() and command_inout_history())
What about the write_read_attribute() and write_read_attributes() calls (if it follows a RESTfull api)
What about the new pipe related calls (too new may be)

For my understanding, if I want to read several attributes in one call, what do I have to give to the GET verb?

Regards

Emmanuel

# 8 years ago
Ingvord	REST becomes popular :) Good… Manu It is not possible to ping a device (ping() method on a DeviceProxy) In case a device is not there client will get standard failure response. If ping does what I think, i.e. checks that Tango server is alive – you simply do not need it. Manu It is not possible to retrieve the device black-box Could you please provide a usage scenario for this? Manu The source parameter is not handled meaning you rely on the default one which is CACHE_DEVICE I think is must be configured on the Web server side via special admin interface. As 99,9% of applications will just get the values and display them this should be fine?! Manu Is it possible to do a "set_attribute_config()"? What about the history related calls (read_attribute_history() and command_inout_history()) This I do not know – what are the usage scenarios? Manu What about the write_read_attribute() and write_read_attributes() calls (if it follows a RESTfull api) Write_read = write + read, i.e. finish write then read? Well any PUT request to an attribute(s) is write_read, i.e. synchronous, unless client specifies async=true Manu What about the new pipe related calls (too new may be) Well, yes… I took Tango 8 when started to work on this spec. Manu For my understanding, if I want to read several attributes in one call, what do I have to give to the GET verb? This is a tricky question. It is not intended to read multiple attributes at once. This is due to built-in HTTP cache. Reading only one value allows setting Last-Modified and Expires response headers. And I think this is more important than multiply reading in this case. Web server still can multiplex several requests into one and send read_attributes to a remote Tango server. Edited 8 years ago

# 8 years ago

Ingvord

REST becomes popular :) Good…

Manu
It is not possible to ping a device (ping() method on a DeviceProxy)

In case a device is not there client will get standard failure response. If ping does what I think, i.e. checks that Tango server is alive – you simply do not need it.

Manu
It is not possible to retrieve the device black-box

Could you please provide a usage scenario for this?

Manu
The source parameter is not handled meaning you rely on the default one which is CACHE_DEVICE

I think is must be configured on the Web server side via special admin interface. As 99,9% of applications will just get the values and display them this should be fine?!

Manu
Is it possible to do a "set_attribute_config()"?
What about the history related calls (read_attribute_history() and command_inout_history())

This I do not know – what are the usage scenarios?

Manu
What about the write_read_attribute() and write_read_attributes() calls (if it follows a RESTfull api)

Write_read = write + read, i.e. finish write then read? Well any PUT request to an attribute(s) is write_read, i.e. synchronous, unless client specifies async=true

Manu
What about the new pipe related calls (too new may be)

Well, yes… I took Tango 8 when started to work on this spec.

Manu
For my understanding, if I want to read several attributes in one call, what do I have to give to the GET verb?

This is a tricky question. It is not intended to read multiple attributes at once. This is due to built-in HTTP cache. Reading only one value allows setting Last-Modified and Expires response headers. And I think this is more important than multiply reading in this case. Web server still can multiplex several requests into one and send read_attributes to a remote Tango server.

Edited 8 years ago