RFC 4 - Tango V10
at the next Tango meeting we will organise a session to discuss the needs and directions for the next major release of Tango V10. We would like to get your input on what you think is required. Please bear in mind that we have to stay backwards compatible while still ensuring the long term sustainability of Tango. This means we were thinking to work more on sustainability rather than new features. We need your input to draw up the requirements of the next major version of Tango.
Please use this forum post to provide input. Your input will be discussed with the kernel developers and the executive committee and summarised during the next Tango meeting in Toulouse.
Inputs from SKA DISH
2016-06-16 19:37 GMT+02:00 Simone Riggi <email@example.com>:
Here are some inputs for the Tango future:
Documentation: This improved a lot since I started using Tango (three years ago) from scratch so I'm happy to see the efforts from the developer. Still there is room for improvements that can help a new developer exposed to Tango for the first time :
- The documentation is still a bit fragmented. Adding more on tools would be great, for example Astor (each time I realize I need to remember how I did some stuff)/Jive. It would be useful to show a complete example covering: "How to create a Device with Pogo", "how to register your device with Jive". Documentation on how to create a device with additional user thread (Yat/omniThread) and shared data/signals is useful. I needed to pick devices from the community to understand a bit the practices used.
- Populating the HowTo would be perfect for some recurrent task that currently you have to search/extrapolate (not so immediate) from the manual. Some possibility are:
1) How to define READ_WRITE attributes and properly write their read_XXX and write_XXX methods. I needed again to search for community devices to understand.
2) How to define different kind of events in a device and subscribe from a client invoking a callback on receipt
3) How to create and fill pipe blobs (also complex)
4) How to create command response for some types: DevVarLongStringArray. This can be found in the manual, but I was asked by sub-elements on how to set command response.
Core: There are some limitations in the core that have been put in evidence during the harmonization work. I guess you will present on this.
The most frustrating for me is the limitation of 1 single command argument in input and in output. Often you are forced to use an external serializer (json, msgpack, protobuf) to have some flexibility. Tango developers says this is to respect OO paradigm and interface but I don't really agree on that (see also a past mail from Andrea De Marco). It is a limitation instead. I know this is probably related to CORBA data types so this goes directly to the key question. In the past they announced the intention of completely replacing CORBA with ZeroMQ also for command (not just for events). It seems from Andy's post that we won't see too many news on Tango on this aspect. It would be nice to understand the road map they are planning.
Another useful feature would be to have some persistence in clients, particularly for event subscriptions.
I will think again if I can find other nice-features-to-have before the conference.
On 16 June 2016 at 19:55, Simone Riggi <firstname.lastname@example.org> wrote:
I forgot also to mention that adding few other States besides to the 14 ones would be easy and at the same time provide the possibility to build Pogo (and instructions).
Inputs from SKA CSP
Sonja <Sonja.Vrcic@nrc-cnrc.gc.ca> wrote:
Thank you for giving us opportunity to add to your list.
Here are suggestions based on CSP needs and experiences:
Transition to ZeroMQ is critical (we know it’s already planned but just to emphasise)
Ability to set multiple parameters using a single command is required. I see no reason why the number of parameters should be limited, once CORBA is gone. We should be able to use a single command to set any number of parameters of any type. Ditto for read access.
Support for multi-threading – we will have TANGO Servers with a large number of (different) TANGO devices, where each TANGO Device communicates with multiple target devices (other TANGO Devices and/or non-TANGO devices, such as application software, microprocessors, FPGAs, GPUs).
Ability to pass a file or a reference to a file (URL or name&location) so that another TANGO Device can fetch (download) the file is a necessity. We could define a non-TANGO workaround for that, but it would be nice to include that in the standard functionality.
Need to improve documentation:
a) Document on Java Server lacks useful and complete example of a TANGO Server with multiple devices (at least two instances of two different devices) and the data base configuration for that (how to add devices to the database).
b) Most people I talked to encounter the same problem: installation steps are not well documented.
c) Suggestion: hire a developer/technical writer who would generate detailed instructions for installation for all three supported languages, and for two or more flavours of Operating Systems. Include concrete examples and detailed instructions how to download and install TANGO, Taurus, data base, and get whole system running. Create instructions and examples for the case when the code for TANGO Servers and Devices are generated using POGO and for the case when developer choses to implement everything manually.
d) Improve the overall description. In many cases well-known general concepts are discussed in detail, and TANGO specific details are skimmed over.
I am sure I missed some important aspects, but this is all for now,
Inputs from SKA AAVS and SKA LFAA
On 16 June 2016 at 21:17, Andrea De Marco <email@example.com> wrote:
Here's input from the experience so far with LFAA. Some of this will overlap with what has already been mentioned. There are workaronds and techniques that work well with TANGO that can be used to circumvent the "need" for the following features. However, if this kind of behaviour becomes part of the native functionality provided by TANGO, we believe it will not only suit SKA better, but also make TANGO an even more elegant product.
1. Ability to transfer data files over the network through a TANGO API call. It is not suggested that the ZeroMQ bus is used for the actual data transfer, but TANGO can be used to marshal the connection between client and server.
2. Support for delayed or future run commands. This could take the form of an extension to the asynchronous command runs in TANGO, with the addition of a "delay" parameter, which takes in a number of seconds as input. The actual command execution cycle would then possibly take the form of a two-stage process when delays are set. Delayed commands are "stored" internally in a queue, which is serviced by a thread when the delay has elapsed.
3. As many have pointed out, it would really be ideal that multiple parameters are supported for commands, to avoid the use of workarounds such as pipes, or handling using JSON etc.
4. The transition to ZeroMQ is not deemed as "critical" - it is well hidden in the TANGO API. However, the limitations imposed by it can make workarounds a "consistent effort" for SKA development.
5. We have found general documentation to be relatively sufficient - though the PyTango documentation is a bit lacking. Some more detail on TANGO-based tools, rather than TANGO-core would be ideal.
6. We think the TANGO model could become more extensible if all states could be custom defined. We realize this is not currently possible since it breaks compatibility with some of the TANGO tools e.g. GUI components. This is not a high priority for LFAA.
7. We feel that the TANGO-core alarms are far from sufficient for our purposes, and the use of the Elettra/Alba alarm extensions is mandatory. It would be ideal if something on the lines of Elettra/Alba extensions find their way inside TANGO-core. A sort of "alarm event" is required - this is in fact the rationale behind the Elettra/Alba extensions.
8. Ideally, there is the ability to define custom event types. We realize this is not an easy thing to implement. From the point of view of LFAA, we have found that it would be good to have one more additional event - a device event, which is not related to any particular attribute/change etc. A device could simply "emit" an event, and other devices/clients subscribed to device-level events would receive the notification.
Inputs from SKA TelMgt, GMRT and TCS
From: Vatsal Trivedi <firstname.lastname@example.org>Date: 17 June 2016 at 14:21
Subject: Re: Inputs for Tango documentation session
Below are consolidated inputs from TelMgt team, GMRT team and TCS GMRT M&C team on Tango documentation, Tango tools and components and feature requests for next Tango version
Suggestion on Tango Documentation
• Provide step-by-step guide for installation of Tango Control System Framework and associated tools such as Taurus, HDB++ Archiver for multiple operating systems such as Windows, Ubuntu, Fedora, RedHat Linux.
• Chapter #1 Introduction of the Tango Control System Manual starts with introduction/explanation of Tango Device Server. For first time readers it is difficult to follow the details provided in this chapter (Chapter #1). Instead this chapter shall provides a brief description of the Tango Control System Framework and its core concepts such as Tango Device Server, Tango Device, Tango Class. Details of the Tango Concepts shall be addressed in the subsequent chapters.
• Known errors and resolutions shall be included in the documentation- One has to browse through the forum, to be able to come to know about it.
• Details related to preferred operating system platforms/ variants / versions etc. shall also be mentioned in the Tango documentation.
Specific to the Tango Tools and Components
• Tango Access Control (TAC) - Very little documentation available. Detailed instructions are required for someone to get started with it. More information on configuration to use TAC is required. For example, checking/setting class properties, how to configure multiple hosts etc. is missing in the documentation. Information about some limitations like inability of Starter DS to directly start/stop DS written in JAVA/python through Astor, if mentioned, would be helpful.
• POGO and JIVE - There are many options provided with these tools, but how the selection of specific choice affect the code / behavior etc. is not available
• Tango Logging Service - The documentation of TANGO logging service, especially for Java specific implementation is inadequate.
• TAURUS - Comparatively, documentation for PyQT and Taurus is good. It can be further improved by including relevant examples
• HDB++ and PANIC - Installation procedure for HDB++ and PANIC needs to be simplified. Manual of these tools, should have a clear information about configuration of archiving/alarm rules through .csv/.txt files. All available Syntax/ functions supported for configuring the alarms rule needs to be documented properly. Details of HdbEventSubscriber/HdbConfugurationManager DS commands should be explained with example.
• Support for multiple input parameters for a command.
• POGO supports creating multiple classes Server project for C++ only and that too on Ubuntu/Linux platform. To add support for creating multiple classes Server project for Java and Python language on all supported operating system from POGO.
• Support for setting log level on target basis.
GMRT and TCS GMRT M&C team members will provide you additional details related to various issues they faced while using Tango.
More inputs from TCS and GMRT:
Here the additional set of general TANGO related points. You may want to take up these, during your discussions with TANGO experts/ owners. I hope, I am not too late to provide these inputs.
* When python-client connects to device server implemented in C++, detecting the events over the network is issue, however it works if the client and server are running on the same computer. This issue is specifically with PyTango and likely to be addressed in next version. But not sure what is the road map/ plan for releasing the newer version.
* If the static attributes generated using the C++ code, they do not reflect in ATK Panel, but this problem is not seen with Java implementation.
* When POGO is used to generate Java code, few syntax errors are observed. These need to be corrected before compiling the code. Also, it doesn’t create ‘bin’ folder if it does not exists. It’s required to explicitly provide the TANGO ‘jar’ files / added into the make file.
* There is scope for the improvement in the TANGO access control. At the moment it uses the computer credentials for authenticating to Tango system. This puts limitation on having multiple users to a tango based application software. It will be nice, if the role based access control mechanism is implemented in it. Similarly the modifications to the HDB++ archiver and PANIC alarm suit to support the user / role based authentication could be considered.
From: Vikas Kumthekar/PNE/TCS
thank you for your comments and requests for TANGO V10. They will be compiled into list to be submitted to the kernel group for discussion in the breakout session at the Tango meeting on Tuesday. We will post the results. Some of the requests require big changes others are more like bug fixes. We will see when and how to integrate them. It will be useful to convert the bugs to bug reports.
Inputs from SKA TM
Sorry for the late addition. I saw this morning that I did not add the SKA TM responses and am adding it now for completeness sake.
Inputs from SKA TM.LMC - Matteo Di Carlo
It is not easy to answer the Andy's question and We can only try to give you our perspective of what is important to have for a documentation.
The main difficulties we found was on understanding how the framework works, for instance how it realizes the polling mechanism works and so on. Ideally an UML class diagram together with other activities diagrams could help a lot in understanding it. Another documentation missing is an E/R model for the database: for instance, what happens if you add a device? what are the main tables affected? So basically I am talking about technical software documentation.
We like CORBA because is a standard and is working very well but we don't like the limitation on the class types you can add in your project. In fact, at the moment, you cannot add (easily) new class in the Corba IDL and use them a run-time. I didn't study the tango pipes and I suppose this overcome the above limitation.
Inputs from SKA TM SE - Paul Swart
From my (little) exposure to tango 9 documentation, I would like the following to be added via sections and diagrams:
• an architecture overview for TANGO,
• diagrammatic representations of how it all fits together (here included enabling things like Taurus).
Inputs from SKA SA LMC Prototyping - Neilen Marais
NM - Comments about TANGO documentation / Finding stuff
Hard to find documentation about the "built in" tango tools, there is a list of tools on the tango home page, but for many it does not even mention the name of the executable.
Many things are quite spread out. "Official" Tango components have documentation hosted over several sites. Many links between these sites are dead, due to web reorganisation?
Bit of confusion about where source code repos live, PyTango moving to github was somewhat hidden if you were not already part of the community.t
Inputs from Synchronization & Timing LMC (SAT.LMC)
The .pdf that we have is more that 2MB (2.7MB). Zipping doesn't work (gets reduced to 2.11MB), hence not able to attach.
Would prefer emailing to the concerned person.
You can mail it to me at - andy dot gotz at esrf dot fr
I am collating all the input into a document.