Reimplement the GUI in pure Python.

[heijligen]

The current GUI is implemented in C++ under src/hdtv/rootext/display/. It is based on Cern's ROOT library, with additional calls directly to libX11, and loaded via the PyROOT interface as ROOT.HDTV.Display. It is the main reason HDTV is not cross-platform capable.
A 2nd dummy implementation exists in src/hdtv/dummy.py. It can replace the original GUI to run the tests or for a headless mode #61. This is done through the monkey_patch_ui class in src/hdtv/util.py:849.

A new interface should be reimplemented in pure Python to overcome the limitations of the current one.
A prototype can be found in https://github.com/hdtv-tool/gui-concept and used as reference. It uses PyQt5, Uproot5, Matplotlib, and Scipy

---

Ideas:

• The same mechanism as Add headless mode #61 can to be used to switch between a old and new GUI
• The Interface should be well documented
• C++ has types, those can be type annotations for the Python Interface making future work easier.

@h-mayr

[YanzhaoW]

Hi,

I feel the underlying GUI, XApplication, seems to be outdated. Using python library could be a solution. But maybe a Web GUI with rootjs could be more portable?

[op3]

I have looked into this in the past, and while I have not investigated your suggested solutions in detail (jsroot or matplotlib), my assumption was always that they are not flexible enough to easily fulfill our requirements. Both are not primarily as user interfaces, but as plotting libraries, and while they each provide some UI capabilities, I am not sure if they scale well enough for our purposes.

I would also like to emphasize some performance requirements: I think it is reasonable to assert that a user wants to load about 100 spectra with 16k bins each, which corresponds to about 32M lines to draw all the histograms, which will be quite resource-intensive. hdtv (currently) doesn’t handle navigating within those spectra very well, but at least you can render it once and then do some fitting. At least I was always struggling with this when using hdtv and would have like to have this capability.

A few years ago, I did some experiments with pyqtgraph and was quite satisfied in general. Unfortunately, their plotting interface does not cover all of our use cases, and pyqtgraph development seems to rather slow/not prioritizing what we need. I was able to add symlog scaling to pyqtgraph using AI as a POC, but submitting that upstream would also require more work.

Alternatively, it might also be possible to rebuild the hdtv UI from scratch based on some primitive UI library. If I am not mistaken, most of the interface is already drawn manually anyways? Essentially, we would replace X11 with a modern, optionally GPU-accelerated drawing library. I am not sure if this is feasible though, or if there are any surprises, e.g., regarding interactivity, or how well this scales to large amounts of large spectra.

[h-mayr]

The matplotlib prototype feels quite well in handling spectra – better than I've expected. Maybe we should try if scalability is really a problem or if matplotlib could serve our purposes.

[heijligen]

@YanzhaoW : My primary idea was to replace current UI with a more maintainable variant of itself. This would presumably keep the scope relative limited. And later changes to a Python only codebase are much easier. Using standard Python components would also mostly solve the problem of HDTV currently being only a Linux (X11) application.
I see the benefit of a Web UI, but I think it's currently not really feasible. Nevertheless, I'm not fixed on this position.

@op3 / @h-mayr : Do you have a set of histograms for benchmarking?

[YanzhaoW]

@heijligen

Yeah, I agree that moving to WebUI is an insurmountable task. But I would suggest to keep the old UI along the new UI to give people an option to go back to the old way if the new way somehow doesn't work properly.

Something like using hdtv --ui=python for python ui or hdtv --ui=root for root ui.

[op3]

I would be careful about the maintenance effort required to maintain two UIs. I don’t think this project is able to sufficiently take care of two UIs existing in parallel; just the ROOT UI is already complex enough.

For any UI implementation intended to supplant the ROOT UI, I would suggest to implement it in an API-compatible way first. Potentially with a “clean” internal API that is better aligned with the new implementation and intended future use cases (the current UI does not provide the right abstractions, and in many places is too deeply integrated into hdtv). And then a backwards-compatible wrapper around the internal API of the new UI that provides an API compatible with the current API of the ROOT UI.

As soon as the new UI is on par with the ROOT UI, and all use cases are covered and sufficiently tested, the ROOT UI can be deleted. Afterwards, its possible to start refactoring the API and make use of the improved abstraction and potentially new features.

[h-mayr]

@YanzhaoW : My primary idea was to replace current UI with a more maintainable variant of itself. This would presumably keep the scope relative limited. And later changes to a Python only codebase are much easier. Using standard Python components would also mostly solve the problem of HDTV currently being only a Linux (X11) application. I see the benefit of a Web UI, but I think it's currently not really feasible. Nevertheless, I'm not fixed on this position.

@op3 / @h-mayr : Do you have a set of histograms for benchmarking?

I have a ton of histograms. Do they need to fulfill any requirements? Should they be large?

[heijligen]

@h-mayr Can you add a set of different files to the tpse25 cloud folder? Best would be a diverse set of smaller and larger files. The root format would be easier to work with than the alternative matrix formats.

about 100 spectra with 16k bins each, which corresponds to about 32M lines

[h-mayr]

I've provided some spectra in different formats. For performance tests, however, it should also help to load the same file several times.