Hello HPS Software folks, particularly, Miriam, Omar, Holly, Matt, Norman, Bradley, Kyle, Sebouh, ...
I would like to see if the time is here to really get going on a discussion about our data summary output. From what I observe, there is a quick hack that started with Sho and that grew organically into what is now the Tuple Maker. There is a lot to be said for this system, including that it is used by quite a few of you, but I don’t think I could argue this is the ideal situation. The recent effort to re-factor it and document it was excellent. Was that enough? Can this system be considered production ready now?
We also have a DST maker, which Omar produced and maintains, which had some thought go into the design, but it seems does not fit people’s needs, else there would be no need for a tuple maker. What would make the DST maker universal?
So, now that we have some experience with all this, I would like to start a discussion on what it is that we actually want for a DST output, and how would we get there. I would like this to be a discussion at our next software meeting, but if that is too soon, we can also discuss it in the next one.
Can someone present the pro side of the tuple maker? Holly? Miriam?
Can someone present the pro side of the DST maker? Omar?
What are the limitations in either of the implementations?
To get you thinking about the topic, let me make some controversial statements:
- Is the underlying reason for the Tuple maker, and the clunky Java to text file to ROOT mechanism, that LCIO is simply a horrible data format?
- So should we get rid of LCIO then, which seems to have hampered our ability to write the data we want to write on many occasions?
- If not, why the text file?
- Is there an issue with how we use, and understand, the ROOT file format, and the various ways in which you can use it to access structured data?
- Do we want the data structured, or do we prefer a simple tuple because it is easier?
- We risk making mistakes in the analysis because we do not know exactly what went into our output. We also risk wasting people’s time if they started out with the “wrong” dst and then have to switch to the other one. How do we avoid these pitfalls?
- Is this not really a concern? Nothing to see here, carry on….
I hope we can find our way out of what appears to me is our current not so great situation.
Best regards,
Maurik