Hi Yuri,

I'll answer a couple of these things to the best of my ability and others can jump in with more complete answers or corrections.

> From the point of view of software tool, option 2) seems workable, provided one can link in both LCIO and ROOT into the same executable.
> Is it the case?

Yes, this is how Omar is making the ROOT DST, if I understand correctly: A C++ program that uses both the ROOT libraries and the LCIO C++ libraries to read an LCIO file and output a ROOT file.

> Having an example of the event loop program that would create a dummy root tree would be a great start. 
> Would it need the full-blown Java framework to work?

Omar's code will probably be a good example, but I think there are other examples, including one that comes with LCIO.   There is a lot of LCIO documentation available on the web, easily discoverable via google search.  No java is required, of course: LCIO has both java and C++ support, unlike ROOT.

> From the point of view of data access - is it practical to run only from the LCIO files? How are they going to be distributed?
> Is LCIO event content fixed? I.e. can it be made lighter by keeping only certain collections? Is that a reasonable way to make
> a microDST? 

Yes, certainly, it is practical.  LCIO was long ago settled on as the EDM for HPS and the LCIO output will be the raw output format of the experiment. Like any persistency framework, we can decide exactly what parts of the events we can afford to output to tape.  That content has not yet been settled.  There are some things that for whatever reason haven't been persisted as LCIO objects in the past (but should be) and there are other things we have been writing to the events that won't be feasible to keep when we have large volumes of data.  One could make an LCIO micro-DST also, but it seems the desire is to convert to ROOT at the point of slimming the data for analysis.

> Related to that question - is it possible to browse the LCIO files? Or at least somehow ask an event what kind of objects it contains?

The LCIO event browser is a standard plugin for JAS (Java Analysis Studio) along with the integrated event display, Wired (which lets you select objects in the event display and get information about them).  For example:

https://confluence.slac.stanford.edu/display/ilc/Using+the+LCSim+Event+Browser

… which I found just by googling "LCIO event browser"… there may be better docs someplace else.

> Thanks
> -y

Sure.  I hope this helps make things a bit clearer.

Tim

> 
> On Mar 6, 2013, at 6:10 PM, McCormick, Jeremy I. wrote:
> 
>> Hi, Homer.
>> 
>> Thanks for the thoughts.
>> 
>> My view is that user analysis has three possible pathways which make sense to consider:
>> 
>> 1) Pure Java analysis using lcsim and outputting histograms to AIDA files, viewable in JAS.
>> 
>> 2) LCIO/ROOT analysis, reading in the LCIO recon files, looping over these events, and making histograms from a ROOT script.
>> 
>> 3) Pure ROOT analysis, operating on a ROOT DST file.
>> 
>> I don't really think that we need a DST containing all of the information which is already present in the final LCIO recon file.  This level of duplication is not desirable.  Rather, the ROOT DST should contain physics objects only, e.g. the equivalent of LCIO ReconstructedParticles, Tracks, and Clusters, along with event information.  This should be sufficient for doing a pure physics analysis, e.g. good enough for most users.  It is also likely that it could be represented using simple arrays rather than classes, which to me is desirable for this kind of format.
>> 
>> If one wants to look at the associated hits of the tracks, or something similarly detailed, then it seems to me that it would be better to use the #1 and #2 approaches, as we can then avoid "reinventing the wheel" by making ROOT files that mimic the structure of the existing LCIO output.  This approach would require working from the LCIO output, but I really don't see a problem there.  It is not onerous at all.  The API is straightforward and well-documented, and examples can be provided.  There is already a simple analysis script in my examples that you linked which plots information from Tracks in an LCIO file using ROOT histogramming.  Similar plots could easily be made for the hits, etc. 
>> 
>> I suppose one could demand that all this data be put into ROOT including the hits, but you're left with the same problem.  Someone still has to learn the API of whatever classes are used to store the data, and the class headers also need to be loaded to interpret the data.  Whether that format is LCIO or ROOT, it is essentially the same level of knowledge that would be required.  My feeling is actually that this will be more difficult/cumbersome to work with in ROOT rather than LCIO.  I wonder why we can't just go with what we already have, e.g. the LCIO API, rather than invent something analogous which does not seem to serve a very clear purpose.  One can already use what's there in the linked example to look at the full events, so can we start there and see how far we get?
>> 
>> If someone has a clear use case where pure ROOT data is needed at the lowest level of detail, I would consider this request, but I have seen nothing concrete so far along these lines.
>> 
>> --Jeremy
>> 
>> -----Original Message-----
>> From: Homer [mailto:[log in to unmask]] 
>> Sent: Wednesday, March 06, 2013 2:51 PM
>> To: Jaros, John A.; Graham, Mathew Thomas; McCormick, Jeremy I.; Graf, Norman A.; Moreno, Omar; Nelson, Timothy Knight
>> Subject: DSTs and work on slcio files using C++
>> 
>> Hi,
>> 
>> I decided not to comment during the meeting because it might have created more contention and I also wanted to hear Jeremy's, Norman's and Omar's responses first before throwing this out there. That said, from the point of view of someone who has been doing lcsim SiD analysis on slcio files I find the problems with using the two formats in HPS a little strange. For SiD we take slcio files and then run jet clustering and flavor tagging using C++ code in the lcfi and 
>> lcfi+ packages. For the flavor tagging we write out root files for 
>> lcfi+ running the
>> TMVA training and then for both the jet clustering and the flavor tagging we write out slcio files. I believe Malachi has done his whole analysis in C++ as a Marlin processor. I had also successfully tested reading slcio files in ROOT using a recipe provided by Jeremy. I dropped using it when I realized that it was quite simple to write the analysis in java. Perhaps one solution is to stick to doing all development, even for the DST, in java/lcsim and to just provide examples of how to access the data from C++/ROOT reading slcio files. Jeremy had documented much of this long ago at:
>> 
>> https://confluence.slac.stanford.edu/display/hpsg/Loading+LCIO+Files+into+ROOT
>> 
>> If we just provide some examples, wouldn't that help to at least put out the current fires? This would also avoid having to support numerous extra sets of data (DSTs and microDSTs in both formats with multiple passes and subsets)??
>> Maybe I'm wrong but I think one can provide simple recipes or modules for accessing any of the slcio file contents in ROOT.
>> 
>>   Homer
>> 
>> 
>> ########################################################################
>> Use REPLY-ALL to reply to list
>> 
>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>> <winmail.dat>
> 
> --------------------------
> Prof. Yuri Gershtein
> [log in to unmask]
> http://physics.rutgers.edu/~gershtein
> (732)445-5500 x1794
> W316 Serin Building
> Department of Physics and Astronomy
> 136 Frelinghuysen Rd
> Rutgers University
> Piscataway, NJ 08854
> 


########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the HPS-SOFTWARE list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1