Hi, Kyle.

Here's what I would like us to do on this.  (Note I am CC'ing the software list here because this discussion is relevant for a number of other people.)

1) There is already an existing ecal_leds table in the database that has LED config information (ecal_channel_id, crate, number, time_delay, amplitude_high, amplitude_low) so this covers some of the extra information that is requested.  We should update this so that it is current and correct.  I know Andrea had wanted to change the names of the amplitude variables to something else (blue and red or something?), etc.  But I'm not sure the mapping here to LED info is correct by channel_id.  Would you be able to check this for me?

2) We should come up with a complete set of the additional information needs to be added which isn't currently represented in the database.  Some of this I can tell from the CSV file you sent me, but I'm not an expert.  So I can't tell what some of this means.  Basically, we should strip out the basic DAQ mapping and the LED config information from this CSV file and the information that remains should be added to the database in an ecal_wiring table where it can easily be turned into a Java class.   It would be keyed on channel_id from the ecal_channels table so as not to repeat information like crate, etc.

3) You should then, once we have added all this information into the conditions API and database, write additional code in your event display to read this information using the conditions system, so that you no longer depend on text files for configuring the event display.  (Leaving the existing text based constructor in place is probably fine and could still be used for debugging.)

Does this sound like a reasonable plan?

--Jeremy

-----Original Message-----
From: Kyle McCarty [mailto:[log in to unmask]]

Sent: Tuesday, December 09, 2014 7:12 PM
To: McCormick, Jeremy I.
Subject: Re: talk + SVN account

Hello Jeremy,

That works for me. I'll trust you on the best way to structure it. As long as I can get the information, I'm good on my end.

Thanks,

Kyle


On Tue, Dec 9, 2014 at 10:08 PM, McCormick, Jeremy I. <[log in to unmask]> wrote:


        Hi,

        I think what we should do is leave the existing EcalChannel condition unchanged.

        We can then have another condition called maybe EcalChannelConfig which would map a channel_id onto the extra information for a given DAQ channel.

        I think that's probably the best way to go....

        --Jeremy

        -----Original Message-----
        From: Kyle McCarty [mailto:[log in to unmask]]

        Sent: Tuesday, December 09, 2014 7:07 PM
        To: McCormick, Jeremy I.
        Subject: Re: talk + SVN account

        Hello Jeremy,


        I had sent the CSV file in a different email, but I'll attach it here so you don't have to go looking for it.


        The counting house specifically asked me to make that information available on the event display because it was difficult to look up efficiently and the text file (which would still be the reference source for anything not on the conditions database) does not really give you an idea of the spatial positioning of anything. It is also useful to be able to easily see which LED is associated with which crystal without having to look up the channel ID or x/y indices and then search through a separate file.

        I think the ideal solution is to add the missing information to the database, and then I can do something like you suggest while still displaying everything that the counting house wants. I agree that the improved consistency is very important.


        Thanks,


        Kyle


        On Tue, Dec 9, 2014 at 9:57 PM, McCormick, Jeremy I. <[log in to unmask]> wrote:


                Hi, Kyle.

                Can you send me this CSV file you're using?  (Perhaps you already did and I missed it...)

                I believe it would be very useful if you had a constructor for the event display that took an EcalChannelCollection object.  Then an LCSim Driver could push this information to the event display in its detectorChanged method.  The EcalChannel API is now the standard way to get this information in the Java framework, and I think it would be best if the event display used it by default rather than a text file.  (You could still leave the text based constructor as well should you want to debug without database access.)

                Using the channel collection, you would then easily be able to setup the event display with the mapping of x, y to crate, slot and channel defined in the channel collection.

                If there is supplementary information in that CSV file that would be missing then just don't display it if it isn't present.  If you really want to use text files for some reason, then I believe you can use the ecal_channel_id from the conditions so as not to repeat the basic DAQ mapping in the text file.  You can instead key on channel_id.

                I'm really not convinced though that you need to put all kinds of DAQ specific information in the display.  Crate, slot and channel should be enough so that you can just find out the other DAQ configuration manually in the ECAL manual appendix.

                On the other hand, if your group decides that you really want all this extra information in the event display, it should not be a problem to add the information to the ecal_channels table.  That's actually brain dead easy.  I just need a text file containing this information.  (Your CSV file I guess?)

                The problem with using loose text files for class configuration is that you have absolutely no control over what text file is actually sent to the constructor by the user.  You would also need to provide this file along with the class for anyone to use it properly.  Please use the database in the future if at all possible, as we are moving away from using text files for this kind of stuff.

                --Jeremy

                -----Original Message-----
                From: Kyle McCarty [mailto:[log in to unmask]]

                Sent: Tuesday, December 09, 2014 6:52 PM
                To: McCormick, Jeremy I.
                Subject: Re: talk + SVN account

                Hello Jeremy,


                I agree that it would better to draw from the database, but what I'm getting from the counting house is that the other information in that CSV file is/will be very important, so it should remain accessible. Currently, I am only able to do that through the CSV file.


                Some information, like splitter number or LED channel can be useful if something goes wrong with the hardware because it makes it easier to track down the affected components quickly. It is also useful to have a more visual option for looking at these things than just a sea of number from a text file, which the event display integration does well. I do think that it is important to have these available for the counting house, though obviously many of these values are not useful for anything offline.

                If you can add the missing fields from the CSV file (I attached to the cross-check results email I sent a little while ago) then I would be happy to switch to the conditions database instead. I just don't want to remove functionality that was added specifically at the request of the counting house.


                Thanks,

                Kyle


                On Tue, Dec 9, 2014 at 9:42 PM, McCormick, Jeremy I. <[log in to unmask]> wrote:


                        Hi,
                        I believe you should add the capability to read from the DAQ map in the conditions system to setup the event display channel mapping.  Otherwise, there is no guarantee the whatever CSV file you are using is actually in sync with it!
                        I don’t see splitter number being all that useful (?) or it can easily be looked up by hand for a given channel.  It could easily be added to the ecal_channels table if we decide it is needed...
                        --Jeremy

                        -----Original Message-----
                        From: Kyle McCarty [mailto:[log in to unmask]]

                        Sent: Tuesday, December 09, 2014 6:34 PM
                        To: McCormick, Jeremy I.
                        Subject: Re: talk + SVN account

                        Hello Jeremy,


                        It seems that, end-of-file bug aside, I can run the monitoring application without any further errors. On the original point of the reading the CSV file versus the conditions database: The CSV file currently has useful information that we would like to have easily available that is not in the conditions database, such as splitter number. I don't want to convert the event display to read from the database because it would lose this information. If you want to add the information from the CSV file to conditions database, then I can perform this conversion. I'm pretty sure I know how to do it now that I have written that test script to compare the two. In the meantime, I would prefer to leave it reading from the CSV instead.

                        Note that the CSV file version will only load if the file exists in a certain location; otherwise it loads he default version which just displays crystal energy, the x-index, and the y-index, so it should have little affect on anyone outside the counting house.


                        Let me know if you add the information to the conditions database, though, and I will look into grabbing it from there instead.

                        Thanks,

                        Kyle


                        On Tue, Dec 9, 2014 at 5:14 PM, Kyle McCarty <[log in to unmask]> wrote:


                                Hello Jeremy,


                                The file is already an LCIO file; wouldn't we expect it to not work with EvioToLcio? If I use the "DummyMonitoring.lcsim" steering file with the same input LCIO file, I get the error message



                                        org.hps.record.RecordProcessingException: Error creating LCIO event.
                                            at org.hps.record.composite.LcioEventAdapter.recordSupplied(LcioEventAdapter.java:100)
                                            at org.freehep.record.loop.DefaultRecordLoop.consumeRecord(DefaultRecordLoop.java:832)
                                            at org.freehep.record.loop.DefaultRecordLoop.loop(DefaultRecordLoop.java:668)
                                            at org.freehep.record.loop.DefaultRecordLoop.execute(DefaultRecordLoop.java:566)
                                            at org.hps.record.composite.CompositeLoop.loop(CompositeLoop.java:211)
                                            at org.hps.record.composite.EventProcessingThread.run(EventProcessingThread.java:48)
                                        Caused by: org.freehep.record.source.NoSuchRecordException
                                            at org.lcsim.util.loop.LCIOEventSource.next(LCIOEventSource.java:136)
                                            at org.hps.record.composite.LcioEventAdapter.recordSupplied(LcioEventAdapter.java:89)
                                            ... 5 more



                                after around 343 / 343 events, but otherwise did not produce errors. It seems that the application has some issue handling the end of a file?


                                Thanks,

                                Kyle


                                On Tue, Dec 9, 2014 at 5:07 PM, McCormick, Jeremy I. <[log in to unmask]> wrote:


                                        Hi, Kyle.

                                        This doesn't look like a problem with the monitoring app itself.  The traceback is from the Driver org.hps.monitoring.ecal.plots.EcalHitPlots which presumably is looking for a collection that is no longer there.  The Driver needs to check that the collection looks correct before using it.  (Maybe discuss with Andrea about fixing this?)

                                        BTW, are you able to run this file okay with the EvioToLcio tool and a dummy steering file?

                                        --Jeremy

                                        -----Original Message-----
                                        From: Kyle McCarty [mailto:[log in to unmask]]

                                        Sent: Tuesday, December 09, 2014 1:31 PM
                                        To: McCormick, Jeremy I.
                                        Subject: Re: talk + SVN account

                                        Hello Jeremy,


                                        Sho and Omar fixed the readout issue so I am able to generate triggered LCIO files again. I loaded the generated file into the monitoring application and set it to run with the EcalMonitoringFinal,lcsim steering file. It correctly loads the GUI and seems to be working, except that whenever an new event is parsed, it produces the following error and does not display anything:



                                                java.lang.ClassCastException: org.lcsim.lcio.SIOGenericObject cannot be cast to org.hps.readout.ecal.TriggerData
                                                    at org.hps.readout.ecal.SSPData.getOrTrig(SSPData.java:146)
                                                    at org.hps.monitoring.ecal.plots.EcalHitPlots.process(EcalHitPlots.java:201)
                                                    at org.lcsim.util.Driver.doProcess(Driver.java:273)
                                                    at org.lcsim.util.Driver.processChildren(Driver.java:284)
                                                    at org.lcsim.util.Driver.process(Driver.java:198)
                                                    at org.lcsim.util.DriverAdapter.recordSupplied(DriverAdapter.java:74)
                                                    at org.hps.record.composite.LcioEventAdapter.recordSupplied(LcioEventAdapter.java:95)
                                                    at org.freehep.record.loop.DefaultRecordLoop.consumeRecord(DefaultRecordLoop.java:832)
                                                    at org.freehep.record.loop.DefaultRecordLoop.loop(DefaultRecordLoop.java:668)
                                                    at org.freehep.record.loop.DefaultRecordLoop.execute(DefaultRecordLoop.java:566)
                                                    at org.hps.monitoring.gui.MonitoringApplication.nextEvent(MonitoringApplication.java:1238)
                                                    at org.hps.monitoring.gui.MonitoringApplication.actionPerformed(MonitoringApplication.java:279)
                                                    at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
                                                    at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2346)
                                                    at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
                                                    at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
                                                    at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
                                                    at java.awt.Component.processMouseEvent(Component.java:6527)
                                                    at javax.swing.JComponent.processMouseEvent(JComponent.java:3321)
                                                    at java.awt.Component.processEvent(Component.java:6292)
                                                    at java.awt.Container.processEvent(Container.java:2234)
                                                    at java.awt.Component.dispatchEventImpl(Component.java:4883)
                                                    at java.awt.Container.dispatchEventImpl(Container.java:2292)
                                                    at java.awt.Component.dispatchEvent(Component.java:4705)
                                                    at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4898)
                                                    at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4533)
                                                    at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4462)
                                                    at java.awt.Container.dispatchEventImpl(Container.java:2278)
                                                    at java.awt.Window.dispatchEventImpl(Window.java:2739)
                                                    at java.awt.Component.dispatchEvent(Component.java:4705)
                                                    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:746)
                                                    at java.awt.EventQueue.access$400(EventQueue.java:97)
                                                    at java.awt.EventQueue$3.run(EventQueue.java:697)
                                                    at java.awt.EventQueue$3.run(EventQueue.java:691)
                                                    at java.security.AccessController.doPrivileged(Native Method)
                                                    at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75)
                                                    at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:86)
                                                    at java.awt.EventQueue$4.run(EventQueue.java:719)
                                                    at java.awt.EventQueue$4.run(EventQueue.java:717)
                                                    at java.security.AccessController.doPrivileged(Native Method)
                                                    at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:75)
                                                    at java.awt.EventQueue.dispatchEvent(EventQueue.java:716)
                                                    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
                                                    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
                                                    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
                                                    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
                                                    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
                                                    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)



                                        I assume that this is related to the monitoring application not displaying any events, but I am not sure. Do you know why this is happening?

                                        Thanks,

                                        Kyle


                                        On Mon, Dec 8, 2014 at 8:06 PM, McCormick, Jeremy I. <[log in to unmask]> wrote:


                                                I think Omar made some changes in the LCIO to EVIO which broke the SVT part.  You might want to correspond with him about it, and please CC hps-software list.  I don't think he thought anyone was trying to use it right now...

                                                -----Original Message-----
                                                From: Kyle McCarty [mailto:[log in to unmask]]

                                                Sent: Monday, December 08, 2014 3:28 PM
                                                To: McCormick, Jeremy I.
                                                Subject: Re: talk + SVN account

                                                I sent you some information over the GChat client.


                                                On Mon, Dec 8, 2014 at 6:22 PM, McCormick, Jeremy I. <[log in to unmask]> wrote:


                                                        Hi, again.

                                                        You might find the example code I added here to be useful

                                                        https://confluence.slac.stanford.edu/display/hpsg/ECAL+Channel+Mapping+Conditions

                                                        under " Accessing ECAL Channel Conditions in Java".

                                                        It shows from Java how to access ECAL channel conditions, perform various types of searches, etc.

                                                        --Jeremy

                                                        -----Original Message-----
                                                        From: Kyle McCarty [mailto:[log in to unmask]]
                                                        Sent: Monday, December 08, 2014 3:11 PM
                                                        To: McCormick, Jeremy I.
                                                        Subject: Re: talk + SVN account


                                                        Hello Jeremy,

                                                        So let's start with getting the monitoring application to actually run, since I need that before I can really test anything else. I can initialize it, but I can't actually run it over anything. Obviously, I can not connect to the ET ring, but I should be able to run over an LCIO file. I can not test that, though, because I am unable to generate a readout file. The HPS2014ReadoutToLcio.lcsim steering file's readout to LCIO driver produces an error whenever I run it:



                                                                java.lang.ClassCastException: org.lcsim.detector.tracker.silicon.HpsSiSensor cannot be cast to org.lcsim.detector.tracker.silicon.HpsTestRunSiSensor
                                                                    at org.hps.evio.SVTHitWriter.makeFpgaData(SVTHitWriter.java:71)
                                                                    at org.hps.evio.SVTHitWriter.writeData(SVTHitWriter.java:176)
                                                                    at org.hps.evio.TestRunTriggeredReconToLcio.process(TestRunTriggeredReconToLcio.java:173)
                                                                    at org.lcsim.util.Driver.doProcess(Driver.java:273)
                                                                    at org.lcsim.util.Driver.processChildren(Driver.java:284)
                                                                    at org.lcsim.util.Driver.process(Driver.java:198)
                                                                    at org.lcsim.util.DriverAdapter.recordSupplied(DriverAdapter.java:74)
                                                                    at org.freehep.record.loop.DefaultRecordLoop.consumeRecord(DefaultRecordLoop.java:832)
                                                                    at org.freehep.record.loop.DefaultRecordLoop.loop(DefaultRecordLoop.java:668)
                                                                    at org.freehep.record.loop.DefaultRecordLoop.execute(DefaultRecordLoop.java:566)
                                                                    at org.lcsim.util.loop.LCSimLoop.loop(LCSimLoop.java:151)
                                                                    at org.lcsim.job.JobControlManager.run(JobControlManager.java:421)
                                                                    at org.lcsim.job.JobControlManager.run(JobControlManager.java:183)
                                                                    at org.lcsim.job.JobControlManager.main(JobControlManager.java:172)



                                                        This error has persisted for a while. Do you know what the problem is? It seems that something is broken with the SVT, which I do not know how to fix. Can you repair this so that I can generate some LCIO data to use with the monitoring application?

                                                        Thanks,

                                                        Kyle



                                                        On Mon, Dec 8, 2014 at 5:54 PM, McCormick, Jeremy I. <[log in to unmask]> wrote:


                                                                Okay, answers below....

                                                                > That might be tricky; in on shift through Thursday so my schedule is rather strange. We might be able to Wednesday morning around noon.

                                                                Wednesday could work fine for me at noon your time.

                                                                > As far as the recent update, you can remove it if you want.

                                                                If you need it for testing, we'll leave it for now.  It doesn't hurt anything really.  I just want to make sure this is not a permanent solution for anything because it isn't.

                                                                > It was a temporary thing for debugging and that was the best way I could think of to get it to the counting house where it could be tested, since I can't run the monitoring application on my computer and the Internet permissions are strange in the counting house.

                                                                Okay, why can't you use the monitoring application on your computer?  The SLAC database which is cloned from JLAB should be accessible to your machine.  If need be you can even clone the entire conditions database to your local machine and run a MySQL server.  It actually isn't that hard to do.  Let me know if you need instructions on this or a SQL file dump in order to do this.  If you are managing your own Linux machine this will only take like 10 minute to do.

                                                                > If you would like me to do this from the conditions database, I need to be able to access it and download the mappings.

                                                                Well, I think you really need to be getting this information from the database instead of a text file in order to reproduce how the monitoring app will actually run in the counting house.

                                                                I have sent several messages to the hps-software list on dumping info from the conditions database using a command line tool.  Can you review them and let me know of any questions?  If you can't find the specific messages let me know and I'll forward to you.

                                                                You can also connect directly to the database through the hpscalibrations account.  (Maybe Nathan can tell you more about this?)

                                                                > I also need to access then by crystal ix and iy, because that is how they are stored in the event display. If you can give me some code that does this I can try to integrate it into the event display.

                                                                Yes, this is easy to do from the class EcalChannel in the conditions module which has ix, iy for geometric ID and the corresponding crate, slot, channel mapping.

                                                                You can see the details here.

                                                                https://confluence.slac.stanford.edu/display/hpsg/ECAL+Channel+Mapping+Conditions

                                                                I'll add a code example there when I get a chance.


                                                                On Dec 8, 2014 5:34 PM, "McCormick, Jeremy I." <[log in to unmask]> wrote:


                                                                        Hi, Kyle.

                                                                        Can we phone meet at some point to talk about plans/status on software that you're working on?  Let me know some times that are good for you.

                                                                        OR

                                                                        If you want to have a quick gchat instead that might be okay too.

                                                                        I need to make sure we're on the same page as far as configuration of the software and related issues.  For instance, you should not any longer be reading conditions type information like DAQ mapping from text files.  That just doesn't work well anymore now that everything is in a database, primarily because the user then needs to have that text file to even run your classes.  And it may not even be in sync with what is in the database.  (Unless I'm misunderstanding what exactly it is that you're doing recently on the event display with csv files.)

                                                                        Also, on an unrelated note, can you change to using '[log in to unmask]' for your SVN login?  It should be setup with the same password as your "old" account.

                                                                        --Jeremy