Thanks, Alessandra.
I just made a few changes to the RunManager so it will get the URL of the db connection from the prop file.
The username, password and database name are still hard-coded and not taken from the prop file.
This resolves the JIRA item ...
https://jira.slac.stanford.edu/browse/HPSJAVA-665
I tested that setting "org.hps.conditions.connection.file" in the environment works properly for the run db.
I didn't bother implementing reading of the prop file from a resource as is possible with the conditions db (it is probably not needed for the run db).
So from now on you shouldn't need to have a hacked copy of RunManager to run offline.
If you have any questions let me know.
--Jeremy
-----Original Message-----
From: Alessandra Filippi [mailto:[log in to unmask]]
Sent: Sunday, December 11, 2016 5:46 AM
To: Maurik Holtrop
Cc: McCormick, Jeremy I.; Omar Moreno; Uemura, Sho; hps-software
Subject: Re: problems in reading a local DB
solved!! (I believe)
I also retrieved a dump of the hps_run_db_v2 DB, and point to this in the hardcoded fix of RunManager.java (telling to get it from localhost instead of hpsdb), while the .prop file always points to local hps_conditions. Is this consistent?
I'm not sure I fully understand why it works.. but it does :-)
Let me know if you want me to add a few notes to the confluence page describing how to run successfully offline, with a local copy of the two DBs.
Thanks, cheers
Alessandra
On Sun, 11 Dec 2016, Alessandra Filippi wrote:
> Hi Maurik, all,
> thanks, with this fix I am able to read my local db.
>
> But, er... as Sho was supposing, I am afraid I am not able to provide
> the proper db dump. I am retrieving the hps_conditions db, is this correct?
> or do I need a different one?
> To get it, I use the command
>
>> mysqldump -f --lock-tables=false -h hpsdb.jlab.org -u hpsuser -p
> hps_conditions > hps_conditions_2016_12_11.mysql
>
> if I don't use the -f flag the dump ends almost immediately with this
> complaint (and the dumpfile is small and clearly incomplete):
>
> mysqldump: Couldn't execute 'show create table
> `conditions_engrun2015_pass2`': SHOW VIEW command denied to user
> [log in to unmask] for table 'conditions_engrun2015_pass2'
> (1142)
>
> If I add the -f flag, it still complains with this and other similar
> warnings (telling that I cannot view conditions_engrun_pass1_view and
> svt_alignments_engrun_view), but the dump goes on and the dumpfile is
> much larger (but this does not mean it contains everything needed, I suppose).
> When I run hps-java, after some happy queries, I get this message:
>
> Exception in thread "main" java.lang.RuntimeException:
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table
> 'hps_conditions.run_summaries' doesn't exist
>
> so clearly I am missing something/doing something wrong with the dump.
> Any idea?
> If it's too complicated and involving privileges I haven't, I would be
> happy just to retrieve a recent dumpfile from somewhere, if someone
> can provide me one copy (I have an old one, dating back to 2015, but I
> guess it's too old - and indeed it does not work properly).
> Thanks for your help.
> cheers
> Alessandra
>
>
>
>
>
>
>
> On Sat, 10 Dec 2016, Maurik Holtrop wrote:
>
>> Hello Alessandra,
>> You are correct in your assessment that the code does not connect to
>> the DB. There are 2 locations in the code where the code tries to
>> connect. In one, it does listen to the prop file, in the other the
>> address is hard coded. In the print outs that you show, the first one
>> … [WARNING]… is the hard coded one not getting a connection with
>> hpsdb.jlab.org, because you are offline. In your second printout, you
>> are online, so you don’t get any message from that, and so now it
>> tells you that the other connection that does use the prop file
>> indeed connected to localhost.
>>
>> The file that causes the issue is
>> here: run-database/src/main/java/org/hps/run/database/RunManager.java
>>
>> Here is the bit of code that I have changed to make it work on our
>> farm
>>
>> package org.hps.run.database;
>>
>> import java.sql.Connection;
>> import java.sql.SQLException;
>> import java.util.List;
>> import java.util.logging.Logger;
>>
>> import org.hps.conditions.database.ConnectionParameters;
>> import org.hps.record.daqconfig.DAQConfig;
>> import org.hps.record.epics.EpicsData; import
>> org.hps.record.scalers.ScalerData;
>> import org.hps.record.svt.SvtConfigData; import
>> org.hps.record.triggerbank.TriggerConfigData;
>> import org.lcsim.conditions.ConditionsEvent;
>> import org.lcsim.conditions.ConditionsListener;
>>
>> /**
>> * Manages access to the run database.
>> *
>> * @author Jeremy McCormick, SLAC
>> */
>> public final class RunManager implements ConditionsListener {
>>
>> /**
>> * The default connection parameters for read-only access to the
>> run database.
>> */
>> private static ConnectionParameters DEFAULT_CONNECTION_PARAMETERS
>> = new ConnectionParameters("hpsuser",
>> "darkphoton", “hps_run_db", “127.0.0.1");
>>
>> /**
>> * The singleton instance of the RunManager.
>> */
>> private static RunManager INSTANCE;
>>
>>
>>
>>
>> The 127.0.0.1 is the same as “localhost”, usually. Depending on your
>> DB setup you may need to change the username “hpsuser” or password or
>> database name.
>>
>> I hope this helps.
>>
>> Best,
>> Maurik
>>
>>
>>
>> On Dec 10, 2016, at 4:18 AM, Alessandra Filippi
>> <[log in to unmask]>
>> wrote:
>>
>> Hi all,
>> so, I guess the issue is that I'm tying to run it completely offline
>> (I just need to debug the recostruction on a few events, but I cannot
>> be online as I'm away). The command I run is the
>> following:
>>
>> java -Dorg.hps.conditions.connection.file=local.prop
>> -DdisableSvtAlignmentConstants -cp hps-distribution.jar
>> org.hps.evio.EvioToLcio -d HPS-EngRun2015-Nominal-v5-0-fieldmap
>> -DoutputFile=test -x
>> EngineeringRun2015FullRecon_Pass2_Gbl.lcsim
>> DATA2015/hps_005772.evio.0-nominalAllevents/hps_005772.evio.0
>>
>> where hps-distribution.jar is pointing to v3.11 hps-java .jar
>> (freshly updated). If I run it offline, I just get
>> this:
>>
>> 2016-12-10 09:58:08 [WARNING]
>> org.hps.conditions.database.ConnectionParameters createConnection ::
>> Failed to connect to database jdbc:mysql://hpsdb.jlab.org:3306/ -
>> Communications link failure
>>
>> The last packet sent successfully to the server was 0 milliseconds
>> ago. The driver has not received any packets from the server.
>> [..]
>> (repeated for 10 times)
>>
>> which tells me that it doesn't care at all about my local.prop (even
>> if I provide the name of a prop file that does not exist).
>>
>> If I run it with the network, indeed the file is read:
>>
>> 2016-12-10 09:52:48 [CONFIG]
>> org.hps.conditions.database.DatabaseConditionsManager
>> setConnectionProperties ::
>> setting connection properties file local.prop [...]
>>
>> 2016-12-10 09:59:55 [INFO]
>> org.hps.conditions.database.DatabaseConditionsManager openConnection ::
>> opening
>> connection ...
>> connection: jdbc:mysql://localhost:3306/
>> host: localhost
>> port: 3306
>> user: filippi
>> database: hps_conditions
>>
>> so, if I understand it correctly, as a provisional shortcut I should
>> hardcode the call to my local db overriding the search over the
>> network of hpsdb.
>> May you tell me where?
>> thanks
>> Alessandra
>>
>>
>>
>>
>>
>>
>>
>>
>> On Fri, 9 Dec 2016, McCormick, Jeremy I. wrote:
>>
>> Alessandra, can you please send the full print output from your
>> job and what command line you are
>> running?
>>
>> We should start by double checking that at least your prop file
>> is being picked up for the conditions db
>> before worrying about what is happening with the run db ...
>>
>> -----Original Message-----
>> From: Maurik Holtrop [mailto:[log in to unmask]]
>> Sent: Friday, December 09, 2016 3:00 PM
>> To: Omar Moreno
>> Cc: McCormick, Jeremy I.; Uemura, Sho; Alessandra Filippi;
>> hps-software
>> Subject: Re: problems in reading a local DB
>>
>> Yes, this has been an issue for quite some time, and Jeremy has
>> been aware of it:
>> https://jira.slac.stanford.edu/projects/HPSJAVA/issues/HPSJAVA-665?filter=allopenissues
>> I think we should either obey the prop file everywhere, OR,
>> hard code it everywhere and not deceive
>> people with a prop file.
>>
>> I have had this issue on our backend farm, which simply cannot
>> see hpsdb.jlab.org, so I went with a
>> local modification that hard codes the db name everywhere.
>>
>> Perhaps Jeremy can comment on why this is hard to fix?
>>
>>
>>
>> On Dec 9, 2016, at 5:21 PM, Omar Moreno <[log in to unmask]> wrote:
>>
>> I think she is trying to run completely offline.
>>
>>
>> On Fri, Dec 9, 2016 at 2:07 PM, McCormick, Jeremy I.
>> <[log in to unmask]> wrote:
>>
>>
>> Sho is right -- the run DB URL is currently separate from the
>> conditions db prop file and hard-coded....
>>
>> Are you trying to run completely offline?
>>
>> You should be able to connect to the run db if you have an
>> internet connection, as it is publically
>> accessible.
>>
>>
>> -----Original Message-----
>> From: [log in to unmask]
>> [mailto:[log in to unmask]
>> <mailto:[log in to unmask]> ] On Behalf Of Sho Uemura
>> Sent: Friday, December 09, 2016 1:51 PM
>> To: Alessandra Filippi
>> Cc: hps-software
>> Subject: Re: problems in reading a local DB
>>
>> The instructions don't cover making a local copy of the run DB,
>> which is a separate database also at
>> JLab, and uses a different set of connection parameters. I
>> think the run DB was added sometime
>> during/after the 2015 run, so you might not have had to deal
>> with it before.
>>
>> On Fri, 9 Dec 2016, Alessandra Filippi wrote:
>>
>> > Hi all,
>> > I'm trying to use a local copy of the conditions DB to run
>> 2015 data
>> > reconstruction. I'm following the instructions given in
>> >
>> https://confluence.slac.stanford.edu/display/hpsg/Installing+the+Condi
>> <https://confluence.slac.stanford.edu/display/hpsg/Installing+the+Condi>
>> > tions+Database+Locally
>> >
>> > (I remember that some time ago they were working). However,
>> when I run
>> > the reconstruction with the suggested command java
>> > -Dorg.hps.conditions.connection.file=local.prop -jar
>> > hps-distribution-bin.jar [args]
>> >
>> > it looks like the db redirection to my local file is ignored - the
>> > default jlab db is always searched for, and there is no
>> complain if I
>> > provide a non-existent .prop filename, as if the -D
>> instruction was neglected.
>> > Has anything changed (class names, etc) as of lately?
>> > or is the issue related to my local db copy?
>> > thanks, cheers
>> > Alessandra
>> >
>> >
>> ######################################################################
>> > ##
>> > Use REPLY-ALL to reply to list
>> >
>> > To unsubscribe from the HPS-SOFTWARE list, click the
>> following
>> link:
>> >
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>> <https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1>
>> >
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>>
>> <https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=
>> 1>
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>>
>> <https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=
>> 1>
>>
>>
>>
>>
>> ________________________________
>>
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the HPS-SOFTWARE list, click the following link:
>>
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
>>
>>
>>
>>
>>
>>
>
> ######################################################################
> ##
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the HPS-SOFTWARE list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the HPS-SOFTWARE list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1
