 |
 |
Ideally, we would rebuild it upon any changes to the dirTable. There was code in indexing.py(I think?) that was a placeholder attempt of code that could generate it. There are a couple ways of generating it: 1) from the range of numbers defined by the min and max chunk number, filter out chunks determined to be non-empty by the existence of dirtable_NNN tables. This is what my scripts did, and it was a hassle to get it to work. 2) Do a special all-chunks query (count(*) or similar), but don't squash on errors-- add them to the empty chunks list. 3) Populate the empty chunks file when you create a database, or when a czar becomes aware that a database exists. Every time you load data, you know what chunks you are creating, so remove those chunks from the empty chunks file/list. The czar always checks the db entry in css to see if its empty chunks file is out of date. It is impossible to delete partitioned rows, so chunks never become empty after being non-empty. 4) Compute a non-empty chunk list from the sec-index list: select distinct chunkId from blah (will go obsolete soon) and create a hash-table or std::map, and use it. The czar can compute and cache this the first time the db is accessed. I know, it's annoying. -Daniel -Daniel On 05/16/2014 11:59 AM, Jacek Becla wrote: > Daniel > > Can you remind me what the plan are with regards to the > emptyChunk.txt? Will it be going away any time soon? > It can still lead to a lot of confusion: I had a fully > working environment based on PT1.1 data set, but after > I run qserv-testdata it silently overwrote the version > I had (in build/dist/etc), and as a result I started > getting an error: > > Table 'LSST.Object_1234567890' doesn't exist > > which I did not connect with bad emptyChunk.txt for > quite some time. > > Thanks, > Jacek > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the QSERV-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1