I said:
On Sep 4, 2013, at 2:45 PM, Douglas Smith wrote:
But I can't seem to join with the Object table yet, we knew this
I guess already:
mysql> SELECT s.ra, s.decl FROM Source s, Object o WHERE s.objectId=o.objectId and o.objectId = 154062577146538502;
Empty set (2 min 17.98 sec)
Not sure what's going on here, but I'll take a look after some more debugging.
Some other notes:
- I added LSST__Source (a copy of LSST__Object) to qservMeta. I probably could have soft-linked it to LSST__Object.*, but for now I just made a copy off the objectId index table.
- The fact that the qserv master doesn't recognize Source.objectId as special and look up a chunk for it in the objectId index table seems like a bug.
So, it turns out that the current qserv master code sub-chunks the above join (Daniel - is that expected?). In particular, I see stuff like this in the worker logs on ccqserv27 for chunk 12833, which contains the objectId in question:
Runner running Task: msg: session=19 chunk=12833 db=LSST frag: q=SELECT s.ra,s.decl FROM LSST.Source_12833 AS s,Subchunks_LSST_12833.Object_12833_0 AS o WHERE o.objectId=154062577146538502 AND s.objectId=o.objectId,...
But I also see the same for a bunch of different chunks, and the same query shows up on other workers. This is what I would call a missed opportunity by the master-side query generation logic. But the only ill effect is that execution isn't as fast as it could be.
Anyway, more to the point, look at what happens if I run the following on ccqserv027:
mysql> SELECT chunkId,subChunkId from LSST.Object_12833 AS o WHERE o.objectId=154062577146538502;
+---------+------------+
| chunkId | subChunkId |
+---------+------------+
| NULL | NULL |
+---------+------------+
1 row in set (0.00 sec)
In fact, if I run this query through qserv:
mysql> select count(*) from Object where subChunkId is not null;
I get back 0.
If I understand things correctly, this means that the Object sub-chunks created on the fly by qserv workers will always be empty, so that Source/Object join and Object near-neighbor queries will always come back with nothing on the IN2P3 cluster. Something must have gone seriously wrong with duplication or partitioning of the Object table.