> select * from mysql.general_log
| 2014-11-17 11:39:54 | qsmaster[qsmaster] @ localhost [] | 26 | 0 | Connect | qsmaster@localhost on qservResult |
| 2014-11-17 11:39:54 | qsmaster[qsmaster] @ localhost [] | 26 | 0 | Query | set autocommit=0 |
| 2014-11-17 11:39:54 | qsmaster[qsmaster] @ localhost [] | 26 | 0 | Query | CREATE TABLE IF NOT EXISTS qservResult.message_28666954 (chunkId INT, code SMALLINT, message CHAR(255), timeStamp FLOAT) ENGINE=MEMORY;LOCK TABLES qservResult.message_28666954 WRITE |
| 2014-11-17 11:39:54 | qsmaster[qsmaster] @ localhost [] | 26 | 0 | Query | UNLOCK TABLES |
| 2014-11-17 11:39:54 | qsmaster[qsmaster] @ localhost [] | 26 | 0 | Quit
On Nov 14, 2014, at 11:29 PM, Jacek Becla <[log in to unmask]> wrote:
> Tatiana, John
>
> Can you do the following:
>
> a) turn on general query log for mysqld by adding
>
> general_log=1
> log-output=TABLE
>
> to /etc/my.cnf under [mysqld]
> and restarting mysql
>
> b) re-run whatever you are typically running that
> is having problems via qserv (have it talk to
> the mysqld with general log enabled)
>
> c) send us the output from
> select * from mysql.general_log
>
> I will try to re-play it on my local qserv instance
> to reproduce things.
>
> Thanks,
> Jacek
>
>
>
> On 11/14/2014 09:01 PM, Jacek Becla wrote:
>> Nope! It is not the "set" query. I run it by hand,
>> and I got:
>>
>>
>> SET @@session.autocommit = 1;
>>
>> +------------------------------------------+
>> | no-field-name |
>> +------------------------------------------+
>> | Ignoring meaningless command (in Qserv). |
>> +------------------------------------------+
>>
>> and after that Qserv was happily running.
>>
>> ---
>>
>> I don't know how deeply you depend on the query
>> that Tatiana mentioned earlier:
>>
>>
>> show columns from DeepSource from LSST;
>>
>> Because that query will consistently fail right now
>> with something like:
>>
>> ERROR 4110 (Proxy): Qserv error: 'ParseException:ANTLR parse
>> error:unexpected token: show:'
>>
>> we are not supporting "show columns" commands.
>>
>> We are working on code that will make implementing this
>> command easier. Adding support for this is in a quick
>> and dirty way right now would be non-trivial.
>>
>> So, Tatiana / Trey, let us know if you can get away
>> without "show columns" queries in the near term.
>>
>> Jacek
>>
>>
>>
>>
>>
>> On 11/12/2014 02:55 PM, Daniel L. Wang wrote:
>>> Hi John,
>>>
>>> I'm betting that the problem occurs here:
>>>
>>> -> 1291 self._execute_query("SET @@session.autocommit =
>>> {0}".format(switch))
>>>
>>> "SET..." is not supported right now.
>>>
>>> Qserv doesn't really handle much other than SELECT statements. For the
>>> longest time, we were focused on handling those well. Though they aren't
>>> perfect, we think qserv handles them well enough to support some
>>> practical usage now. One area that has gotten very little attention is
>>> in supporting things other than SQL SELECT queries. There is some
>>> short-circuit code that handles the few extra commands that the
>>> command-line "mysql" binary sends without user intervention, because
>>> that's what we test with.
>>>
>>> I think in this case, the best idea might be to save the proxy logs from
>>> connecting with the JDBC driver and with the mysql-connector. None of us
>>> (AFAIK) have tested with those, but we could see putting in some more
>>> short-circuit code to triage the "extracurricular" queries they send (we
>>> can put in some semi-reasonable canned responses), and that could very
>>> well get things working. We haven't set aside any space in our Nov
>>> sprint to work on this, but we might be able to squeeze it in, deferring
>>> existing tickets (Jacek will have to decide).
>>>
>>> In the meantime, you might be able to workaround this by calling the
>>> mysql binary in a shell process and then processing the output. In
>>> python, this wouldn't be too bad, and I might claim that the code is
>>> only incrementally more complex than using the API, and the performance
>>> (in practice) is mostly indistinguishable for, say, up to 10k or 100k
>>> result rows.
>>>
>>> FYI, qserv internally uses mysqldb when connecting to mysqld through
>>> Python, and the raw mysql C-API when connecting with C++.
>>>
>>> Hope this helps,
>>> -Daniel
>>>
>>>
>>> On 11/11/2014 08:27 PM, John Rector wrote:
>>>> Hi Daniel,
>>>>
>>>> This is a follow-up on my earlier note about trouble accessing Qserv
>>>> with JDBC. I tried a python module as well. (Results are not in the
>>>> logs I sent you, but I can send them if it's helpful.)
>>>>
>>>> ~> pip list | grep mysql
>>>> *mysql-connector-python (2.0.2)*
>>>>
>>>> It worked when I tried to connect to a standard MySQL db, but not when
>>>> I tried to connect to qserv. Immediately following the failure, I cut
>>>> and pasted the qserv connection values I had used into a /mysql/
>>>> command in the same terminal session. That connect to the LSST db.
>>>> Below is the output from the Python connection attempt.
>>>>
>>>> BTW: is there a preferred Python module to use with qserv? I have a
>>>> few, but tried only one tonight—the one from the MySQL site. One more
>>>> question, has anyone tried to fill a Pandas data frame directly from a
>>>> qserv query?
>>>>
>>>> My ipython session:
>>>>
>>>> In [1]: *import mysql.connector*
>>>>
>>>> In [2]:*cnx = mysql.connector.connect(user='qsmaster',
>>>> host='lsst-db1.ipac.caltech.edu <http://lsst-db1.ipac.caltech.edu>',
>>>> database='LSST', port=4040)*
>>>>
>>>> ---------------------------------------------------------------------------
>>>>
>>>> InternalError Traceback (most recent call
>>>> last)
>>>> <ipython-input-2-15e0754e8f3a> in <module>()
>>>> ----> 1 cnx = mysql.connector.connect(user='qsmaster',
>>>> host='lsst-db1.ipac.caltech.edu <http://lsst-db1.ipac.caltech.edu>',
>>>> database='LSST', port=4040)
>>>>
>>>> /Users/johnrector/anaconda/lib/python2.7/site-packages/mysql/connector/__init__.pyc
>>>>
>>>> in connect(*args, **kwargs)
>>>> 157
>>>> 158 # Regular connection
>>>> --> 159 return MySQLConnection(*args, **kwargs)
>>>> 160 Connect = connect # pylint: disable=C0103
>>>> 161
>>>>
>>>> /Users/johnrector/anaconda/lib/python2.7/site-packages/mysql/connector/connection.pyc
>>>>
>>>> in __init__(self, *args, **kwargs)
>>>> 127
>>>> 128 if len(kwargs) > 0:
>>>> --> 129 self.connect(**kwargs)
>>>> 130
>>>> 131 def _get_self(self):
>>>>
>>>> /Users/johnrector/anaconda/lib/python2.7/site-packages/mysql/connector/connection.pyc
>>>>
>>>> in connect(self, **kwargs)
>>>> 453 self.disconnect()
>>>> 454 self._open_connection()
>>>> --> 455 self._post_connection()
>>>> 456
>>>> 457 def shutdown(self):
>>>>
>>>> /Users/johnrector/anaconda/lib/python2.7/site-packages/mysql/connector/connection.pyc
>>>>
>>>> in _post_connection(self)
>>>> 433 """
>>>> 434 self.set_charset_collation(self._charset_id)
>>>> --> 435 self.autocommit = self._autocommit
>>>> 436 if self._time_zone:
>>>> 437 self.time_zone = self._time_zone
>>>>
>>>> /Users/johnrector/anaconda/lib/python2.7/site-packages/mysql/connector/connection.pyc
>>>>
>>>> in set_autocommit(self, value)
>>>> 1289 """Toggle autocommit"""
>>>> 1290 switch = 'ON' if value else 'OFF'
>>>> -> 1291 self._execute_query("SET @@session.autocommit =
>>>> {0}".format(switch))
>>>> 1292 self._autocommit = value
>>>> 1293
>>>>
>>>> /Users/johnrector/anaconda/lib/python2.7/site-packages/mysql/connector/connection.pyc
>>>>
>>>> in _execute_query(self, query)
>>>> 1495 """
>>>> 1496 if self._unread_result is True:
>>>> -> 1497 raise errors.InternalError("Unread result found.")
>>>> 1498
>>>> 1499 self.cmd_query(query)
>>>>
>>>> InternalError: Unread result found.
>>>>
>>>> ______________________________
>>>> John Rector
>>>> IPAC, California Institute of Technology
>>>> MS 100-22
>>>> 770 S. Wilson Ave.
>>>> Pasadena, CA 91125
>>>> USA
>>>> Tel: (626) 395—8577
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the QSERV-L list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>>
>>
>
>
>
> --
> BEGIN-ANTISPAM-VOTING-LINKS
> ------------------------------------------------------
>
> Teach CanIt if this mail (ID 04NfHvOH3) is spam:
> Spam: https://canit.ipac.caltech.edu/canit/b.php?i=04NfHvOH3&m=f032d505f023&c=s
> Not spam: https://canit.ipac.caltech.edu/canit/b.php?i=04NfHvOH3&m=f032d505f023&c=n
> Forget vote: https://canit.ipac.caltech.edu/canit/b.php?i=04NfHvOH3&m=f032d505f023&c=f
> ------------------------------------------------------
> END-ANTISPAM-VOTING-LINKS
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
