 |
 |
Hi Fabrice Looks great! If you have access to this machine for longer, can you try the last test with 2 GB, 16 GB and 48 GB sort buffer size? I see one issue: why are you torturing our poor qserv repo so much? It looks like you have a branch u/fjammes/mysqlLargeTables for that, but I can't get to your code by checking out your branch, and I had to do: git checkout 0b84d416e95a which is in "detached from HEAD" state. Please create directory "mysqlLargeTable" directory in dbutils and move your scripts there (git clone [log in to unmask]:/contrib/dbutils.git dbutils to get there) thanks Jacek On 05/28/2013 07:56 AM, Fabrice Jammes wrote: > Hello Jacek, > > I've done and documented the tests run with mariadb, myisamchk, and > RunDeepForcedSource of Winter2013. > It's here : > https://dev.lsstcorp.org/trac/wiki/mysqlLargeTablesAtIn2p3 > > Feel free to ask for any additional information. > > Fabrice > > On 05/07/2013 09:51 PM, Jacek Becla wrote: >> Fabrice >> >>> Of course we can have a look a it. >> >> That is great! >> >>> Is there additional instructions for b) point ? If yes, could you send >>> it to us ? >> >> 1) Have a RunDeepForcedSource table ready. It should not be >> a production database, because we will be doing bad things >> to it. >> >> 2) Disable indexes (alter table RunDeepForcedSource disable index, >> than ) >> >> 3) do not use that table through mysql while doing everything >> below. Either shut down mysqld, or run "flush cache" and >> don't touch that table >> >> 4) then test myisamchk, note that this is command line tool >> >> time /usr/local/mysql/bin/myisamchk >> -v -t <tmp_dir> >> --force >> -o >> --update-state >> -p >> --key_buffer_size=8M >> --sort_buffer_size= <~1/2 of your RAM available> >> --read_buffer_size=16M >> --write_buffer_size=16M >> <mysql data dir>/<db>/RunDeepForcedSource.MYI >> >> >> that will most likely fail, see >> https://dev.lsstcorp.org/trac/wiki/mysqlLargeTables >> >> Then apply the patch and retry, Monty mentioned some >> extra switches (--force --force, etc), see his email, >> and try them >> >> Please capture output (and the timing!) for each >> thing you run. >> >> it is probably better to use a machine that has >> reasonable amount of memory (say 32 or 64G), not >> insanely large amount of memory (on a machine with >> 256G ram I couldn't reproduce this problem). >> >> It is important that the <tmp dir> has enough space, >> at least the size of one copy of RunDeepForcedSource.* >> files. >> >> thanks >> Jacek >> > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1