To IN2P3 team:
Do you have any spare time to try a patch that mysql team
implemented for us? It is related to rebuilding index for
a very large table. To test it, you'd need to
a) install mysql 5.5.30 (mariadb distribution)
b) have a full W13 RunForcedDeepSource table ready
(3+ billion rows) and try rebuilding an index,
it will likely fail as described at
https://dev.lsstcorp.org/trac/wiki/mysqlLargeTables#Enableindexes
I can send you the instructions.
c) apply the patch and try again
It'd be useful to try it for several reasons (useful for Summer2013,
Monty is the creator of mysql and I am trying to initiate
useful collaboration), but I am totally swamped, and on vacation all
next week.
Thanks
Jacek
-------- Original Message --------
Subject: Updated myisamchk to handle LARGE files
Date: Mon, 6 May 2013 16:51:11 -0700
From: Michael Widenius <[log in to unmask]>
Reply-To: [log in to unmask] <[log in to unmask]>
To: Becla, Jacek <[log in to unmask]>
CC: [log in to unmask]
<[log in to unmask]>
Hi!
Here is a patch that should fix your myisamchk issues.
This is to be applied to MariaDB 5.5.30 source.
Can you verify that this works so I can add it to MariaDB 10.0 ?
Some notes:
- If you run repair with two --force options, like this:
myisamchk --repair --force --force ...
Then myisamchk will use the full myisam_sort_buffer_size for keys. Other
buffers will be allocated as needed.
This means that with this option one will never get the
"myisam_sort_buffer_size is too small" error.
I also added the option --create-missing-keys, which will create keys
just for any disabled keys.
Other things:
- Better error message for 'myisam_sort_buffer_size is too small' errors.
- Somewhat larger buffers as default.
- Changed a lot of variables from uint -> ulonglong to try to ensure
that big
files will not cause any problems.
- Removed some checks that limited the sort_buffer_size to 4G.
- Fixed code that would cause problems if sort_buffer_size would be > 8G
- Fixed printing of longlong variables.
- Fixed possible core dump if 'alloc_dynamic()' would fail.
- Fixed some possible problems on system where uint would be <= ulong.
Hope this helps!
Regards,
Monty
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
|