LISTSERV mailing list manager LISTSERV 16.5

Help for QSERV-L Archives


QSERV-L Archives

QSERV-L Archives


QSERV-L@LISTSERV.SLAC.STANFORD.EDU


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

QSERV-L Home

QSERV-L Home

QSERV-L  May 2013

QSERV-L May 2013

Subject:

Re: Fwd: Updated myisamchk to handle LARGE files

From:

Fabrice Jammes <[log in to unmask]>

Reply-To:

General discussion for qserv (LSST prototype baseline catalog)

Date:

Tue, 7 May 2013 09:07:04 +0200

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (106 lines)

Hello Jacek,

Of course we can have a look a it.
Is there additional instructions for b) point ? If yes, could you send 
it to us ?

Thanks,

Fabrice

On 05/07/2013 02:14 AM, Jacek Becla wrote:
> To IN2P3 team:
>
> Do you have any spare time to try a patch that mysql team
> implemented for us? It is related to rebuilding index for
> a very large table. To test it, you'd need to
>
> a) install mysql 5.5.30 (mariadb distribution)
>
> b) have a full W13 RunForcedDeepSource table ready
>    (3+ billion rows) and try rebuilding an index,
>    it will likely fail as described at
>    https://dev.lsstcorp.org/trac/wiki/mysqlLargeTables#Enableindexes
>    I can send you the instructions.
>
> c) apply the patch and try again
>
> It'd be useful to try it for several reasons (useful for Summer2013,
> Monty is the creator of mysql and I am trying to initiate
> useful collaboration), but I am totally swamped, and on vacation all 
> next week.
>
>
> Thanks
> Jacek
>
>
>
>
>
> -------- Original Message --------
> Subject: Updated myisamchk to handle LARGE files
> Date: Mon, 6 May 2013 16:51:11 -0700
> From: Michael Widenius <[log in to unmask]>
> Reply-To: [log in to unmask] <[log in to unmask]>
> To: Becla, Jacek <[log in to unmask]>
> CC: [log in to unmask] 
> <[log in to unmask]>
>
>
> Hi!
>
> Here is a patch that should fix your myisamchk issues.
>
> This is to be applied to MariaDB 5.5.30 source.
>
> Can you verify that this works so I can add it to MariaDB 10.0 ?
>
> Some notes:
>
> - If you run repair with two --force options, like this:
>
> myisamchk --repair --force --force ...
>
> Then myisamchk will use the full myisam_sort_buffer_size for keys. Other
> buffers will be allocated as needed.
>
> This means that with this option one will never get the
> "myisam_sort_buffer_size is too small" error.
>
> I also added the option --create-missing-keys, which will create keys
> just for any disabled keys.
>
> Other things:
> - Better error message for 'myisam_sort_buffer_size is too small' errors.
> - Somewhat larger buffers as default.
> - Changed a lot of variables from uint -> ulonglong to try to ensure 
> that big
>   files will not cause any problems.
> - Removed some checks that limited the sort_buffer_size to 4G.
> - Fixed code that would cause problems if sort_buffer_size would be > 8G
> - Fixed printing of longlong variables.
> - Fixed possible core dump if 'alloc_dynamic()' would fail.
> - Fixed some possible problems on system where uint would be <= ulong.
>
> Hope this helps!
>
> Regards,
> Monty
>
>
>
>
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

March 2018
February 2018
January 2018
December 2017
August 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012

ATOM RSS1 RSS2



LISTSERV.SLAC.STANFORD.EDU

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager

Privacy Notice, Security Notice and Terms of Use