Hi Fabrice,
I can make separate option for unzipped file location but I do not
think it has to be a required option. There will be cases when people
load uncompressed data and I don't want them to specify options
that are not used. Probably we can require it only if there are
compressed files given as input. Can you open a ticket for it?
Cheers,
Andy
Fabrice Jammes wrote on 2014-12-19:
> Hi Andy,
>
> Maybe chunkDir could be renamed to workDir as it is also used to unzip
> input data for non partitioned table?
> If yes, it's documentation could also be updated.
>
> Indeed I first thought this directory wasn't used for non partitioned
> table so I didn't specified it, and its default value was used.
> Maybe we could also remove its default value (loader_chunks/ in the
> execution directory) and force the user to specify
> it? Indeed for now it creates this directory in the execution directory,
> this is dangerous in case of large amount of data.
> Imagine a sysadmin/DBA which would launch it on / and fill it.
>
> Cheers,
>
> Fabrice
>
> On 12/19/2014 03:19 PM, Fabrice Jammes wrote:
>> Hi Andy,
>>
>> partitioner derails on w13 input data, that's why I would need a
>> --keep-input-data option in data-loader, in order to easily test the
>> partitioner cmd-line.
>> This is not straightforward to add because input data is unzipped in
>> chunkDir, so if --keep-chunks option isn't specified input data will
>> be deleted.
>> It is of course also deleted if --keep-chunks is specified.
>>
>> For now I'll comment the code which remove this directory, but could
>> you please provide me this option?
>>
>> Thanks,
>>
>> Fabrice
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the QSERV-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
|