 |
 |
Hello Fabrizio Thanks, I will try it out right now. The timeout will become an issue. The time cksum needs is mostly IO-bound (I guess). Right now with an unused disk and no other write/read activity you get about 60MB/s so the file you tested (~1.6GB) took about 27 secs. However, ones the machines are used the IO-rate (cksum gets) will drop significantly (few MB/s) and the checksum for such a large file will take couple of minutes. The pre-merge collections for skimming are small (tens of MB ?), so it might finish before the timeout. How can one change the timeout parameter ? Also, the checksum request is not issued to a redirector but to the data server directly, which just means the client will request the checksum from the same server many times and therefore slow down the machine even more. Wilko On Thu, 18 Nov 2004, Fabrizio Furano wrote: > Hi Wilko, > > the chksumming seems ok now to me. You can try it by running > tinytestXTNetAdmin.pl. > > A little consideration: I noted that the checksumming takes a long > amount of time (about 10 secs in this small test). If the time needed > for computing the checksum is greater than the XrdClient timeout (30 or > 60 secs), the client will initiate its retry operations because it will > think that the server is dead. The checksum will be asked to the > redirector or to another server, which eventually will timeout too... a > loop which will fail after a number of retries. > > If this problem will one day show up, I believe that the correct way > to fix it could be to use the kxr_wait mechanism. No changes at the > client side would be needed at all. > > However, please let me know if it works for you. I'll read the mail > tomorrow, then I'll be out for a few days. > > Fabrizio > > Wilko Kroeger wrote: > > Hello Fabrizio > > > > Checksumming on bbrprod01 should work. What is the file name you pass to > > GetChecksum ? It should be > > /prod/store/PRskims/R14/16.0.1a/AllEvents/23/AllEvents_2301.01.root > > > > Cheers, > > Wilko > > > > > > On Thu, 18 Nov 2004, Fabrizio Furano wrote: > > > > > >>Hi Wilko, > >> > >> I changed that function in order to handle any checksum size. > >>Unfortunately I am not able to find a server/file giving me any checksum. > >> > >>This is what I get from bbrprod01: > >> > >> 041118 06:15:44 001 Xrd: SendGenCommand Server declared error > >>3010:Check summing path > >>'/kanga/prod/store/PRskims/R14/16.0.1a/AllEvents/23/AllEvents_2301.01.root' > >>is disallowed. > >> > >>Please, can you tell me something I can checksum somewhere? > >> > >>I will commit the change when I am able to test it at least once. > >>Differently from the former version, which I committed without any > >>available server doing checksums at the time... > >> > >>Fabrizio > >> > >> > >>Wilko Kroeger wrote: > >> > >>>Hello Fabrizio > >>> > >>>I tried the GetChecksum perl interface (on linux) with the > >>>latest version. It kind of works but produces a core after > >>>calling. > >>>The problem is in the usage of the shared buffer (char *sharedbuf) > >>>in XrdClientAdmin_c.cc. > >>> > >>>XrdGetChecksum() uses this buffer to hold the checksum returned from the > >>>server (AFAIK). I think the core is produced if the buffer is to small. > >>>I changed in XrdGetChecksum() > >>> > >>>from: > >>>memset(sharedbuf, 0, sizeof(sharedbuf)); > >>>adminst->GetChecksum((kXR_char *)path, (kXR_char *)sharedbuf); > >>> > >>>to: > >>>SharedBufRealloc(30); > >>>memset(sharedbuf, 0, 30); > >>>adminst->GetChecksum((kXR_char *)path, (kXR_char *)sharedbuf); > >>> > >>>and then it works fine. Is there a better way to allocate the buffer? > >>>I just choose a size (=30) that is safe for our purpose but it isn't > >>>guaranteed that it will work if a different checksum algorithm is used > >>>(Also not only the checksum but also a string is returned e.g.: > >>>crc32 3140090365) > >>> > >>>Cheers, > >>> Wilko > >>> > >> >