 |
 |
Hi Matevz, Probably good enough. I haven't had a cance to follow up on this. I do see that you have the sched option so the load would be taken into account and the less loaded servers would be selected. On the other hand, I see that you specified 0 for each of the weights so load factoring would be effectively disabled. Was this the actual setting when you did the test? Andy On Fri, 7 Mar 2014, Matevz Tadel wrote: > Hi Andy, > > Is this good enough or I should prepare something else? > > Matevz > > On 02/27/14 10:49, Matevz Tadel wrote: >> Hi Andy, >> >> I had "cms.trace all" all along. >> >> This is the extract of redirects: >> http://uaf-2.t2.ucsd.edu/~matevz/tmp/cmsd-redirect.txt >> >> The full log: >> http://uaf-2.t2.ucsd.edu/~matevz/tmp/cmsd.log >> >> And a sortable table of a set of ~200 files opened with 1 second interval: >> http://uaf-2.t2.ucsd.edu/~matevz/tmp/ucsd-openfiles.html >> - you can sort it by open time (similar to redirect extract); >> - or by server name to see the distribution over servers. >> >> Our servers are uaf-[3-9], cabinet-8-8-[0-8], cabinet-8-8-[10-13]. >> >> You'll see that cabinet 0, 2, 3, 7, 8 and 10 do not get selected at all in >> this >> 200 file test and that uaf-4, 5 and 9 are only selected 2 or 3 times. I >> checked >> there is no weirdness on xrootd / cmsd logs on the under provisioned nodes >> (and >> that I can talk to them directly). >> >> Ah, just noticed ... the cabinet nodes that don't get selected do have a >> higher >> load & cpu usage and the ones that do are not doing anything (which is >> really >> unusual, that's why I didn't even check it at first). So my cms.sched >> settings >> seem to get ignored! >> >> The full config, redirector is xrootd.t2.ucsd.edu: >> http://uaf-2.t2.ucsd.edu/~matevz/tmp/xrootd.cfg >> >> Matevz >> >> On 02/27/14 01:05, Andrew Hanushevsky wrote: >>> Hi Matevz, >>> >>> The only way to find out is to turn on redirect debugging in the cmsd for >>> a >>> while and see what the decisions were. We can go from there once we have a >>> timeline. >>> >>> Andy >>> >>> On Wed, 26 Feb 2014, Matevz Tadel wrote: >>> >>>> On 02/26/14 09:22, Matevz Tadel wrote: >>>>> Hi, >>>>> >>>>> We have ~20 of xrootd servers at UCSD, all of them do something else, >>>>> too, and >>>>> are thus under different load. This led to practically all requests >>>>> going to a >>>>> few servers only so I set cms.sched to do round-robin. But this does't >>>>> help >>>>> much, the open requests are still mostly sent to the same few servers. >>>>> >>>>> Could it be that "cms.dfs lookup distrib" causes the redirector to send >>>>> the >>>>> client to the "fastest to respond" server instead of decoupling verify >>>>> and >>>>> redirect steps? >>>> >>>> OK, that wasn't it ... I got hdfs configured on our redirector and tried >>>> lookup central but it didn't change anything. >>>> >>>> What could cause the redirector to only redirect to a few servers? I have >>>> this >>>> now ... so it should be pure round-robin, right? >>>> cms.sched cpu 0 io 0 mem 0 pag 0 runq 0 space 0 fuzz 100 refreset >>>> 3600 >>>> >>>> >>>> Matevz >>>> >>>> ######################################################################## >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>> >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-DEV list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-DEV list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1