Hi Pail,
And to add additional Informat, Ilija has also done this. Here is what he
has to say....
------------------
Yes I did it long time ago. I had a python script that was receiving
summary stream, repackaging it into ES documents and sending them to
indexing. These days I only collect pfc stream. It gets sent to our
logstash and from there to ES.
If needed I can try to dig out that summary receiving code, I think CMS
people use it even today for stashcache monitoring.
------------------
So, there appear to be two options here.
Andy
On Thu, 25 Mar 2021, Robert Currie wrote:
> Hi Paul,
>
> We've recently done this in Edinburgh as part of GridPP to support the UK
> XCache efforts (viewable results here:
> https://gridpp.monitoring.edi.scotgrid.ac.uk/app/dashboards#/view/52a44100-75ec-11eb-9e06-fbd4e6c2af73)
>
> We also gave short presentation on this to the UK community here
> https://drive.google.com/drive/folders/1PywvuULoFtWVHTtumX4kajks3a8Ng5bR and
> have submitted a vCHEP'21 abstract/writeup on the work done.
>
> Short answer for how we did it was to locally fork an existing Python3 based
> collector and then to build a message-queue ingestion system based on a
> cluster of docker containers to ingest this into our main elasticsearch
> instance.
> It's not yet perfect, but this works well enough for us to monitor both our
> transparent XRrootD proxy instance (and those at remote sites) as well as the
> XRootD components of our DPM-SE in realtime.
>
> Happy to give some pointers if you're interested on working on something like
> this if we can help avoid duplication of effort.
>
> Rob
>
> On 2021-03-25 08:43, Paul MUSSET wrote:
>> Hello,
>>
>> Has anyone put the XRootD summary monitoring (obtained via xrd.report
>> and mpxstats) into elasticsearch ? If yes, can you tell me how you did
>> it ?
>>
>> Cheers
>> Paul
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
|