Hi Julia,
The problem is that it would require significant architectutal changes,
not to mention a whole new set of documentation. We are taking a different
track to address this problem that we think is a more general solution
that should address a wider range of needs.
We already have two collectors that know how to piece the information
together to essentially provide what you are requesting. The problem is
that if one of the required UDP packets is lost it makes it impossible
to provide complete information. The plan is to provide a local UDP
collector where the likelihood of packet loss is minimal and convert
the stream to a MQ stream (e.g. rabbitMQ). That would minimize
information loss. We have one prototype and our collaborators are working
on some others.
I don't have an exact timeline but given your request I will try to
establish one that is hopefully timely. This particular effort is
community supported so we need to take that into account. However, we do
have OSG and two other collaborative partners participating so it
shouldn't take too long to provide a solution. I hope that this addresses
your request.
Andy
On Thu, 24 Jun 2021, Julia Andreeva wrote:
> Dear colleagues,
>
> We are discussing various options to improve the situation with xrootd monitoring on the WLCG infrastructure.
> One of the options we consider is to try ALICE approach which consists of processing only file close reports. This looks to be efficient regarding optimization of the amount of data volume to be processed by the collector. However, ALICE uses dedicated xrootd servers, while for monitoring of the xrootd traffic on the WLCG infrastructure, we need to know the owner of the operation in terms of the VO.
>
> Two other things which would be useful to have in the file close report is the start time stamp of the operation and application level meta info which is now possible to provide with the monitoring reports, but if I am not wrong , it is send in the beginning.
>
> So our question is, would it be possible to extend the file close report with information about: VO, start time stamp of the operation and application level meta info.
>
> Please, let us know.
>
> Thank you
>
> Kind regards
> Julia
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
|