Hi all,

From my knowledge, there is a serious limitation when using platforms as Grid5000. Mainly, that the data is not resident on the platform, and the disk space gets recycled once the job is over. In practice, it means that we may have to reload the data each time we perform a test the Grid5000 for test, which is highly incovenient.
Obviously, if it were possible to overcome this limitation, and having some resident use of the memory (for 10's of TB !!) it would make Grid5000 very valuable indeed. 

The presence of this limitation is one of the reasons for which there is the development of platforms dedicated to experimentation on Big Data, which would allow for a resident data set to be stored in the system and to be shared by many users for their experimentations. The DELL cluster at CCIN2P3 as well as the Galactica platform in Auvergne are precisely tailored to meet this requirement, which makes them useful for the experimentation.

Now, I think it would be really interesting to know if Grid5000 can be used in a Data-orinted way. Perhaps more discussion at the technical level should be scheduled with Lucas, and including our research experts on this field.

Cheers,

Emmanuel

Le 15 déc. 2015 à 11:15, Fabrice Jammes a écrit :

> Hi,
> 
> Lucas Nussbaum, from french research center in computer science INRIA, propose us to use their large Cloud Computing infrastructure: Grid5000 to test Qserv.
> We could install here our own OS on bare-metal+Docker, and then launch integration tests.
> 
> Lucas also told me that; in a second step, it would be possible to use the cluster for Large Scale Tests with our 35TB dataset on clusters up to 144 nodes.
> This could be very useful to debug issues met at CC-IN2P3, and in the future on our production platforms.
> 
> Could we please speak about that to our tomorrow meeting?
> 
> Grid5000 website: https://www.grid5000.fr/mediawiki/index.php/Grid5000:Home
> 
> Regards,
> 
> Fabrice
> 
> 
> -------- Forwarded Message --------
> Subject:	LSST sur Grid5000 ?
> Date:	Mon, 14 Dec 2015 18:02:50 +0100
> From:	Lucas Nussbaum <[log in to unmask]>
> To:	[log in to unmask]
> 
> Bonjour Fabrice,
> 
> j'ai vu ta présentation aux JRES, et je me disais que ça serait
> intéressant de jouer avec l'idée d'utiliser Grid'5000 pour
> tester/évaluer votre code.
> 
> Concrètement, vous pourriez y réserver des ressources (des machines
> physiques) et y déployer toute votre pile logicielle. Et donc bénéficier
> de tout le support de Grid'5000 pour l'expérimentation afin, par
> exemple, de rester des évolutions.
> 
> A priori, je ne vois pas de gros verrous techniques qui empêcheraient
> d'y arriver.
> 
> Est-ce que c'est qqchose qui pourrait t'intéresser ?
> 
> Si tu ne connais pas bien Grid'5000, tu peux regarder ces slides
> présentés aux journées SUCCES:
> 
> http://succes2015.sciencesconf.org/conference/succes2015/grid5000.pdf
> 
> 
> A+
> -- 
> | Lucas Nussbaum      Assistant professor @ Univ. de Lorraine |
> | 
> [log in to unmask]
>                      LORIA / MADYNES |
> | 
> http://www.loria.fr/~lnussbau/
>             +33 3 54 95 86 19 |
> 
> 
> 

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1