Oscar's notes on EOS storage

This describes the basic commands to use EOS storage at CERN. It should be noted that they are doing some changes on the way EOS is used because it may be come "the new AFS". For example, EOS is now mounted in /eos on the lxplus cluster (and on the batch?) and a similar thing can be done in any Desktop at CERN, according to the informations.

On the other hand, the preferred way to access EOS is using XRootD, whose main tools are described in this other page since it usage is much more generic than just EOS. In this document we just describe the specific ways that allow to access EOS.

As seen in the section describing the iteration with the Grid (here), the Grid-oriented tools are also available for interacting with EOS, using the proper syntax and server.

EOS and XRootD In order to access EOS for the CMS area using XRootD one has to use the eoscms.cern.ch, like in the examples:

         xrdfs eoscms.cern.ch ls /store/group

         xrdcp root://eoscms.cern.ch//store/group/... localfile

where it should be noted that the directory starts with /store (which is the standard in the CMS areas) but it actually corresponds to /eos/cm/store/. the use of the full path is also supported.

In case we want to access other areas, the correct area must be indicated, since if not the permission is not granted

Access to files using the eos command

Although the XRootD is the currently preferred way and the future is to access it directly via de filesystem (see below), there is still some commands that are used to handle the files and directories under EOS.

Of course, these are not standard with other grid-related tools. And it is likely some just work within CERN.

To list the content of a directory (with extended version) in EOS:

         eos ls [-l] /eos/cms/store/user/oglez

One can use the command help to retrieve the full list of possibilities to interact with EOS.
It should be noted that the area above is actually the CMS-standard to identified the local area for users in the Grid places. I use that area as the equivalent at CERN of my area at T2_ES_CIEMAT.
Additionally there is an area (for the group cafcern) which I identified as the variable $EOSCAF_HOME. Also we have the area for the CERNBox which is also in EOS (it seems to be /eos/user/o/oglez).

To create a directory:

        eos eoscms mkdir /eos/cms/store/user/oglez/test

It should be noted that the eoscms part is not needed in practice since my default is to go to the CMS server (it is set at loading the CMS config).

To set the permissions:
In case one needs to set the permissions to allow copy from a group, which does not seem to be the case, that can be done using:

        eos chmod -r 775 /eos/cms/store/user/oglez/test

To copy a file to EOS:

        eos cp test.txt /eos/cms/store/user/oglez/

To copy a file from EOS:

        eos cp /eos/cms/store/user/oglez/test.txt .

To check the size of a directory or a file from EOS:

       eos find --size /store/user/oglez/test.txt
       eos find --size /eos/cms/store/user/oglez | awk -F'=' '{size+=$3}END{print size/1024/1024/1024/1024" TB"}'

To delete a file or a directory in EOS:

        eos rm [-r] /eos/cms/store/user/oglez/test.txt
        eos rmdir /eos/cms/store/user/oglez/test

To check whether a file is online (and other information)

        xrd eoscms fileinfo /eos/cms/store/user/oglez/test.txt

To check the quota:

        eos quota
        eos quota | grep -A 4 "Quota Node: /eos/cms/store/user/" | head -5
        eos quota | less    [and look for "oglez" there]

Regarding the quota it seems that it is assigned to the user, independently of the directory and owner. i.e. it is global, although one can write in several subareas (which makes the output hard to process)

Mounting EOS on a CERN Desktop

Esto me causó estragos con un log enorme y problemas de llenado de áreas... no usar en mis ordenadores, tal vez en lxplus... y pensarlo muy mucho.

It is possible to mount the EOS area in a CERN Desktop by issuing commands like the following:

     mkdir $HOME/eos
     eosmount $HOME/eos

and then use it as any other directory. For example my user area related to CAF becomes $HOME$EOSCAF_HOME. It is strongly recommended to use it only for a good reason and try to unmount the area as soon as it is done! In fact it gives a lot of troubles with the expiration of the AFS tokens.
To unmount it:

     eosumount $HOME/eos

On the other hand it is possible to mount it outside the AFS area and it should give no troubles, although in the current approach it seems they prefer to have centralized mounted EOS areas. I guess it behaves better or more efficiently. Those areas are in /eos for lxplus although it should be made available in any Desktop at CERN.
In the mounted area all the commands that work for the usual Unix filesystems are available and work as expected. Of course, as in the case of AFS, the specific information about the EOS system needs to be retrieved with an EOS command.

Comments about the web page: Oscar González