CVMFS at JLab

This page is a preview of the above content, for the full web page follow the link above.

 

What is it?

The CernVM-File System (CernVM-FS) provides a scalable, reliable and low- maintenance software distribution service. It was developed to assist High Energy Physics (HEP) collaborations to deploy software on the worldwide- distributed computing infrastructure used to run data processing applications. CernVM-FS is implemented as a POSIX read-only file system in user space (a FUSE module). Files and directories are hsted on standard web servers and mounted in the universal namespace /cvmfs.

JLab Specifics

CVMFS is preconfigured on ifarm/farm nodes. Here's the setting used for the local squid caches:

Example usage on ifarm

Use the software stored there just like you would locally installed packages.

Getting data into OASIS

Data written to the source areas listed below will be rsynced and then published to OASIS for use via CVMFS.

The individual project directories at the top-level will be rsynced within the scheduled cronjob. This requires an update to the cronjob script for new directories. Submit ServiceNow incident to request updates or changes.

 

Adding and removing files from the src directory will be reflected at roughly at the cronjob interval. Once all rsyncs are completed, if file changes were made, an update job is scheduled within the OSG.

Log files

Each project will have an associated log file. These provide a great resource to verify successful rsyncs (ex. no permissions errors). They will be rotated and purged.

Rsync Excludes

By default, each top-level project directory will be given an exclude file (oasis_exclude.txt). This file is passed to rsync's `--exclude-from` option (see the man page for details). Things added to this file after they have already been published, will be deleted.

Need help?

Submit a ServiceNow incident for help regarding adding a new project or any issues getting your files updated.