Gravitational Wave Open Science Center

Accessing Data Using CernVM-FS

Overview

It is now possible to access the larger bulk data sets from Observation Runs S5, S6, O1 and O2 using CernVM-FS. This distributed filesystem will allow you to mount the data local to your computer if you have one of the supported platforms (a number of Linux distributions and MacOS X 10.11 or beyond). Once you have installed and configured CernVM-FS, you will be able to access data from these observation runs as files in subdirectories on your computer.

IMPORTANT: These instructions require that you have superuser (sudo) privileges. If you do not, please seek assistance from your system administrator.

CernVM-FS Quick-Start Page

Instructions for installing CernVM-FS can be found on their Quick-Start webpage. Please note that for the MacOS X installation you will need to install Fuse for MacOS before proceeding with the CernVM-FS install (this is documented on the CernVM-FS Quick-Start webpage under the section dedicated to MacOS-X).

NOTE: If you would like to learn more about CernVM-FS, their main website has considerable documentation.

Install and configure CernVM-FS for GWOSC Bulk Data

Execute these steps to configure and access the GWOSC Bulk Data:

  1. If on a mac, install FUSE for macOS
  2. Install CERNVM-FS. See hints on the CERNVM-FS Quickstart Page.
  3. cd /etc/cvmfs/
  4. create and edit default.local to have this one line:
  5. CVMFS_HTTP_PROXY=DIRECT
  6. cd /etc/cvmfs/default.d/
  7. create and edit 60-osg.conf to have these lines:
  8. # /etc/cvmfs/default.d/60-osg.conf
    #
    # DO NOT EDIT THIS FILE
    # It will be replaced on upgrade. To override, edit /etc/cvmfs/default.local
    #
    CVMFS_SEND_INFO_HEADER=yes
    CVMFS_KEYS_DIR=/etc/cvmfs/keys/opensciencegrid.org
    CVMFS_USE_GEOAPI=yes
    CVMFS_CONFIG_REPOSITORY=config-osg.opensciencegrid.org
    CVMFS_CONFIG_REPO_REQUIRED=yes
    CVMFS_FALLBACK_PROXY=http://cvmfsbproxy.cern.ch:3126;http://cvmfsbproxy.fnal.gov:3126
  9. create and edit config-osg.opensciencegrid.org.conf to have single line:
  10. CVMFS_SERVER_URL=http://cvmfs-s1bnl.opensciencegrid.org:8000/cvmfs/@fqrn@;http://cvmfs-s1fnal.opensciencegrid.org:8000/cvmfs/@fqrn@;http://cvmfs-s1goc.opensciencegrid.org:8000/cvmfs/@fqrn@
  11. make this mount directory:
  12. sudo mkdir -p /cvmfs/config-osg.opensciencegrid.org
  13. mount this directory to cvmfs:
  14. sudo mount -t cvmfs config-osg.opensciencegrid.org /cvmfs/config-osg.opensciencegrid.org
  15. make this mount directory:
  16. sudo mkdir -p /cvmfs/oasis.opensciencegrid.org
  17. mount this directory to cvmfs:
  18. sudo mount -t cvmfs oasis.opensciencegrid.org /cvmfs/oasis.opensciencegrid.org
  19. make this mount directory:
  20. sudo mkdir /cvmfs/gwosc.osgstorage.org
  21. mount this directory to cvmfs:
  22. sudo mount -t cvmfs gwosc.osgstorage.org /cvmfs/gwosc.osgstorage.org
  23. to find the bulk data from observation runs, change directory to be /cvmfs/gwosc.osgstorage
  24. cd /cvmfs/gwosc.osgstorage.org/
  25. This is the top leve directory for the bulk data sets. Here you will find a README.gwosc text file and the top directory gwosc
  26. ls -ltr

    -rw-r--r--@ 1 cvmfs cvmfs 114 May 21 22:52 README.gwosc
    drwxr-xr-x@ 5 cvmfs cvmfs 4096 May 21 22:52 gwdata
  27. You can explore content from different observation runs, detectors, and GPS time epochs in the underlying subdirectories:
    ls -ltr gwdata/O1/strain.16k/hdf.v1/H1/1125122048/

    -rw-r--r--@ 1 cvmfs cvmfs 365508539 Jun 13 18:39 H-H1_LOSC_16_V1-1126072320-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 10847857 Jun 13 19:21 H-H1_LOSC_16_V1-1126137856-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 139399156 Jun 13 19:21 H-H1_LOSC_16_V1-1126146048-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 516889819 Jun 13 19:22 H-H1_LOSC_16_V1-1126158336-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517152364 Jun 13 19:22 H-H1_LOSC_16_V1-1126100992-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517187025 Jun 13 19:22 H-H1_LOSC_16_V1-1126084608-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517182930 Jun 13 19:22 H-H1_LOSC_16_V1-1126088704-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517203258 Jun 13 19:22 H-H1_LOSC_16_V1-1126092800-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 4450103 Jun 13 19:22 H-H1_LOSC_16_V1-1126117376-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 458845587 Jun 13 19:22 H-H1_LOSC_16_V1-1126154240-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 481284969 Jun 13 19:22 H-H1_LOSC_16_V1-1126121472-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 201561491 Jun 13 19:23 H-H1_LOSC_16_V1-1126113280-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 161740498 Jun 13 19:23 H-H1_LOSC_16_V1-1126125568-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517125367 Jun 13 19:23 H-H1_LOSC_16_V1-1126105088-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517193654 Jun 13 19:23 H-H1_LOSC_16_V1-1126080512-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517126977 Jun 13 19:23 H-H1_LOSC_16_V1-1126096896-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 454093443 Jun 13 19:23 H-H1_LOSC_16_V1-1126162432-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517101155 Jun 13 19:24 H-H1_LOSC_16_V1-1126109184-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 516842564 Jun 13 19:24 H-H1_LOSC_16_V1-1126150144-4096.hdf5
    -rw-r--r--@ 1 cvmfs cvmfs 517244930 Jun 13 19:24 H-H1_LOSC_16_V1-1126076416-4096.hdf5

Troubleshooting

At the bottom of the Quick-Start webpage are a set of mostlikely resolutions under "Debugging Hints". Here you will also find links to further debugging hints and support links.

Return to the Data Page to learn more about Bulk Data Releases..