stackyter 0.12

Optional arguments

An option used on the command line will always overwrite the content of the configuration file for the same option, if it exists. See the next section for a description on how to use the configuration file. Available options are:

-h, --help            show this help message and exit
-c CONFIG, --config CONFIG
                      Name of the configuration to use, taken from your
                      default configuration file (~/.stackyter-config.yaml
                      or $STACKYTERCONFIG). Default if to use the
                      'default_config' defined in this file. The content of
                      the configuration file will be overwritten by any
                      given command line options. (default: None)
-f CONFIGFILE, --configfile CONFIGFILE
                      Configuration file containing a set of option values.
                      The content of this file will be overwritten by any
                      given command line options. Mostly for testing your
                      configuration file before making it the default.
                      (default: None)
-u USERNAME, --username USERNAME
                      Your CC-IN2P3 user name. If not given, ssh will try to
                      figure it out from you ~/.ssh/config or will use your
                      local user name. (default: None)
-H HOST, --host HOST  Name of the target host. This option may allow you to
                      avoid potential conflit with the definition of the
                      same host in your $HOME/.ssh/config, or to connect to
                      an other host than the CC-IN2P3 ones (Jupyter must
                      also be available on these hosts). Default if to
                      connect to CC-IN2P3. (default: cca7.in2p3.fr)
-w WORKDIR, --workdir WORKDIR
                      Your working directory at CC-IN2P3 (default:
                      /pbs/throng/lsst/users/<username>/notebooks)
-j JUPYTER, --jupyter JUPYTER
                      Either launch a jupiter notebook or a jupyter lab.
                      (default: notebook)
--vstack VSTACK       Version of the stack you want to set up. (E.g. v14.0,
                      w_2017_43 or w_2017_43_py2) (default: v14.0)
--packages PACKAGES   A list of packages you want to setup. Coma separated
                      from command line, or a list in the config file. You
                      can use the `lsst_distrib` package to set up all
                      available packages from a given distrib. (default:
                      lsst_distrib)
--desc                Setup a DESC environment giving you access to DESC
                      catalogs ('proto-dc2_v2.0' is for now the only
                      available catalog). This option overwrites the '--
                      vstack' and '--mysetup' options. (default: False)
--mysetup MYSETUP     Path to a setup file (at CC-IN2P3) that will be used
                      to set up the working environment. Be sure that a
                      Python installation with Jupyter (and jupyterlab) is
                      available to make this work. The LSST stack won't be
                      set up in this mode. 'vstack', 'libs', 'bins' and
                      'labpath' options will be ignored. (default: None)
--libs LIBS           Path(s) to local Python librairies. Will be added to
                      your PYTHONPATH. Coma separated to add more than one
                      paths, or a list in the config file. A default path
                      for jupyter will be choose if not given. (default:
                      None)
--bins BINS           Path(s) to local binaries. Will be added to your PATH.
                      Coma separated to add more than one paths, or a list
                      in the config file. A default path for jupyter will be
                      choose if not given. (default: None)
--labpath LABPATH     You must provide the path in which jupyterlab has been
                      installed in case it differs from the (first) path you
                      gave to the --libs option. A default path for
                      jupyterlab will be choose if not given. (default:
                      None)

Configuration file

A configuration dictionnary can contain any options available through the command line. The options found in the configuration file will always be overwritten by the command line.

The configuration file can be given in different ways, and can contains from a single configuration dictionnary to several configuration dictionnaries. The --config and --congfile options can be used (or not) in several different ways:

• stackyter.py --configfile myfile.yaml. myfile.yaml must contain your configuration, with your set of options.

• stackyter.py --config myconfig. In that case, no configuration is directly given by the user, and stakyter will look for a default configuration file. The default file must be either ~/stackyter-config.yaml or defined by the STACKYTERCONFIG environment variable (if the default path does not fit your need). The myconfig key will be looked for in this default configuration file to get the configuration dictionnary you asked for.

• stackyter.py. In that case, stackyter will also look for a default configuration file (see above), and for a default configuration called default_config in this file. This default value must be there if you have mor ethan one confiuration defined. It must point to the configuration you would like to use by default.

In principal, your default configuration file must look like that:

{
 'default_config': 'ccin2p3',
 'ccin2p3': {
             'host': 'cca7.in2p3.fr',  # or ccjupyter if your ~/.ssh/config if configured
             'jupyter': 'lab',
             'packages': ["lsst_distrib"],
             'username': 'nchotard',
             'vstack': 'v14.0',
             'workdir': '/sps/lsst/dev/nchotard/',
            },
 'othersite': {
               'host': 'otherhost.fr',
               'username': 'chotard',
               'mysetup': 'pathtomysetup'
              },
}

or simply as followed if only one configuration is defined:

{
 'ccin2p3': {
             'host': 'cca7.in2p3.fr',  # or ccjupyter if your ~/.ssh/config if configured
             'jupyter': 'lab',
             'packages': ["lsst_distrib"],
             'username': 'nchotard',
             'vstack': 'v14.0',
             'workdir': '/sps/lsst/dev/nchotard/',
            },
}

Version of the LSST stack

All available versions of the LSST stack at CC-IN2P3 can be found under:

/sps/lsst/software/lsst_distrib/

These versions (and all the others) have been built under CentOS7, and must be used under a compatible system (CentOS7 or Ubuntu). To connect to a CentOS7 machine on CC-IN2P3, use --host cca7.in2p3.fr instead of --host ccage.in2p3.fr (cca7 is the default value of this script).

Python 2 (2.7) and 3 (>3.4) are available for almost all weeklies, with the following nomencalture:

Latest releases of the LSST stack, as of 11-07-2017, are:

Keep in mind that using Python 2 in an LSST context is not encouraged by the community, and will not be supported anymore. The latest weekly for which Python 2 has been installed at CC-IN2P3 is w_2017_4 (see online documentation).

Note: Since version w_2017_40, the ipython module is included in the stack installation at CC-IN2P3 as an add-on. This module is not part of the officiel LSST distribution and will not be set up with the lsst_distrib package.

Use the LSST stack

Many examples on how to use the LSST stack and how to work with its outputs are presented there.

A few data sets have already been re-processed using the LSST stack, and their outputs are available for analysis at different places on CC-IN2P3:

• SXDS data from HSC: /sps/lsst/dev/lsstprod/hsc/SXDS/output

• CFHT data (containing clusters): /sps/lsst/data/clusters

• CFHT D3 fieald: /sps/lsst/data/CFHT/D3

Additional features

• ds9 is automatically available since version 0.9, and can be called in a Jupyter terminal.