Set up PSDM software: Difference between revisions

From cctbx_xfel
Jump to navigation Jump to search
(Added BNL to known installations.)
No edit summary
 
(13 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Prerequisites ==
THIS IS DEPRECATED.  psana is now provided using conda.  [https://exafel.github.io/docs/psana-cctbx-instal Directions].


<!-- XXX What's PSDM? -->
Note that throughout the instructions <code>$</code> denotes the prompt of any unprivileged user on the system.  Certain shells may be configured to use <code>%</code> or <code>&gt;</code> for that purpose.  The prompt should <em>not</em> be included in the commands typed to the shell.


The PSDM software distribution must be installed.  PSDM provides installation and maintenance instructions at [https://confluence.slac.stanford.edu/display/PCDS/PSDM+software+distribution PSDM software distribution].  The distribution can be installed anywhere on the filesystem; it is generally possible to install the PSDM software distribution without superuser privileges.
== Prerequisites ==


The installation root directory affects the absolute paths used in these instructions and differs from site to site.  Some common installation root directories are listed below.
The [https://confluence.slac.stanford.edu/display/PSDM/Software+Distribution Photon Science Data Management (PSDM)] software distribution must be installed.  The PSDM software distribution provides ''psana'', which is a critical dependency of ''cctbx.xfel''.  Its installation root directory affects the absolute paths used in the following instructions and differs from site to site.  Some common installation root directories are listed below.
{| class="wikitable sortable"
{| class="wikitable sortable"
|-
|-
! Site
! Site
!class="unsortable"| Installation root directory
! class="unsortable"| Installation root directory
!class="unsortable"| Notes
! class="unsortable"| Notes
|-
|-
| BNL
| BNL
Line 19: Line 19:
| <code>/net/viper/raid1/psdm</code>
| <code>/net/viper/raid1/psdm</code>
| Only available on viper.lbl.gov
| Only available on viper.lbl.gov
|-
| NERSC
| <code>/global/project/projectdirs/lcls/psdm-carver</code>
| Only to be used on the carver system
|-
|-
| SLAC
| SLAC
Line 24: Line 28:
|
|
|}
|}
The PSDM software distribution is supported on the most popular flavors of Linux and is installed at several sites worldwide.  Separate installation and maintenance instructions are provided by the PSDM project at [https://confluence.slac.stanford.edu/display/PSDM/Software+Distribution PSDM software distribution].  The package can be installed anywhere on the filesystem, and it is generally possible to install it without superuser privileges.


== Create a test release and analysis package ==
== Create a test release and an analysis package ==


In what follows it is assumed that the PSDM software distribution is installed at <code>/reg/g/psdm</code>.  Substitute <code>/reg/g/psdm</code> with the appropriate path in the instructions below.
A <i>test release</i>, which refers to a particular release of the PSDM distribution, is represented by a directory in the file system, often called <code>analysis-rel</code> or <code>myrelease</code>. An analysis package within the test release in turn refers to the files implementing the analysis modules of the package. Details about test releases and analysis packages are covered in [https://confluence.slac.stanford.edu/display/PCDS/Analysis+Workbook.+Quick+Tour Analysis Workbook.  Quick Tour].  To prepare the environment for using the PSDM software distribution, bash-users should source <code>ana_env.sh</code>,
$ . <b><i>/path/to/psdm</i></b>/etc/ana_env.sh
where <code><b><i>/path/to/psdm</i></b></code> should be replaced with the path to the installation root directorycsh-users should source <code>ana_env.csh</code> instead,
% source <b><i>/path/to/psdm</i></b>/etc/ana_env.csh
Sourcing <code>ana_env.sh</code> or <code>ana_env.csh</code> only modifies the environment of the <em>current shell</em>, and would have to be repeated every time a new shell is started.  To make the changes persistent, add
test -r <b><i>/path/to/psdm</i></b>/etc/ana_env.sh && . <b><i>/path/to/psdm</i></b>/etc/ana_env.sh
to <code>~/.bashrc</code>, or
test -r <b><i>/path/to/psdm</i></b>/etc/ana_env.csh && source <b><i>/path/to/psdm</i></b>/etc/ana_env.csh
to <code>~/.cshrc</code> as appropriate.  This ensures the environment is properly prepared for non-interactive shells, <i>e.g.</i> the shells that are used to run jobs submitted to the cluster.  To also be able to use the PSDM software from interactive shells, it is recommended that bash-users put
test -r ~/.bashrc && . ~/.bashrc
near the top of <code>~/.bash_profile</code>.  There is no need to edit any additional files for csh-users, because csh always reads <code>~/.cshrc</code>.


A ''test release'', which refers to a particular release of the LCLS packages, is represented by a directory in the file system, often called <code>analysis-rel</code> or <code>myrelease</code>. An analysis package within the test release in turn refers to the files implementing the analysis modules of the package. Details about installing of a test release and analysis packages are covered in [https://confluence.slac.stanford.edu/display/PCDS/Analysis+Workbook.+Quick+Tour Analysis Workbook.  Quick Tour].  The PSDM environment is prepared in a bash(1) shell using
A test released based on the most current PSDM distribution is then set up in the current working directory using
$ . /reg/g/psdm/etc/ana_env.sh
or
% source /reg/g/psdm/etc/ana_env.csh
for a csh(1).  Sourcing <code>ana_env.sh</code> or <code>ana_env.csh</code> modifies the environment of the current shell, and would have to be repeated every time a new shell is started.  To make the changes persistent, add
test -r /reg/g/psdm/etc/ana_env.sh && . /reg/g/psdm/etc/ana_env.sh
to <code>~/.bashrc</code>, or
test -r /reg/g/psdm/etc/ana_env.csh && source /reg/g/psdm/etc/ana_env.csh
to <code>~/.cshrc</code> as appropriate.  A test released based on the most current LCLS software is then set up in the current working directory using
<!-- $ cd -->
  $ newrel ana-current <i>myrelease</i>
  $ newrel ana-current <i>myrelease</i>
  $ cd <i>myrelease</i>
  $ cd <i>myrelease</i>
  $ sit_setup
  $ sit_setup
  $ newpkg <i>my_ana_pkg</i>
  $ newpkg <i>my_ana_pkg</i>
Note that the names for the test release and the analysis package, <i>myrelease</i> and <i>my_ana_pkg</i>, are common, but nevertheless arbitrary, choices.
Note that the names for the test release and the analysis package, <i>myrelease</i> and <i>my_ana_pkg</i> above, are common, but nevertheless arbitrary, choices.
 
 
== Keeping up to date with PSDM releases==
 
Since the ''cctbx.xfel'' modules do not critically depend on any particular version of the PSDM software, it is generally possible to closely track the latest version of the PSDM distribution.  To upgrade to the current PSDM release, do
$ cd myrelease
$ relupgrade ana-current
If <code>relinfo</code> does not output <code>ana-current</code> after that, one can edit <code>~/myrelease/.sit_release</code> manually to that effect.  Then rebuild
$ scons -c
$ scons


These steps only need to be performed once.


== External links ==
== External links ==


* More detailed manual: [https://confluence.slac.stanford.edu/display/PCDS/Pyana+User+Manual Pyana User Manual]
* [https://confluence.slac.stanford.edu/display/PSDM/psana+-+User+Manual Psana User Manual]
* [https://confluence.slac.stanford.edu/display/PCDS/Analysis+Workbook.+Packages+and+Releases Analysis Workbook. Packages and Releases]
* [https://confluence.slac.stanford.edu/display/PSDM/Packages+and+Releases Packages and Releases]

Latest revision as of 21:31, 6 November 2018

THIS IS DEPRECATED. psana is now provided using conda. Directions.

Note that throughout the instructions $ denotes the prompt of any unprivileged user on the system. Certain shells may be configured to use % or > for that purpose. The prompt should not be included in the commands typed to the shell.

Prerequisites

The Photon Science Data Management (PSDM) software distribution must be installed. The PSDM software distribution provides psana, which is a critical dependency of cctbx.xfel. Its installation root directory affects the absolute paths used in the following instructions and differs from site to site. Some common installation root directories are listed below.

Site Installation root directory Notes
BNL /usr/local/crys/psdm
CCI, LBNL /net/viper/raid1/psdm Only available on viper.lbl.gov
NERSC /global/project/projectdirs/lcls/psdm-carver Only to be used on the carver system
SLAC /reg/g/psdm

The PSDM software distribution is supported on the most popular flavors of Linux and is installed at several sites worldwide. Separate installation and maintenance instructions are provided by the PSDM project at PSDM software distribution. The package can be installed anywhere on the filesystem, and it is generally possible to install it without superuser privileges.

Create a test release and an analysis package

A test release, which refers to a particular release of the PSDM distribution, is represented by a directory in the file system, often called analysis-rel or myrelease. An analysis package within the test release in turn refers to the files implementing the analysis modules of the package. Details about test releases and analysis packages are covered in Analysis Workbook. Quick Tour. To prepare the environment for using the PSDM software distribution, bash-users should source ana_env.sh,

$ . /path/to/psdm/etc/ana_env.sh

where /path/to/psdm should be replaced with the path to the installation root directory. csh-users should source ana_env.csh instead,

% source /path/to/psdm/etc/ana_env.csh

Sourcing ana_env.sh or ana_env.csh only modifies the environment of the current shell, and would have to be repeated every time a new shell is started. To make the changes persistent, add

test -r /path/to/psdm/etc/ana_env.sh && . /path/to/psdm/etc/ana_env.sh

to ~/.bashrc, or

test -r /path/to/psdm/etc/ana_env.csh && source /path/to/psdm/etc/ana_env.csh

to ~/.cshrc as appropriate. This ensures the environment is properly prepared for non-interactive shells, e.g. the shells that are used to run jobs submitted to the cluster. To also be able to use the PSDM software from interactive shells, it is recommended that bash-users put

test -r ~/.bashrc && . ~/.bashrc

near the top of ~/.bash_profile. There is no need to edit any additional files for csh-users, because csh always reads ~/.cshrc.

A test released based on the most current PSDM distribution is then set up in the current working directory using

$ newrel ana-current myrelease
$ cd myrelease
$ sit_setup
$ newpkg my_ana_pkg

Note that the names for the test release and the analysis package, myrelease and my_ana_pkg above, are common, but nevertheless arbitrary, choices.

These steps only need to be performed once.

External links