Installation Draft - Revision history

Aaron: Redirected page to Installation

2015-03-20T16:54:29Z

Redirected page to Installation

← Older revision		Revision as of 16:54, 20 March 2015
Line 1:		Line 1:
	~~== Prerequisites ==~~		#REDIRECT [[Installation]]

	While these tutorials assume you wish to process XTC streams at SLAC, some users have stills collected from other sources and do not need the full PSDM suite. If this is the case, see the installation directions below for installing on a non PSDM system. Otherwise, it is assumed that the [[Set up PSDM software \| PSDM software distribution has been set up]]. Note again that several sites already have ''cctbx.xfel'' installed, and so regular users not involved in the development of the software will not need the instructions here.

	~~Once ''cctbx.xfel'' has been installed it must be [[Setup \| set up]] before it can be used. Developers may additionally want to [[Set up ssh-agent \| set up an ssh-agent~~]].

	~~Finally, you must have a user account on cci.lbl.gov in order to proceed, or else you will not be able to download the sources.~~

	~~== Standard installation using PSDM ==~~

	This step has to be performed on a host with Internet access. Not all hosts at SLAC have that, but the members of <i>e.g.</i> pslogin pool do. Make and change to a working directory to contain the new source code and build (this directory should be accessible from any computing nodes ''cctbx.xfel'' will be run. Then download these bootstrap modules:

	~~svn export svn://svn.code.sf.net/p/cctbx/code/trunk/libtbx/auto_build/bootstrap.py~~

	~~Then:~~

	~~python bootstrap.py hot update --builder=xfel --cciuser=<cciusername> --sfuser=<sourceforgeusername>~~

	This command instructs bootstrap.py to download static packages required for ''cctbx.xfel'' (hot), and then to checkout the rest of the sources from cci.lbl.gov and sourceforge.net (update), using the user accounts specified by the cciuser and sfuser parameters, respectively.

	After downloading the sources, you need to be sure you have the appropriate compilers before executing the next command. At SLAC, that means you need to ssh to one of the psana nodes, as the pslogin nodes do not have the requisite compilers. When ready, configure and compile thusly:

	~~python bootstrap.py build --builder=xfel --with-python=`which python`~~

	On SLAC's interactive nodes, this takes just over 6 minutes. To avoid problems with run-time dynamic linking of Python extensions, the Python interpreter required for the above command must be the one provided by the PSDM software distribution. That interpreter can be located using find, <i>e.g.</i>
	~~$ find $SIT_ROOT/sw/external/python -perm /0111 -type f -wholename "/$SIT_ARCH//python" 2> /dev/null~~
	~~At SLAC this interpreter is located somewhere under <code>/reg/g/psdm/sw/external/python</code>. Here we use `which python` to get this path automatically.~~

	~~Initialise the running shell using the newly created configuration files. bash-users should~~
	~~$ . build/setpaths.sh~~
	~~while csh-users will instead need to run~~
	~~% source build/setpaths.csh~~

	~~To finalize the installation, see [[Setup]].~~

	~~Note, you may also follow this procedure if you are on a machine where you have your own python that you want to use, instead of the one provided by PSDM.~~

	~~== Installation on a non-PSDM system ==~~

	If you find yourself on a machine without PSDM and won't be dealing with XTC directly, for example if you have your own data collected as stills from a non-XFEL source, you can use the below commands in a new directory (again assuming you have a cci user account):

	~~svn export svn://svn.code.sf.net/p/cctbx/code/trunk/libtbx/auto_build/bootstrap.py~~
	~~python bootstrap.py hot update base build --builder=xfel --cciuser=<cciusername> --sfuser=<sourceforgeusername>~~

	Here, in addition to hot, update and build, there's a command called 'base', which downloads a python and needed packages and compiles it. After this process completes, source the installation (for details see above), and then add scipy to it:

	~~. build/setpaths.sh~~
	~~libtbx.python -m easy_install scipy~~

	~~Please contact the authors if with any issues. Some gotchas:~~

	~~1) On a Mac, you will need to make sure you have the latest version of xcode installed. Consider the command xcode-select --install.~~

	~~2) On linux you will likely need to install some dependencies, including gFortran.~~

	~~== External links ==~~

	* [http://cctbx.sourceforge.net/current_cvs/installation.html#manually-building-from-sources-under-unix Manually building from sources under Unix]


	~~<!-- POSSBILE ADDITIONS:~~

	~~COMMON PITFALLS:~~

	~~forget to source the shell configuration files~~
	~~forget to run sit_setup~~
	~~running cxi.pyana from outside the myrelease directory~~

	~~DEBUGGING:~~

	~~And a few more things to check: output of "ls -lR arch/*", output of "which python2.7", and the "scons.out" file that results from "scons TRACE=7 > scons.out 2>&1".~~

	What Linux distribution are you running this on (could you past the output of "uname -a" and "cat /etc/issue")? Could you also tell me which scons you are using (output of "which scons"). Last, could you attach a dump of the environment (env > env.out)?

	~~Andy's information request on debugging: full output from these commands:~~

	~~% rm -rf arch build~~
	~~% printenv~~
	~~% which scons~~
	~~% which python~~
	~~% which python2.7~~
	~~% scons TRACE=9~~
	~~% ls -lR arch~~
	~~% ls -lR my_ana_pkg~~

	~~-->~~

Aaron: /* Installation on a non-PSDM system */

2015-02-28T08:20:36Z

Installation on a non-PSDM system

← Older revision		Revision as of 08:20, 28 February 2015
Line 43:		Line 43:
	python bootstrap.py hot update base build --builder=xfel --cciuser=<cciusername> --sfuser=<sourceforgeusername>		python bootstrap.py hot update base build --builder=xfel --cciuser=<cciusername> --sfuser=<sourceforgeusername>

	Here, in addition to hot, update and build, there's a command called 'base', which downloads a python and needed packages and compiles it. After this process completes, source the installation, and then add scipy to it:		Here, in addition to hot, update and build, there's a command called 'base', which downloads a python and needed packages and compiles it. After this process completes, source the installation (for details see above), and then add scipy to it:

	. build/setpaths.sh		. build/setpaths.sh
	libtbx.python -m easy_install scipy		libtbx.python -m easy_install scipy

	Please contact the authors if with any issues. Some gotchas:		Please contact the authors if with any issues. Some gotchas:

	1) On a ~~mac~~, you will need to make sure you have the latest version of xcode installed. Consider the command xcode-select --install.		1) On a Mac, you will need to make sure you have the latest version of xcode installed. Consider the command xcode-select --install.

	2) On linux you will likely need to install some dependencies, including gFortran.		2) On linux you will likely need to install some dependencies, including gFortran.

Aaron: re-write

2015-02-28T08:19:31Z

re-write

← Older revision		Revision as of 08:19, 28 February 2015
Line 1:		Line 1:
	~~UNDER CONSTRUCTION~~		== Prerequisites ==

	~~== Prerequisites ==~~		While these tutorials assume you wish to process XTC streams at SLAC, some users have stills collected from other sources and do not need the full PSDM suite. If this is the case, see the installation directions below for installing on a non PSDM system. Otherwise, it is assumed that the [[Set up PSDM software \| PSDM software distribution has been set up]]. Note again that several sites already have ''cctbx.xfel'' installed, and so regular users not involved in the development of the software will not need the instructions here.

	~~While these tutorials assume you wish to process XTC streams at SLAC, some users have stills collected from other sources and do not need the full PSDM suite~~. ~~If this is the case, see the installation directions below. Otherwise,~~ it ~~is assumed that the~~ [[~~Set up PSDM software~~ \| ~~PSDM software distribution has been~~ set up]]. Developers may additionally want to [[Set up ssh-agent \| set up an ssh-agent]]. Note again that several sites already have ''cctbx.xfel'' installed, and that this step can be skipped for regular users not involved in the development of the software. Once ''cctbx.xfel'' has been installed it must be [[Setup \| set up]] before it can be used.		Once ''cctbx.xfel'' has been installed it must be [[Setup \| set up]] before it can be used. Developers may additionally want to [[Set up ssh-agent \| set up an ssh-agent]].

	Finally, you must have a user account on cci.lbl.gov in order to proceed, or else you will not be able to download the sources.		Finally, you must have a user account on cci.lbl.gov in order to proceed, or else you will not be able to download the sources.
Line 34:		Line 34:
	To finalize the installation, see [[Setup]].		To finalize the installation, see [[Setup]].

	== Installation ~~using your own Python~~ ==		Note, you may also follow this procedure if you are on a machine where you have your own python that you want to use, instead of the one provided by PSDM.

			== Installation on a non-PSDM system ==

	If you find yourself on a machine without PSDM and won't be dealing with XTC directly, for example if you have your own data collected as stills from a non-XFEL source, you can use the below commands in a new directory (again assuming you have a cci user account):		If you find yourself on a machine without PSDM and won't be dealing with XTC directly, for example if you have your own data collected as stills from a non-XFEL source, you can use the below commands in a new directory (again assuming you have a cci user account):

Aaron: Add installing using your own python.

2015-02-26T16:47:27Z

Add installing using your own python.

@@ Line 32: / Line 32: @@
   % source build/setpaths.csh
- To finalize the installation, see [[Setup]].
+To finalize the installation, see [[Setup]].
 == External links ==

Aaron at 12:59, 26 February 2015

2015-02-26T12:59:01Z

← Older revision		Revision as of 12:59, 26 February 2015
Line 3:		Line 3:
	== Prerequisites ==		== Prerequisites ==

	While these tutorials ~~assuming~~ you wish to process XTC streams at SLAC, some users have stills collected from other sources and do not need the full PSDM suite. If this is the case, see the installation directions below. Otherwise, it is assumed that the [[Set up PSDM software \| PSDM software distribution has been set up]]. Developers may additionally want to [[Set up ssh-agent \| set up an ssh-agent]]. Note again that several sites already have ''cctbx.xfel'' installed, and that this step can be skipped for regular users not involved in the development of the software. Once ''cctbx.xfel'' has been installed it must be [[Setup \| set up]] before it can be used.		While these tutorials assume you wish to process XTC streams at SLAC, some users have stills collected from other sources and do not need the full PSDM suite. If this is the case, see the installation directions below. Otherwise, it is assumed that the [[Set up PSDM software \| PSDM software distribution has been set up]]. Developers may additionally want to [[Set up ssh-agent \| set up an ssh-agent]]. Note again that several sites already have ''cctbx.xfel'' installed, and that this step can be skipped for regular users not involved in the development of the software. Once ''cctbx.xfel'' has been installed it must be [[Setup \| set up]] before it can be used.

	Finally, you must have a user account on cci.lbl.gov in order to proceed, or else you will not be able to download the sources.		Finally, you must have a user account on cci.lbl.gov in order to proceed, or else you will not be able to download the sources.
Line 17:		Line 17:
	python bootstrap.py hot update --builder=xfel --cciuser=<cciusername> --sfuser=<sourceforgeusername>		python bootstrap.py hot update --builder=xfel --cciuser=<cciusername> --sfuser=<sourceforgeusername>

	This ~~commands instruct~~ bootstrap.py to download static packages required for ''cctbx.xfel'' (hot), and then to checkout the rest of the sources from cci.lbl.gov and sourceforge.net, using the user accounts specified by the cciuser and sfuser parameters, respectively.		This command instructs bootstrap.py to download static packages required for ''cctbx.xfel'' (hot), and then to checkout the rest of the sources from cci.lbl.gov and sourceforge.net (update), using the user accounts specified by the cciuser and sfuser parameters, respectively.

	After downloading the sources, you need to be sure you have the appropriate compilers before executing the next command. At SLAC, that means you need to ssh to one of the psana nodes, as the pslogin nodes do not have the requisite compilers. When ready, configure and compile thusly:		After downloading the sources, you need to be sure you have the appropriate compilers before executing the next command. At SLAC, that means you need to ssh to one of the psana nodes, as the pslogin nodes do not have the requisite compilers. When ready, configure and compile thusly:

Aaron: First draft

2015-02-26T12:55:25Z

First draft

New page

UNDER CONSTRUCTION

== Prerequisites ==

While these tutorials assuming you wish to process XTC streams at SLAC, some users have stills collected from other sources and do not need the full PSDM suite. If this is the case, see the installation directions below. Otherwise, it is assumed that the [[Set up PSDM software | PSDM software distribution has been set up]]. Developers may additionally want to [[Set up ssh-agent | set up an ssh-agent]]. Note again that several sites already have ''cctbx.xfel'' installed, and that this step can be skipped for regular users not involved in the development of the software. Once ''cctbx.xfel'' has been installed it must be [[Setup | set up]] before it can be used.

Finally, you must have a user account on cci.lbl.gov in order to proceed, or else you will not be able to download the sources.

== Standard installation using PSDM ==

This step has to be performed on a host with Internet access. Not all hosts at SLAC have that, but the members of <i>e.g.</i> pslogin pool do. Make and change to a working directory to contain the new source code and build (this directory should be accessible from any computing nodes ''cctbx.xfel'' will be run. Then download these bootstrap modules:

svn export svn://svn.code.sf.net/p/cctbx/code/trunk/libtbx/auto_build/bootstrap.py

Then:

python bootstrap.py hot update --builder=xfel --cciuser=<cciusername> --sfuser=<sourceforgeusername>

This commands instruct bootstrap.py to download static packages required for ''cctbx.xfel'' (hot), and then to checkout the rest of the sources from cci.lbl.gov and sourceforge.net, using the user accounts specified by the cciuser and sfuser parameters, respectively.

After downloading the sources, you need to be sure you have the appropriate compilers before executing the next command. At SLAC, that means you need to ssh to one of the psana nodes, as the pslogin nodes do not have the requisite compilers. When ready, configure and compile thusly:

python bootstrap.py build --builder=xfel --with-python=`which python`

On SLAC's interactive nodes, this takes just over 6 minutes. To avoid problems with run-time dynamic linking of Python extensions, the Python interpreter required for the above command must be the one provided by the PSDM software distribution. That interpreter can be located using find, <i>e.g.</i>
$ find $SIT_ROOT/sw/external/python -perm /0111 -type f -wholename "*/$SIT_ARCH/*/python" 2> /dev/null
At SLAC this interpreter is located somewhere under <code>/reg/g/psdm/sw/external/python</code>. Here we use `which python` to get this path automatically.

Initialise the running shell using the newly created configuration files. bash-users should
$ . build/setpaths.sh
while csh-users will instead need to run
% source build/setpaths.csh

To finalize the installation, see [[Setup]].

== External links ==

* [http://cctbx.sourceforge.net/current_cvs/installation.html#manually-building-from-sources-under-unix Manually building from sources under Unix]