Difference between revisions of "Preparatory steps pyana version"
(Created page with "The CS-PAD detector at LCLS is sensitive to temperature distortions and is not automatically dark-subtracted, like images from a synchrotron source. Because of this it is nec...") |
(→Create a dark average) |
||
| Line 6: | Line 6: | ||
== Create a dark average == | == Create a dark average == | ||
| + | |||
| + | Inputs: dark.cfg, run of interest. Execute these commands: | ||
| + | |||
| + | $ cd ~/myrelease | ||
| + | $ sit_setup | ||
| + | $ cp /reg/d/ffb/cxi/temp/cctbx/tutorials/preprocessing/dark.cfg . | ||
| + | |||
| + | Open dark.cfg. It will look something like this: | ||
| + | |||
| + | <pre> | ||
| + | # -*- Mode: Conf -*- | ||
| + | # | ||
| + | # Since mod_average must be run on a single host, it cannot be | ||
| + | # submitted using lsf.sh. | ||
| + | # | ||
| + | # bsub -n 6 -J r0069 -o /reg/d/ffb/cxi/temp/cctbx/tutorials/scratch/<username>/r0146.out -q psnehmpiq -R "span[hosts=1]" | ||
| + | "cxi.pyana -c ~/myrelease/dark.cfg /reg/d/ffb/cxi/temp/Feb2013calib/e290-r0069-s0*.xtc" | ||
| + | # | ||
| + | |||
| + | [pyana] | ||
| + | modules = my_ana_pkg.mod_average | ||
| + | num-cpu = 8 | ||
| + | |||
| + | [my_ana_pkg.mod_average] | ||
| + | calib_dir = /reg/d/ffb/cxi/temp/cctbx/metrology/CSPad/run4/CxiDs1.0:Cspad.0 | ||
| + | avg_dirname = /reg/d/ffb/cxi/temp/cctbx/tutorials/scratch/<username>/darks | ||
| + | stddev_dirname = /reg/d/ffb/cxi/temp/cctbx/tutorials/scratch/<username>/darks | ||
| + | detz_offset = 581 | ||
| + | address = CxiDs1-0|Cspad-0 | ||
| + | avg_basename = Ds1-r0069-avg | ||
| + | stddev_basename = Ds1-r0069-stddev | ||
| + | </pre> | ||
| + | |||
| + | The config file is broken into three sections: | ||
| + | === Header Section === | ||
| + | The comments at the top of the file include a bsub command. This is the command to be entered from the ~/myrelease directory to do queue the job for processing. Bsub will submit the job, asking for 6 CPUs (-n), naming the job r0069 (-J), placing a log file in the -o location, and so forth. There are several queues available for use, the psnehmpiq is just one of them. The -R option restricts the job to running on a single host, which is required for averaging. Finally, the cxi.pyana command in quotes is the command that will actually be executed on the host computer, and specifies the config file and the streams to use. Note the asterisk, which allows averaging all the streams from a single run at one time. | ||
| + | |||
| + | Before executing the command, replace <username> in the command and in the config file with your user name. | ||
| + | |||
| + | === Pyana Section === | ||
| + | Pyana operates by having a root section in the config file which specifies which modules to run. In this case we are running "cctbx.xfel's" mod_average module. If we were running multiple modules, we could chain them together here, and each module would be executed in sequence using the parameters specified below. We also request a number of cpus to parallelize the job with. Even though averaging must be done on a single host, on that host the processing can be split up among multiple cores. Note, other aspects of "cctbx.xfel", such as indexing and integration, do not have this restriction. | ||
| + | |||
| + | === Module Section === | ||
| + | The final section specifies parameters specific to averaging: | ||
| + | * calib_dir: this directory is provided by LCLS | ||
Revision as of 17:09, 26 September 2013
The CS-PAD detector at LCLS is sensitive to temperature distortions and is not automatically dark-subtracted, like images from a synchrotron source. Because of this it is necessary to periodically collect a dark run with the shutters closed during an LCLS experiment. The dark run should contain on the order of X images. Before an experimental run can be processed, this dark must be averaged, so that subsequent experimental data can have it subtracted. Further, events during collection can necessitate a new dark, such as ice in the beam oversaturating creating dead pixels.
In addition, inactive and non-bonded pixels, as well as hypersensitive pixels and any beam stop (if used) should be masked out.
Finally, averaging a light run in general is a useful tool for examining diffraction throughout a data collection run. This tutorial covers averaging, dark subtraction and masking.
Create a dark average
Inputs: dark.cfg, run of interest. Execute these commands:
$ cd ~/myrelease $ sit_setup $ cp /reg/d/ffb/cxi/temp/cctbx/tutorials/preprocessing/dark.cfg .
Open dark.cfg. It will look something like this:
# -*- Mode: Conf -*- # # Since mod_average must be run on a single host, it cannot be # submitted using lsf.sh. # # bsub -n 6 -J r0069 -o /reg/d/ffb/cxi/temp/cctbx/tutorials/scratch/<username>/r0146.out -q psnehmpiq -R "span[hosts=1]" "cxi.pyana -c ~/myrelease/dark.cfg /reg/d/ffb/cxi/temp/Feb2013calib/e290-r0069-s0*.xtc" # [pyana] modules = my_ana_pkg.mod_average num-cpu = 8 [my_ana_pkg.mod_average] calib_dir = /reg/d/ffb/cxi/temp/cctbx/metrology/CSPad/run4/CxiDs1.0:Cspad.0 avg_dirname = /reg/d/ffb/cxi/temp/cctbx/tutorials/scratch/<username>/darks stddev_dirname = /reg/d/ffb/cxi/temp/cctbx/tutorials/scratch/<username>/darks detz_offset = 581 address = CxiDs1-0|Cspad-0 avg_basename = Ds1-r0069-avg stddev_basename = Ds1-r0069-stddev
The config file is broken into three sections:
Header Section
The comments at the top of the file include a bsub command. This is the command to be entered from the ~/myrelease directory to do queue the job for processing. Bsub will submit the job, asking for 6 CPUs (-n), naming the job r0069 (-J), placing a log file in the -o location, and so forth. There are several queues available for use, the psnehmpiq is just one of them. The -R option restricts the job to running on a single host, which is required for averaging. Finally, the cxi.pyana command in quotes is the command that will actually be executed on the host computer, and specifies the config file and the streams to use. Note the asterisk, which allows averaging all the streams from a single run at one time.
Before executing the command, replace <username> in the command and in the config file with your user name.
Pyana Section
Pyana operates by having a root section in the config file which specifies which modules to run. In this case we are running "cctbx.xfel's" mod_average module. If we were running multiple modules, we could chain them together here, and each module would be executed in sequence using the parameters specified below. We also request a number of cpus to parallelize the job with. Even though averaging must be done on a single host, on that host the processing can be split up among multiple cores. Note, other aspects of "cctbx.xfel", such as indexing and integration, do not have this restriction.
Module Section
The final section specifies parameters specific to averaging:
- calib_dir: this directory is provided by LCLS
