Preparatory steps pyana version - Revision history

Aaron: /* Running and testing the command */

2015-05-08T17:37:13Z

Running and testing the command

← Older revision		Revision as of 17:37, 8 May 2015
Line 74:		Line 74:
	Before starting the averaging job the command, replace <code><b><i><username></i></b></code> in the configuration file with your user name. To average run 2 using the modified configuration file, execute		Before starting the averaging job the command, replace <code><b><i><username></i></b></code> in the configuration file with your user name. To average run 2 using the modified configuration file, execute
	$ cxi.lsf -c ~/myrelease/dark.cfg -i /reg/d/ana11/cxi/data/Mar2013calib/xtc \		$ cxi.lsf -c ~/myrelease/dark.cfg -i /reg/d/ana11/cxi/data/Mar2013calib/xtc \
	-o /reg/g/cctbx/tutorials/scratch/<b><i><username></i></b> -p 6 -q ~~psanacsq~~ -r 2 -s		-o /reg/g/cctbx/tutorials/scratch/<b><i><username></i></b> -p 6 -q psanaq -r 2 -s
	This will request 6 processors, overriding the <code>num-cpu</code> parameter set in the configuration file. The directory specified in the <code>-i</code> option will be searched for XTC streams, and output will be arranged in the directory defined by the value of the <code>-o</code> option. The <code>-s</code> option tells ''cxi.lsf'' to process all the streams on the same host—this is a requirement for averaging. The job will be scheduled in the queue called <code>~~psanacsq~~</code>, which is one of several available queues. Note that the job must be submitted from the <code>~/myrelease</code> directory.		This will request 6 processors, overriding the <code>num-cpu</code> parameter set in the configuration file. The directory specified in the <code>-i</code> option will be searched for XTC streams, and output will be arranged in the directory defined by the value of the <code>-o</code> option. The <code>-s</code> option tells ''cxi.lsf'' to process all the streams on the same host—this is a requirement for averaging. The job will be scheduled in the queue called <code>psanaq</code>, which is one of several available queues. Note that the job must be submitted from the <code>~/myrelease</code> directory.

	Type <code>bjobs</code> to see if the job is running. When it is, follow the output thusly:		Type <code>bjobs</code> to see if the job is running. When it is, follow the output thusly:
Line 83:		Line 83:
	When the average has been completed, the output can be inspected using the ''cctbx'' image viewer:		When the average has been completed, the output can be inspected using the ''cctbx'' image viewer:
	$ cctbx.image_viewer /reg/g/cctbx/tutorials/scratch/<b><i><username></i></b>/r0002/000/out/Ds1-r0002-*.pickle		$ cctbx.image_viewer /reg/g/cctbx/tutorials/scratch/<b><i><username></i></b>/r0002/000/out/Ds1-r0002-*.pickle


	== Averaging an experimental light run ==		== Averaging an experimental light run ==

Aaron: Aaron moved page Preparatory steps to Preparatory steps pyana version

2014-12-12T00:40:55Z

Aaron moved page Preparatory steps to Preparatory steps pyana version

← Older revision	Revision as of 00:40, 12 December 2014
(No difference)

Aaron: /* Creating a mask image */

2014-08-22T05:57:08Z

Creating a mask image

← Older revision		Revision as of 05:57, 22 August 2014
Line 144:		Line 144:

	The mask can be examined with ''cctbx.image_viewer''.		The mask can be examined with ''cctbx.image_viewer''.

			==== Applying the mask ====

			Once the mask is created, it needs to be referenced by your configuration file:

			mask_path = <absolute path to mask file>

			Further, the indexing software needs to know what value to use as the masked-out value. Add the following to your phil files for indexing:

			integration.mask_pixel_value=-2

			Where -2 is the default mask value, but it can be whatever you specify using the -x option cxi.make_mask (see below).

	=== Options ===		=== Options ===

Nicksauter: /* Options */

2014-08-19T21:21:03Z

Options

← Older revision		Revision as of 21:21, 19 August 2014
Line 156:		Line 156:
	;-x, --mask_pix_val (default -2)		;-x, --mask_pix_val (default -2)
	: specifies the value to use when masking out a pixel. In other words, bad pixels in images in the XTC stream will be replaced with this value.		: specifies the value to use when masking out a pixel. In other words, bad pixels in images in the XTC stream will be replaced with this value.
			;-o, --output (default mask_.pickle)
			: specifies the output file path, should be *.pickle, can include directory.

Hattne: Updated and reorganized averaging to cover cxi.lsf's -s option

2014-03-14T20:05:02Z

Updated and reorganized averaging to cover cxi.lsf's -s option

Show changes

Hattne: Changed CS-PAD to CSPAD for consistency.

2014-03-14T18:11:33Z

Changed CS-PAD to CSPAD for consistency.

New page

The CSPAD detector at LCLS is sensitive to temperature distortions and is not automatically dark-subtracted, unlike images from a synchrotron source. Because of this it is necessary to periodically collect a dark run with the shutters closed during an LCLS experiment. The dark run should contain on the order of at least 1000 images. Before an experimental run can be processed, this dark run must be averaged so that subsequent experimental data can have the dark subtracted from each image. Further, events during collection can necessitate a new dark, such as ice in the beam oversaturating creating dead pixels or a change in temperature.

In addition, averaging a lighted run in general is a useful tool for examining diffraction throughout a data collection run.

Finally, inactive and non-bonded pixels, as well as hypersensitive pixels and any beam stop (if used) should be masked out. This tutorial covers averaging dark and light runs, and creating mask images.

== Setup your scratch area ==

Create a scratch folder and populate it with some directories that will be used during this tutorial, changing <username> to your SLAC account name:

$ cd /reg/g/cctbx/tutorials/scratch/
$ mkdir <username>
$ cd <username>
$ mkdir darks
$ mkdir averages
$ mkdir results

Or you may use a directory in your home folder.

== Create a dark average ==

To create the average, start by copying the dark.cfg file to your pyana directory:

$ cd ~/myrelease
$ sit_setup
$ cp /reg/g/cctbx/tutorials/preprocessing/dark.cfg .

Open dark.cfg. It will look something like this:


<pre>
# -*- Mode: Conf -*-
#
# Since mod_average must be run on a single host, it cannot be
# submitted using cxi.lsf.
#
# bsub -n 6 -J r0002 -o /reg/g/cctbx/tutorials/scratch/<username>/dark_r0002.out -q psanacsq -R "span[hosts=1]" \
# "cxi.pyana -c ~/myrelease/dark.cfg /reg/d/ana11/cxi/data/Mar2013calib/xtc/e236-r0002-s0*.xtc"
#

[pyana]
modules = my_ana_pkg.mod_average
num-cpu = 8

[my_ana_pkg.mod_average]
calib_dir = /reg/g/cctbx/sources/cctbx_project/xfel/metrology/CSPad/run4/CxiDs1.0_Cspad.0
avg_dirname = /reg/g/cctbx/tutorials/scratch/<username>/darks
stddev_dirname = /reg/g/cctbx/tutorials/scratch/<username>/darks
max_dirname = /reg/g/cctbx/tutorials/scratch/<username>/darks
detz_offset = 581
address = CxiDs1-0|Cspad-0
avg_basename = Ds1-r0002-avg
stddev_basename = Ds1-r0002-stddev
max_basename = Ds1-r0002-max
</pre>

The config file is broken into three sections, the header, the pyana section and the modules section.

=== Header Section ===
The comments at the top of the file include a bsub command. This is the command to be entered from the ~/myrelease directory to queue the job for processing. Bsub will submit the job, asking for 6 CPUs (-n), naming the job r0002 (-J), placing a log file in the -o location, and so forth. There are several queues available for use, the psanacsq is just one of them. The -R option restricts the job to running on a single host, which is required for averaging. Finally, the cxi.pyana command in quotes is the command that will actually be executed on the host computer, and specifies the config file and the streams to use. Note the asterisk, which allows averaging all the streams from a single run at one time.

'''Before executing the command, replace <username> in the command and in the config file with your user name.'''

=== Pyana Section ===
Pyana operates by having a root section in the config file which specifies which modules to run. In this case we are running ''cctbx.xfel's'' mod_average module. If we were running multiple modules, we could chain them together here, and each module would be executed in sequence using the parameters specified below. We also request a number of cpus to parallelize the job with. Even though averaging must be done on a single host, on that host the processing can be split up among multiple cores. Note, other aspects of ''cctbx.xfel'', such as indexing and integration, do not have this restriction.

=== Module Section ===
The final section specifies parameters specific to averaging:
* calib_dir: this directory is provided by LCLS and includes the layout information of the 64 tiles of the CSPAD detector
* avg_dirname: directory to place the average image
* stddev_dirname: directory to place the standard deviation image
* max_dirname: directory to place the maximum projection image
* detz_offset: experiment-specific number used to calculate the detector distance. This is the distance in millimeters between the sample interaction region and the far end of the detector stage. The XTC stream contains the distance from the detector stage to the detector. These two numbers are subtracted to calculate the distance between sample interaction region and the detector. We determine this number experimentally using a grid search, testing a range of potential detz_offsets and choosing the one that shows the greatest success at indexing images
* address: the name of the detector in the XTC stream
* avg_basename: this name is prepended to a datestamp to name the average image
* stddev_basename: this name is prepended to a datestamp to name the standard deviation image
* max_basename: this name is prepended to a datestamp to name the maximum projection image image

=== Running and testing the command ===
Submit the job as specified above, using the bsub command in the header:
$ bsub -n 6 -J r0002 -o /reg/g/cctbx/tutorials/scratch/<username>/dark_r0002.out -q psanacsq -R "span[hosts=1]"
"cxi.pyana -c ~/myrelease/dark.cfg /reg/d/ana11/cxi/data/Mar2013calib/xtc/e236-r0002-s0*.xtc"

Type bjobs to see if the job is running. When it is, follow the output thusly:
$ tail -f /reg/g/cctbx/tutorials/scratch/<username>/dark_r0002.out

When the average has been completed, the output can be inspected using the "cctbx" image viewer:
$ cctbx.image_viewer /reg/g/cctbx/tutorials/scratch/<username>/darks/Ds1-r0002-*.pickle

== Averaging an experimental light run ==

Once the dark run is made, experimental runs can be averaged to see the extent of diffraction and potential pathologies in the run. Copy the average.cfg file from the tutorials directory to your pyana directory:

$ cd ~/myrelease
$ cp /reg/g/cctbx/tutorials/preprocessing/average.cfg .

The cfg will look like this:
<pre>
# -*- Mode: Conf -*-
#
# Since mod_average must be run on a single host, it cannot be
# submitted using cxi.lsf.
#
# bsub -n 6 -J r0003 -o /reg/g/cctbx/tutorials/scratch/<username>/average_r0003.out -q psanacsq -R "span[hosts=1]" \
# "cxi.pyana -c average.cfg /reg/d/ana11/cxi/data/Mar2013calib/xtc/e236-r0003-s0*.xtc"
#

[pyana]
modules = my_ana_pkg.mod_average
num-cpu = 8

[my_ana_pkg.mod_average]
calib_dir = /reg/g/cctbx/sources/cctbx_project/xfel/metrology/CSPad/run4/CxiDs1.0_Cspad.0
dark_path = /reg/g/cctbx/tutorials/scratch/<username>/darks/<name of dark average>.pickle
dark_stddev = /reg/g/cctbx/tutorials/scratch/<username>/darks/<name of dark stddev>.pickle
avg_dirname = /reg/g/cctbx/tutorials/scratch/<username>/averages
stddev_dirname = /reg/g/cctbx/tutorials/scratch/<username>/averages
max_dirname = /reg/g/cctbx/tutorials/scratch/<username>/averages
detz_offset = 581
address = CxiDs1-0|Cspad-0
avg_basename = Ds1-r0003-avg
stddev_basename = Ds1-r0003-stddev
max_basename = Ds1-r0003-max
</pre>

The only difference is the addition of two parameters:
* dark_path: path to the average dark image created before
* dark_stddev: path to the standard deviation dark image created before

The job is ran as before using the bsub command in the header, again remembering to change all instances of <username> to your own SLAC user name. Here you must also change <name of dark average> and <name of dark stddev> to the names of the average and standard deviation images created in the previous step. The result of the run will be three new images that can be viewed with cctbx.image_viewer in the same manner as the darks were.

== Information contained in the light average ==

Firstly, the light and dark images can be used to mask of any bad pixels, as detailed below. The light images (maximum in particular) also serve as a virtual, dark-subtracted, powder pattern that can used to inspect the data for diffraction. The light maximum should also flag up obvious errors in metrology, because rings will be non-continuous if the quadrants are misaligned. Intensity values from the corners of the light average or maximum are also useful to estimate background when taking first guesses at thresholds for hit-finding and integration.

== Creating a mask image ==
''cctbx.xfel'' uses three images to create a mask. Those images, and the purposes of them are listed below:
* Average image from a dark run: pixels ≤ 0 are considered dead, pixels > 2000 are too sensitive
* Standard deviation from a dark run: pixels ≤ 0 are considered dead, pixels ≥ 10 are too uncertain
* Maximum projection from a lighted run (''i.e.'' an experimental run): pixels < 300 are considered non-bonded or in a shadow. [N.B.--this assumes a lot--300 may be appropriate for XPP data collected at atmosphere; not CXI data in vacuo--NKS.] The presence of diffracting data is not needed, but also will not interfere.

To create the mask, execute these commands:
$ cd /reg/g/cctbx/tutorials/scratch/<username>/
$ mkdir masks
$ cd masks
$ cxi.make_mask -m 10 ../darks/<name of dark average>.pickle ../darks/<name of dark standard deviation>.pickle
../averages/<name of light maximum projection>.pickle

A new image is created where each pixel has one of two values: -2 (masked out pixels), and 0 (good pixels). Note, the maximum projection minimum pixel value is overridden here to a value that is more sensible for the tutorial's lysozyme data. You will likely need to adjust this parameter based on your data (see below for details and for further overrides).

Use the resultant mask image during processing with ''cctbx.xfel'' by adding this flag to your pyana config file:

mask_path = <path to mask file>

In addition, a polygon and a circle can be specified using the --poly_mask and --circle_mask parameters:

$ cxi.make_mask --poly_mask=17,854,76,967,760,982,762,879 --circle_mask=855,939,103 Cspad-avg.pickle Cspad-stddev.pickle Cspad-max.pickle

This example creates a mask image with a polygon [with vertices (17,854), (76,967), (760,982), and (762,879)] masked out, and a circle with center (855,939) and radius 103 masked out. Only one polygon and circle can be specified at the moment.

The mask can be examined with cctbx.image_viewer.

=== Options ===

Some additional options are available for specifying the details of the masking operation:
;-a, --avg_max (default 2000)
: specifies the maximum value in ADU a pixel in the average image can have before it's masked out. The minimum is always zero.
;-s, --stddev_max (default 10)
: specifies the maximum value in ADU a pixel in the standard deviation image can have before it's masked out. The minimum is always zero.
;-m, --maxproj_min (default 300)
: specifies the minimum value in ADU a pixel in the maximum projection image can have before it's masked out. Of the three parameters for controlling cutoffs in ''cxi.make_mask'', the -m option is unique from the -a and -s options in that it will likely vary the according to your sample's background. Carefully examine the corners of your lighted maximum projection and choose a value lower than the ADU values displayed. Also note that there is no cutoff on the high end of the maximum projection image specified in the mask, as that is defined as the saturation value for the detector.
;-x, --mask_pix_val (default -2)
: specifies the value to use when masking out a pixel. In other words, bad pixels in images in the XTC stream will be replaced with this value.

Hattne: Replaced ./lsf.sh with cxi.lsf.

2014-02-11T00:36:10Z

Replaced ./lsf.sh with cxi.lsf.

New page

The CS-PAD detector at LCLS is sensitive to temperature distortions and is not automatically dark-subtracted, unlike images from a synchrotron source. Because of this it is necessary to periodically collect a dark run with the shutters closed during an LCLS experiment. The dark run should contain on the order of at least 1000 images. Before an experimental run can be processed, this dark run must be averaged so that subsequent experimental data can have the dark subtracted from each image. Further, events during collection can necessitate a new dark, such as ice in the beam oversaturating creating dead pixels or a change in temperature.

In addition, averaging a lighted run in general is a useful tool for examining diffraction throughout a data collection run.

Finally, inactive and non-bonded pixels, as well as hypersensitive pixels and any beam stop (if used) should be masked out. This tutorial covers averaging dark and light runs, and creating mask images.

== Setup your scratch area ==

Create a scratch folder and populate it with some directories that will be used during this tutorial, changing <username> to your SLAC account name:

$ cd /reg/g/cctbx/tutorials/scratch/
$ mkdir <username>
$ cd <username>
$ mkdir darks
$ mkdir averages
$ mkdir results

Or you may use a directory in your home folder.

== Create a dark average ==

To create the average, start by copying the dark.cfg file to your pyana directory:

$ cd ~/myrelease
$ sit_setup
$ cp /reg/g/cctbx/tutorials/preprocessing/dark.cfg .

Open dark.cfg. It will look something like this:


<pre>
# -*- Mode: Conf -*-
#
# Since mod_average must be run on a single host, it cannot be
# submitted using cxi.lsf.
#
# bsub -n 6 -J r0002 -o /reg/g/cctbx/tutorials/scratch/<username>/dark_r0002.out -q psanacsq -R "span[hosts=1]" \
# "cxi.pyana -c ~/myrelease/dark.cfg /reg/d/ana11/cxi/data/Mar2013calib/xtc/e236-r0002-s0*.xtc"
#

[pyana]
modules = my_ana_pkg.mod_average
num-cpu = 8

[my_ana_pkg.mod_average]
calib_dir = /reg/g/cctbx/sources/cctbx_project/xfel/metrology/CSPad/run4/CxiDs1.0_Cspad.0
avg_dirname = /reg/g/cctbx/tutorials/scratch/<username>/darks
stddev_dirname = /reg/g/cctbx/tutorials/scratch/<username>/darks
max_dirname = /reg/g/cctbx/tutorials/scratch/<username>/darks
detz_offset = 581
address = CxiDs1-0|Cspad-0
avg_basename = Ds1-r0002-avg
stddev_basename = Ds1-r0002-stddev
max_basename = Ds1-r0002-max
</pre>

The config file is broken into three sections, the header, the pyana section and the modules section.

=== Header Section ===
The comments at the top of the file include a bsub command. This is the command to be entered from the ~/myrelease directory to queue the job for processing. Bsub will submit the job, asking for 6 CPUs (-n), naming the job r0002 (-J), placing a log file in the -o location, and so forth. There are several queues available for use, the psanacsq is just one of them. The -R option restricts the job to running on a single host, which is required for averaging. Finally, the cxi.pyana command in quotes is the command that will actually be executed on the host computer, and specifies the config file and the streams to use. Note the asterisk, which allows averaging all the streams from a single run at one time.

'''Before executing the command, replace <username> in the command and in the config file with your user name.'''

=== Pyana Section ===
Pyana operates by having a root section in the config file which specifies which modules to run. In this case we are running ''cctbx.xfel's'' mod_average module. If we were running multiple modules, we could chain them together here, and each module would be executed in sequence using the parameters specified below. We also request a number of cpus to parallelize the job with. Even though averaging must be done on a single host, on that host the processing can be split up among multiple cores. Note, other aspects of ''cctbx.xfel'', such as indexing and integration, do not have this restriction.

=== Module Section ===
The final section specifies parameters specific to averaging:
* calib_dir: this directory is provided by LCLS and includes the layout information of the 64 tiles of the CS-PAD detector
* avg_dirname: directory to place the average image
* stddev_dirname: directory to place the standard deviation image
* max_dirname: directory to place the maximum projection image
* detz_offset: experiment-specific number used to calculate the detector distance. This is the distance in millimeters between the sample interaction region and the far end of the detector stage. The XTC stream contains the distance from the detector stage to the detector. These two numbers are subtracted to calculate the distance between sample interaction region and the detector. We determine this number experimentally using a grid search, testing a range of potential detz_offsets and choosing the one that shows the greatest success at indexing images
* address: the name of the detector in the XTC stream
* avg_basename: this name is prepended to a datestamp to name the average image
* stddev_basename: this name is prepended to a datestamp to name the standard deviation image
* max_basename: this name is prepended to a datestamp to name the maximum projection image image

=== Running and testing the command ===
Submit the job as specified above, using the bsub command in the header:
$ bsub -n 6 -J r0002 -o /reg/g/cctbx/tutorials/scratch/<username>/dark_r0002.out -q psanacsq -R "span[hosts=1]"
"cxi.pyana -c ~/myrelease/dark.cfg /reg/d/ana11/cxi/data/Mar2013calib/xtc/e236-r0002-s0*.xtc"

Type bjobs to see if the job is running. When it is, follow the output thusly:
$ tail -f /reg/g/cctbx/tutorials/scratch/<username>/dark_r0002.out

When the average has been completed, the output can be inspected using the "cctbx" image viewer:
$ cctbx.image_viewer /reg/g/cctbx/tutorials/scratch/<username>/darks/Ds1-r0002-*.pickle

== Averaging an experimental light run ==

Once the dark run is made, experimental runs can be averaged to see the extent of diffraction and potential pathologies in the run. Copy the average.cfg file from the tutorials directory to your pyana directory:

$ cd ~/myrelease
$ cp /reg/g/cctbx/tutorials/preprocessing/average.cfg .

The cfg will look like this:
<pre>
# -*- Mode: Conf -*-
#
# Since mod_average must be run on a single host, it cannot be
# submitted using cxi.lsf.
#
# bsub -n 6 -J r0003 -o /reg/g/cctbx/tutorials/scratch/<username>/average_r0003.out -q psanacsq -R "span[hosts=1]" \
# "cxi.pyana -c average.cfg /reg/d/ana11/cxi/data/Mar2013calib/xtc/e236-r0003-s0*.xtc"
#

[pyana]
modules = my_ana_pkg.mod_average
num-cpu = 8

[my_ana_pkg.mod_average]
calib_dir = /reg/g/cctbx/sources/cctbx_project/xfel/metrology/CSPad/run4/CxiDs1.0_Cspad.0
dark_path = /reg/g/cctbx/tutorials/scratch/<username>/darks/<name of dark average>.pickle
dark_stddev = /reg/g/cctbx/tutorials/scratch/<username>/darks/<name of dark stddev>.pickle
avg_dirname = /reg/g/cctbx/tutorials/scratch/<username>/averages
stddev_dirname = /reg/g/cctbx/tutorials/scratch/<username>/averages
max_dirname = /reg/g/cctbx/tutorials/scratch/<username>/averages
detz_offset = 581
address = CxiDs1-0|Cspad-0
avg_basename = Ds1-r0003-avg
stddev_basename = Ds1-r0003-stddev
max_basename = Ds1-r0003-max
</pre>

The only difference is the addition of two parameters:
* dark_path: path to the average dark image created before
* dark_stddev: path to the standard deviation dark image created before

The job is ran as before using the bsub command in the header, again remembering to change all instances of <username> to your own SLAC user name. Here you must also change <name of dark average> and <name of dark stddev> to the names of the average and standard deviation images created in the previous step. The result of the run will be three new images that can be viewed with cctbx.image_viewer in the same manner as the darks were.

== Information contained in the light average ==

Firstly, the light and dark images can be used to mask of any bad pixels, as detailed below. The light images (maximum in particular) also serve as a virtual, dark-subtracted, powder pattern that can used to inspect the data for diffraction. The light maximum should also flag up obvious errors in meteorology, because rings will be non-continuous if the quadrants are misaligned. Intensity values from the corners of the light average or maximum are also useful to estimate background when taking first guesses at thresholds for hit-finding and integration.

== Creating a mask image ==
''cctbx.xfel'' uses three images to create a mask. Those images, and the purposes of them are listed below:
* Average image from a dark run: pixels ≤ 0 are considered dead, pixels > 2000 are too sensitive
* Standard deviation from a dark run: pixels ≤ 0 are considered dead, pixels ≥ 10 are too uncertain
* Maximum projection from a lighted run (''i.e.'' an experimental run): pixels < 300 are considered non-bonded or in a shadow. The presence of diffracting data is not needed, but also will not interfere.

To create the mask, execute these commands:
$ cd /reg/g/cctbx/tutorials/scratch/<username>/
$ mkdir masks
$ cd masks
$ cxi.make_mask -m 10 ../darks/<name of dark average>.pickle ../darks/<name of dark standard deviation>.pickle
../averages/<name of light maximum projection>.pickle

A new image is created where each pixel has one of two values: -2 (masked out pixels), and 0 (good pixels). Note, the maximum projection minimum pixel value is overridden here to a value that is more sensible for the tutorial's lysozyme data. You will likely need to adjust this parameter based on your data (see below for details and for further overrides).

Use the resultant mask image during processing with ''cctbx.xfel'' by adding this flag to your pyana config file:

mask_path = <path to mask file>

In addition, a polygon and a circle can be specified using the --poly_mask and --circle_mask parameters:

$ cxi.make_mask --poly_mask=17,854,76,967,760,982,762,879 --circle_mask=855,939,103 Cspad-avg.pickle Cspad-stddev.pickle Cspad-max.pickle

This example creates a mask image with the polygon with points (17,854), (76,967), (760,982), and (762,879) masked out, and the circle with center (855,939) and radius 103 masked out. Only one polygon and circle can be specified at the moment.

The mask can be examined with cctbx.image_viewer.

=== Options ===

Some additional options are available for specifying the details of the masking operation:
;-a, --avg_max (default 2000)
: specifies the maximum value in ADU a pixel in the average image can have before it's masked out. The minimum is always zero.
;-s, --stddev_max (default 10)
: specifies the maximum value in ADU a pixel in the standard deviation image can have before it's masked out. The minimum is always zero.
;-m, --maxproj_min (default 300)
: specifies the minimum value in ADU a pixel in the maximum projection image can have before it's masked out. Of the three parameters for controlling cutoffs in ''cxi.make_mask'', the -m option is unique from the -a and -s options in that it will likely vary the according to your sample's background. Carefully examine the corners of your lighted maximum projection and choose a value lower than the ADU values displayed. Also note that there is no cutoff on the high end of the maximum projection image specified in the mask, as that is defined as the saturation value for the detector.
;-x, --mask_pix_val (default -2)
: specifies the value to use when masking out a pixel. In other words, bad pixels in images in the XTC stream will be replaced with this value.

Hattne: Moved scratch-area setup to tutorial.

2014-02-05T18:51:59Z

Moved scratch-area setup to tutorial.

← Older revision		Revision as of 18:51, 5 February 2014
Line 4:		Line 4:

	Finally, inactive and non-bonded pixels, as well as hypersensitive pixels and any beam stop (if used) should be masked out. This tutorial covers averaging dark and light runs, and creating mask images.		Finally, inactive and non-bonded pixels, as well as hypersensitive pixels and any beam stop (if used) should be masked out. This tutorial covers averaging dark and light runs, and creating mask images.


			== Setup your scratch area ==

			Create a scratch folder and populate it with some directories that will be used during this tutorial, changing <username> to your SLAC account name:

			$ cd /reg/g/cctbx/tutorials/scratch/
			$ mkdir <username>
			$ cd <username>
			$ mkdir darks
			$ mkdir averages
			$ mkdir results


	== Create a dark average ==		== Create a dark average ==

Aaron: /* Averaging an experimental light run */

2014-01-29T23:38:37Z

Averaging an experimental light run

← Older revision		Revision as of 23:38, 29 January 2014
Line 114:		Line 114:
	* dark_stddev: path to the standard deviation dark image created before		* dark_stddev: path to the standard deviation dark image created before

	The job is ran as before using the bsub command in the header, again remembering to change all instances of <username> to your own SLAC user name. It can be ~~tested~~ with cctbx.image_viewer in the same manner as ~~well~~.		The job is ran as before using the bsub command in the header, again remembering to change all instances of <username> to your own SLAC user name. Here you must also change <name of dark average> and <name of dark stddev> to the names of the average and standard deviation images created in the previous step. The result of the run will be three new images that can be viewed with cctbx.image_viewer in the same manner as the darks were.

	== Information contained in the light average ==		== Information contained in the light average ==

Aaron: /* Averaging an experimental light run */

2014-01-29T23:29:21Z

Averaging an experimental light run

← Older revision		Revision as of 23:29, 29 January 2014
Line 79:		Line 79:

	$ cd ~/myrelease		$ cd ~/myrelease
	$ cp /reg/~~d/ffb/cxi/temp~~/cctbx/tutorials/preprocessing/average.cfg .		$ cp /reg/g/cctbx/tutorials/preprocessing/average.cfg .

	The cfg will look like this:		The cfg will look like this: