Difference between revisions of "Phil"

From cctbx_xfel
Jump to: navigation, search
(Metrology phil)
(One intermediate revision by one other user not shown)
Line 13: Line 13:
 
# without target_cell set.
 
# without target_cell set.
 
target_cell = 38 79 79 90 90 90
 
target_cell = 38 79 79 90 90 90
target_cell_centring_type = P
 
 
known_setting = 9
 
known_setting = 9
  
Line 65: Line 64:
 
Several lines of parameters are given, then a few parameter blocks are specified, enclosed in {} brackets.  The parameters in detail:
 
Several lines of parameters are given, then a few parameter blocks are specified, enclosed in {} brackets.  The parameters in detail:
 
* include file metrology-7.1.phil: this line specifies another ''phil'' file to include.  In this case, the included file specifies metrology corrections (see below).
 
* include file metrology-7.1.phil: this line specifies another ''phil'' file to include.  In this case, the included file specifies metrology corrections (see below).
* target_cell: the known unit cell for this sample.  In the form a, b, c, alpha, beta, gamma.  Must be in the primitive setting, or specify a target_cell_centring_typeOr, convert the dimensions first to the primitive setting with ''iotbx.lattice_symmetry''.
+
* target_cell: known unit cell for this sample.  In the form a, b, c, alpha, beta, gamma.  Must be in the primitive setting as currently implementedTherefore, if the cell is centered, convert the dimensions to the primitive setting with "iotbx.lattice_symmetry".
* target_cell_centring_type: the centering type of the Bravais lattice: P (default), C, I, R, or F.
+
* known_setting: < Nick will add a description and how to derive this soon >
* known_setting: this is the setting number of the true Bravais symmetry as output by the autoindexing procedure in ''LABELIT''.  The indexing process fundamentally determines the triclinic cell, but also a list of possible Bravais lattices.  Normally we associate the following setting numbers with the crystal systems:
+
* distl_highres_limit and force_method2_resolution_limit: only process to this resolution limit.   
**Triclinic=1
+
**Monoclinic=2
+
**Orthorhombic=5
+
**Rhombohedral=5
+
**Tetragonal=9
+
**Hexagonal=12
+
**Cubic=22
+
**However, the assumptions break down if there is pseudosymmetry, i.e., if a higher-symmetry Bravais lattice is close to the true Bravais symmetry.  In these cases, a careful inspection of the ''LABELIT'' output is necessary to choose the setting, and it may be necessary to implement new code in ''cctbx.xfel'' for this (contact the authors).
+
* distl_highres_limit and force_method2_resolution_limit: only process to this resolution limit.  [Note these are #1 and #2 of 3 resolution flags that all must be set together]
+
 
* mosaicity_limit: maximum moisicity before a frame is rejected
 
* mosaicity_limit: maximum moisicity before a frame is rejected
 
* distl_minimum_number_spots_for_indexing: indexing will not proceed unless there are at least this many good spots found on the image
 
* distl_minimum_number_spots_for_indexing: indexing will not proceed unless there are at least this many good spots found on the image
 
* Subcategory distl: parameters specific to spot finding
 
* Subcategory distl: parameters specific to spot finding
** res.outer: resolution limit for ''spotfinder'' [Noet this is #3 of 3 resolution flags that all must be set together]
+
** res.outer: resolution limit for ''spotfinder''
 
** minimum_signal_height: in units of background noise sigma, how much signal is needed for a spot
 
** minimum_signal_height: in units of background noise sigma, how much signal is needed for a spot
 
** minimum_spot_height: minimum height for a pixel to be considered a maximum (after it's determined to be signal)
 
** minimum_spot_height: minimum height for a pixel to be considered a maximum (after it's determined to be signal)
** minimum_spot_area: minimum area in pixels for each spot.  NOTE: there is a longstanding bug in Spotfinder; increment this number by +1 to get the actually-used minimum area.
+
** minimum_spot_area: minimum area in pixels for each spot
 
** spot_area_maximum_factor: in multiples of minimum spot area, how large spots are allowed to be
 
** spot_area_maximum_factor: in multiples of minimum spot area, how large spots are allowed to be
** compactness_filter: (Use False).  This is an experimental algorithm that insists that spots need to be compact; e.g., a line 4 pixels long is not an acceptable spot.  However, it turns out that many XFEL spots are not compact, so this algorithm degrades XFEL performance.
+
** compactness_filter: < Nick to add a description here >
** method2_cutoff_percentage: Controls how method2 (see the Spotfinder paper) chooses the resolution cutoff.  Synchrotron data: 25 is OK.  XFEL stills: requires low values, try 5.  Lower numbers result in  the determination of more expansive values for the resolution cutoff.
+
** method2_cutoff_percentage: < Nick to add a description here >
 
** peak_intensity_maximum_factor: a peak intensity filter
 
** peak_intensity_maximum_factor: a peak intensity filter
 
* Subcategory indexing:
 
* Subcategory indexing:
** verbose_cv: if true, correction vectors are generated.  This is verbose output required for the metrology cutoff.  Not for routine users.
+
** verbose_cv: if true, correction vectors are generated < more detail here >
 
* Subcategory: integration
 
* Subcategory: integration
 
** background_factor:
 
** background_factor:
Line 98: Line 88:
  
 
== Metrology phil ==
 
== Metrology phil ==
 
'''Update 12/11/13: quad translations and tile translations should no longer be specified in the phil file.  These live in the code for now.  We are re-working the file storage type and in the future we won't need these at all.'''
 
  
 
''cctbx.xfel'' applies 4 levels of metrology:
 
''cctbx.xfel'' applies 4 levels of metrology:
 
* Optical metrology: this is supplied by LCLS.  The pyana config file specifies a path to a directory where initial tile placements are specified.
 
* Optical metrology: this is supplied by LCLS.  The pyana config file specifies a path to a directory where initial tile placements are specified.
* Quadrant translations: adjustments to each of the 4 quadrants as a whole. ''(12/11/13: specified in the codebase)''
+
* Quadrant translations: adjustments to each of the 4 quadrants as a whole.
* Unit-pixel or whole-pixel metrology: a series of whole-pixel translations applied to each tile. ''(12/11/13: specified in the codebase)''
+
* Unit-pixel or whole-pixel metrology: a series of whole-pixel translations applied to each tile.
 
* Sub-pixel metrology: fractional corrections including translations and rotations of each tile.  Used during integration.
 
* Sub-pixel metrology: fractional corrections including translations and rotations of each tile.  Used during integration.
  
Sub-pixel metrology corrections are contained in the metrology ''phil'' file.  For this tutorial that is metrology-7.1.phil:
+
The latter two corrections are contained in the metrology ''phil'' file.  For this tutorial that is metrology-7.1.phil:
  
 
<pre>
 
<pre>
 
distl {
 
distl {
 +
  detector_format_version = "CXI 7.1"
 
   detector_tiling = None
 
   detector_tiling = None
 
   peripheral_margin = 1
 
   peripheral_margin = 1
 +
 +
  quad_translations =  2 -6  3 -6 -7  0 -1 -4
 +
  tile_translations =  2  1  1  1  1  2  0  2  3  0  0  0  4  1  3  1 \
 +
  ...
 
)
 
)
 
integration {
 
integration {
Line 129: Line 122:
 
These parameters are in detail:
 
These parameters are in detail:
 
* distl subsection: quadrant translations and single-pixel tile translations
 
* distl subsection: quadrant translations and single-pixel tile translations
 +
** detector_format_version: deprecated.
 
** detector_tiling: < add >
 
** detector_tiling: < add >
 
** peripheral_margin: how many pixels to leave as a border around the final image
 
** peripheral_margin: how many pixels to leave as a border around the final image
 +
** quad_translations: set of x/y adjustmants to each quadrant
 +
** tile_translations: set of x/y adjustments to each tile (128 total)
 
* integration subsection: sub-pixel adjustments applied during reflection integration:
 
* integration subsection: sub-pixel adjustments applied during reflection integration:
 
** subpixel_joint_model: root tag for the sub-pixel corrections
 
** subpixel_joint_model: root tag for the sub-pixel corrections
 
*** rotations: angles in degrees for each tile (64)
 
*** rotations: angles in degrees for each tile (64)
 
*** translation: x/y translations for each tile (128)
 
*** translation: x/y translations for each tile (128)

Revision as of 02:35, 4 October 2013

As described in the overview, phil files contain parameters used during hitfinding, indexing and integration. This tutorial uses two phil files during the indexing and integration step: Ls04-lysozyme.phil and metrology-7.1.phil. The former specifies parameters specific to the processing run, while the latter specifies whole-pixel and sub-pixel metrology corrections applied to the 64 CSPAD sensor tiles.

Hitfinding/indexing/integration phil file

For the tutorial, Ls04-lysozyme.phil, stored in the /reg/d/ffb/cxi/temp/cctbx/tutorials/indexing directory but copied to your ~/myrelease directory during use, contains configuration settings we worked out that best process this data. The file will look like this:

# -*- mode: Conf -*-

include file metrology-7.1.phil

# From looking at 35 images integrated with detz_offset = 581 and
# without target_cell set.
target_cell = 38 79 79 90 90 90
known_setting = 9

distl_highres_limit = 2.0
force_method2_resolution_limit = 2.0

mosaicity_limit = 1

# Set to True to pick up second lattice, if present.
#outlier_detection_switch = True

# TEST
distl_minimum_number_spots_for_indexing = 20

distl {
  res.outer = 2.0
  minimum_signal_height = 5
  #minimum_spot_height = 10
  minimum_spot_height = 5
  minimum_spot_area = 1
  spot_area_maximum_factor = 20
  compactness_filter = False
  #method2_cutoff_percentage = 5
  method2_cutoff_percentage = 2.5

  # Avoids intensity filter.
  #peak_intensity_maximum_factor = 10000
  peak_intensity_maximum_factor = 100
}

indexing {
  # Set to True to generate correction vectors.
  verbose_cv = True
}

integration {
  background_factor = 2

  #detector_gain = 7.5
  detector_gain = 1.0

  #model = use_case_3_simulated_annealing_7
  model = user_supplied

  signal_penetration = 0.5
  #spot_shape_verbose = True
  spotfinder_subset = spots_non-ice
}

Several lines of parameters are given, then a few parameter blocks are specified, enclosed in {} brackets. The parameters in detail:

  • include file metrology-7.1.phil: this line specifies another phil file to include. In this case, the included file specifies metrology corrections (see below).
  • target_cell: known unit cell for this sample. In the form a, b, c, alpha, beta, gamma. Must be in the primitive setting as currently implemented. Therefore, if the cell is centered, convert the dimensions to the primitive setting with "iotbx.lattice_symmetry".
  • known_setting: < Nick will add a description and how to derive this soon >
  • distl_highres_limit and force_method2_resolution_limit: only process to this resolution limit.
  • mosaicity_limit: maximum moisicity before a frame is rejected
  • distl_minimum_number_spots_for_indexing: indexing will not proceed unless there are at least this many good spots found on the image
  • Subcategory distl: parameters specific to spot finding
    • res.outer: resolution limit for spotfinder
    • minimum_signal_height: in units of background noise sigma, how much signal is needed for a spot
    • minimum_spot_height: minimum height for a pixel to be considered a maximum (after it's determined to be signal)
    • minimum_spot_area: minimum area in pixels for each spot
    • spot_area_maximum_factor: in multiples of minimum spot area, how large spots are allowed to be
    • compactness_filter: < Nick to add a description here >
    • method2_cutoff_percentage: < Nick to add a description here >
    • peak_intensity_maximum_factor: a peak intensity filter
  • Subcategory indexing:
    • verbose_cv: if true, correction vectors are generated < more detail here >
  • Subcategory: integration
    • background_factor:
    • detector_gain: ADU units per photon
    • model: labelit has several integration models, and allows users to provide their own. The model listed here is custom for these xfel applications
    • signal_penetration: thickness of the CSPAD sensors.
    • spotfinder_subset: which spots found by spotfinder to use. Choose from: < add >

Metrology phil

cctbx.xfel applies 4 levels of metrology:

  • Optical metrology: this is supplied by LCLS. The pyana config file specifies a path to a directory where initial tile placements are specified.
  • Quadrant translations: adjustments to each of the 4 quadrants as a whole.
  • Unit-pixel or whole-pixel metrology: a series of whole-pixel translations applied to each tile.
  • Sub-pixel metrology: fractional corrections including translations and rotations of each tile. Used during integration.

The latter two corrections are contained in the metrology phil file. For this tutorial that is metrology-7.1.phil:

distl {
  detector_format_version = "CXI 7.1"
  detector_tiling = None
  peripheral_margin = 1

  quad_translations =  2 -6  3 -6 -7  0 -1 -4
  tile_translations =  2  1  1  1  1  2  0  2  3  0  0  0  4  1  3  1 \
  ...
)
integration {
  # This is L785_119.
  subpixel_joint_model {
    rotations = \
       0.1056719175    0.1080664532  -0.2151669703  -0.2099097279   \
       ...
    translations = \
       0.2700895641   -0.6117579705  -0.5111648376  -0.8676594815   \
       ...
  }
}

These parameters are in detail:

  • distl subsection: quadrant translations and single-pixel tile translations
    • detector_format_version: deprecated.
    • detector_tiling: < add >
    • peripheral_margin: how many pixels to leave as a border around the final image
    • quad_translations: set of x/y adjustmants to each quadrant
    • tile_translations: set of x/y adjustments to each tile (128 total)
  • integration subsection: sub-pixel adjustments applied during reflection integration:
    • subpixel_joint_model: root tag for the sub-pixel corrections
      • rotations: angles in degrees for each tile (64)
      • translation: x/y translations for each tile (128)