Phil

As described in the overview, phil files contain parameters used during hitfinding, indexing and integration. This tutorial uses two phil files during the indexing and integration step: Ls04-lysozyme.phil and metrology-7.1.phil. The former specifies parameters specific to the processing run, while the latter specifies whole-pixel and sub-pixel metrology corrections applied to the 64 CSPAD sensor tiles.

Hitfinding/indexing/integration phil file

For the tutorial, Ls04-lysozyme.phil, stored in the /reg/d/ffb/cxi/temp/cctbx/tutorials/indexing directory but copied to your ~/myrelease directory during use, contains configuration settings we worked out that best process this data. The file will look like this:

# -*- mode: Conf -*-

include file metrology-7.1.phil

# From looking at 35 images integrated with detz_offset = 581 and
# without target_cell set.
target_cell = 38 79 79 90 90 90
target_cell_centring_type = P
known_setting = 9

distl_highres_limit = 2.0
force_method2_resolution_limit = 2.0

mosaicity_limit = 1

# Set to True to pick up second lattice, if present.
#outlier_detection_switch = True

# TEST
distl_minimum_number_spots_for_indexing = 20

distl {
  res.outer = 2.0
  minimum_signal_height = 5
  #minimum_spot_height = 10
  minimum_spot_height = 5
  minimum_spot_area = 1
  spot_area_maximum_factor = 20
  compactness_filter = False
  #method2_cutoff_percentage = 5
  method2_cutoff_percentage = 2.5

  # Avoids intensity filter.
  #peak_intensity_maximum_factor = 10000
  peak_intensity_maximum_factor = 100
}

indexing {
  # Set to True to generate correction vectors.
  verbose_cv = True
}

integration {
  background_factor = 2

  #detector_gain = 7.5
  detector_gain = 1.0

  #model = use_case_3_simulated_annealing_7
  model = user_supplied

  signal_penetration = 0.5
  #spot_shape_verbose = False [This time consuming option prints out a detailed shape analysis of every spot]
  spotfinder_subset = spots_non-ice
}

Several lines of parameters are given, then a few parameter blocks are specified, enclosed in {} brackets. The parameters in detail:

• include file metrology-7.1.phil: this line specifies another phil file to include. In this case, the included file specifies metrology corrections (see below).
• target_cell: the known unit cell for this sample. In the form a, b, c, alpha, beta, gamma. Must be in the primitive setting, or specify a target_cell_centring_type. Or, convert the dimensions first to the primitive setting with iotbx.lattice_symmetry.
• target_cell_centring_type: the centering type of the Bravais lattice: P (default), C, I, R, or F.
• known_setting: this is the setting number of the true Bravais symmetry as output by the autoindexing procedure in LABELIT. The indexing process fundamentally determines the triclinic cell, but also a list of possible Bravais lattices. Normally we associate the following setting numbers with the crystal systems:
  • Triclinic=1
  • Monoclinic=2
  • Orthorhombic=5
  • Rhombohedral=5
  • Tetragonal=9
  • Hexagonal=12
  • Cubic=22
  • However, the assumptions break down if there is pseudosymmetry, i.e., if a higher-symmetry Bravais lattice is close to the true Bravais symmetry. In these cases, a careful inspection of the LABELIT output is necessary to choose the setting, and it may be necessary to implement new code in cctbx.xfel for this (contact the authors).
• distl_highres_limit and force_method2_resolution_limit: only process to this resolution limit. [Note these are #1 and #2 of 3 resolution flags that all must be set together]
• mosaicity_limit: maximum moisicity before a frame is rejected
• distl_minimum_number_spots_for_indexing: indexing will not proceed unless there are at least this many good spots found on the image
• Subcategory distl: parameters specific to spot finding. NOTE: More extensive documentation on spotfinder and other parameters is at the spot finder web site
  • res.outer: resolution limit for spotfinder [Note this is #3 of 3 resolution flags that all must be set together]
  • minimum_signal_height: in units of background noise sigma, how much signal is needed for a spot
  • minimum_spot_height: minimum height for a pixel to be considered a maximum (after it's determined to be signal)
  • minimum_spot_area: minimum area in pixels for each spot. NOTE: there is a longstanding bug in Spotfinder; increment this number by +1 to get the actually-used minimum area.
  • spot_area_maximum_factor: in multiples of minimum spot area, how large spots are allowed to be
  • compactness_filter: (Use False). This is an experimental algorithm that insists that spots need to be compact; e.g., a line 4 pixels long is not an acceptable spot. However, it turns out that many XFEL spots are not compact, so this algorithm degrades XFEL performance.
  • method2_cutoff_percentage: Controls how method2 (see the Spotfinder paper) chooses the resolution cutoff. Synchrotron data: 25 is OK. XFEL stills: requires low values, try 5. Lower numbers result in the determination of more expansive values for the resolution cutoff.
  • peak_intensity_maximum_factor: a peak intensity filter
• Subcategory indexing:
  • verbose_cv: if true, correction vectors are generated. This is verbose output required for the metrology cutoff. Not for routine users.
• Subcategory integration:
  • background_factor: ratio of # pixels used for background subtraction : # pixels in spot
  • detector_gain: ADU units per photon
  • model: labelit has several integration models, and allows users to provide their own. The model listed here is custom for these xfel applications
  • signal_penetration: thickness of the CSPAD sensors.
  • spotfinder_subset: which spots found by spotfinder to use. Choose from: < add >

Metrology phil

Update 12/11/13: quad translations and tile translations should no longer be specified in the phil file. These live in the code for now. We are re-working the file storage type and in the future we won't need these at all.

cctbx.xfel applies 4 levels of metrology:

• Optical metrology: this is supplied by LCLS. The pyana config file specifies a path to a directory where initial tile placements are specified.
• Quadrant translations: adjustments to each of the 4 quadrants as a whole. (12/11/13: specified in the codebase)
• Unit-pixel or whole-pixel metrology: a series of whole-pixel translations applied to each tile. (12/11/13: specified in the codebase)
• Sub-pixel metrology: fractional corrections including translations and rotations of each tile. Used during integration.

Sub-pixel metrology corrections are contained in the metrology phil file. For this tutorial that is metrology-7.1.phil:

distl {
  detector_tiling = None
  peripheral_margin = 1
)
integration {
  # This is L785_119.
  subpixel_joint_model {
    rotations = \
       0.1056719175    0.1080664532  -0.2151669703  -0.2099097279   \
       ...
    translations = \
       0.2700895641   -0.6117579705  -0.5111648376  -0.8676594815   \
       ...
  }
}

These parameters are in detail:

• distl subsection: quadrant translations and single-pixel tile translations
  • detector_tiling: < add >
  • peripheral_margin: how many pixels to leave as a border around the final image
• integration subsection: sub-pixel adjustments applied during reflection integration:
  • subpixel_joint_model: root tag for the sub-pixel corrections
    • rotations: angles in degrees for each tile (64)
    • translation: x/y translations for each tile (128)