Phil

From cctbx_xfel
Jump to navigation Jump to search

As described in the overview, phil files contain parameters used during hitfinding, indexing and integration. This tutorial uses two phil files during the indexing and integration step: Ls04-lysozyme.phil and metrology-7.1.phil. The former specifies parameters specific to the processing run, while the latter specifies whole-pixel and sub-pixel metrology corrections applied to the 64 CSPAD sensor tiles.

Hitfinding/indexing/integration phil file

For the tutorial, Ls04-lysozyme.phil, stored in the /reg/d/ffb/cxi/temp/cctbx/tutorials/indexing directory but copied to your ~/myrelease directory during use, contains configuration settings we worked out that best process this data. The file will look like this: 

# -*- mode: Conf -*-

include file metrology-7.1.phil

# From looking at 35 images integrated with detz_offset = 581 and
# without target_cell set.
target_cell = 38 79 79 90 90 90
target_cell_centring_type = P
known_setting = 9

distl_highres_limit = 2.0
force_method2_resolution_limit = 2.0

mosaicity_limit = 1

# Set to True to pick up second lattice, if present.
#outlier_detection_switch = True

# TEST
distl_minimum_number_spots_for_indexing = 20

distl {
  res.outer = 2.0
  minimum_signal_height = 5
  #minimum_spot_height = 10
  minimum_spot_height = 5
  minimum_spot_area = 1
  spot_area_maximum_factor = 20
  compactness_filter = False
  #method2_cutoff_percentage = 5
  method2_cutoff_percentage = 2.5

  # Avoids intensity filter.
  #peak_intensity_maximum_factor = 10000
  peak_intensity_maximum_factor = 100
}

indexing {
  # Set to True to generate correction vectors.
  verbose_cv = True
}

integration {
  background_factor = 2

  #detector_gain = 7.5
  detector_gain = 1.0

  #model = use_case_3_simulated_annealing_7
  model = user_supplied

  signal_penetration = 0.5
  #spot_shape_verbose = False [This time consuming option prints out a detailed shape analysis of every spot]
  spotfinder_subset = spots_non-ice
}

Several lines of parameters are given, then a few parameter blocks are specified, enclosed in {} brackets. The parameters in detail:

  • include file metrology-7.1.phil: this line specifies another phil file to include. In this case, the included file specifies metrology corrections (see below).
  • target_cell: the known unit cell for this sample. In the form a, b, c, alpha, beta, gamma. Must be in the primitive setting, or specify a target_cell_centring_type. Or, convert the dimensions first to the primitive setting with iotbx.lattice_symmetry.
  • target_cell_centring_type: the centering type of the Bravais lattice: P (default), C, I, R, or F.
  • known_setting: this is the setting number of the true Bravais symmetry as output by the autoindexing procedure in LABELIT. The indexing process fundamentally determines the triclinic cell, but also a list of possible Bravais lattices. Normally we associate the following setting numbers with the crystal systems:
    • Triclinic=1
    • Monoclinic=2
    • Orthorhombic=5
    • Rhombohedral=5
    • Tetragonal=9
    • Hexagonal=12
    • Cubic=22
    • However, the assumptions break down if there is pseudosymmetry, i.e., if a higher-symmetry Bravais lattice is close to the true Bravais symmetry. In these cases, a careful inspection of the LABELIT output is necessary to choose the setting, and it may be necessary to implement new code in cctbx.xfel for this (contact the authors).
  • distl_highres_limit and force_method2_resolution_limit: only process to this resolution limit. [Note these are #1 and #2 of 3 resolution flags that all must be set together]
  • mosaicity_limit: maximum moisicity before a frame is rejected
  • distl_minimum_number_spots_for_indexing: indexing will not proceed unless there are at least this many good spots found on the image
  • Subcategory distl: parameters specific to spot finding. NOTE: More extensive documentation on spotfinder and other parameters is at the spot finder web site
    • res.outer: resolution limit for spotfinder [Note this is #3 of 3 resolution flags that all must be set together]
    • minimum_signal_height: in units of background noise sigma, how much signal is needed for a spot
    • minimum_spot_height: minimum height for a pixel to be considered a maximum (after it's determined to be signal)
    • minimum_spot_area: minimum area in pixels for each spot. NOTE: there is a longstanding bug in Spotfinder; increment this number by +1 to get the actually-used minimum area.
    • spot_area_maximum_factor: in multiples of minimum spot area, how large spots are allowed to be
    • compactness_filter: (Use False). This is an experimental algorithm that insists that spots need to be compact; e.g., a line 4 pixels long is not an acceptable spot. However, it turns out that many XFEL spots are not compact, so this algorithm degrades XFEL performance.
    • method2_cutoff_percentage: Controls how method2 (see the Spotfinder paper) chooses the resolution cutoff. Synchrotron data: 25 is OK. XFEL stills: requires low values, try 5. Lower numbers result in the determination of more expansive values for the resolution cutoff.
    • peak_intensity_maximum_factor: a peak intensity filter
  • Subcategory indexing:
    • verbose_cv: if true, correction vectors are generated. This is verbose output required for the metrology cutoff. Not for routine users.
  • Subcategory integration:
    • background_factor: ratio of # pixels used for background subtraction : # pixels in spot
    • detector_gain: ADU units per photon
    • model: labelit has several integration models, and allows users to provide their own. The model listed here is custom for these xfel applications
    • signal_penetration: thickness of the CSPAD sensors.
    • spotfinder_subset: which spots found by spotfinder to use. Choose from: < add >

Metrology phil

Update 12/11/13: quad translations and tile translations should no longer be specified in the phil file. These live in the code for now. We are re-working the file storage type and in the future we won't need these at all.

cctbx.xfel applies 4 levels of metrology:

  • Optical metrology: this is supplied by LCLS. The pyana config file specifies a path to a directory where initial tile placements are specified.
  • Quadrant translations: adjustments to each of the 4 quadrants as a whole. (12/11/13: specified in the codebase)
  • Unit-pixel or whole-pixel metrology: a series of whole-pixel translations applied to each tile. (12/11/13: specified in the codebase)
  • Sub-pixel metrology: fractional corrections including translations and rotations of each tile. Used during integration.

Sub-pixel metrology corrections are contained in the metrology phil file. For this tutorial that is metrology-7.1.phil: 

distl {
  detector_tiling = None
  peripheral_margin = 1
)
integration {
  # This is L785_119.
  subpixel_joint_model {
    rotations = \
       0.1056719175    0.1080664532  -0.2151669703  -0.2099097279   \
       ...
    translations = \
       0.2700895641   -0.6117579705  -0.5111648376  -0.8676594815   \
       ...
  }
}

These parameters are in detail:

  • distl subsection: quadrant translations and single-pixel tile translations
    • detector_tiling: < add >
    • peripheral_margin: how many pixels to leave as a border around the final image
  • integration subsection: sub-pixel adjustments applied during reflection integration:
    • subpixel_joint_model: root tag for the sub-pixel corrections
      • rotations: angles in degrees for each tile (64)
      • translation: x/y translations for each tile (128)

Comparing two phil files

Often phil files are edited repeatedly by users. cxi.mpi_submit saves copies of phil files as they are used as a record of which trials were processed using which parameters. Now, say you have two different phil files that you want to see the differences between them. Due to comments or rearrangements, diffing the two files isn't making your life easier. Do this: 

$ cxi.parameters > default.phil
$ libtbx.phil --diff default.phil file1.phil > a.txt
$ libtbx.phil --diff default.phil file2.phil > b.txt
$ diff a.txt b.txt

This will show the true differences between the phil files. Note also that cxi.parameters is a useful tool for seeing all the default parameters available in cctbx.xfel.

Retrieved from "http://viper.lbl.gov:8080/cctbx.xfel/index.php?title=Phil&oldid=976"