Helenginn: Starting "indexing with DIALS" page

2016-05-04T00:58:54Z

Starting "indexing with DIALS" page

New page

This covers the generation of indexing solutions using DIALS within the [[Cppxfel|''cppxfel'' distribution]]. This assumes a successful [[Cppxfel Installation | installation of cppxfel]] including the DIALS dependency.

== Downloading test data ==

Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.

<pre>
tar zxvf ginn_jac_cpv17.tgz
</pre>

== Running DIALS on the set of 1000 images ==

The scripts included in ''cppxfel'' to run DIALS uses four cores by default. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:

<pre>
export NSLOTS=16
</pre>

DIALS uses the options found in the two files <code>find_spots.options</code> and <code>index.options</code> for spotfinding/indexing the data. These should be generated for <code>find_spots.options</code> with these parameters:

<pre>
cat > find_spots.options << EOF
gain=14.0 min_spot_size=1 global_threshold=100
EOF
</pre>

Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=1 global_threshold=100</code>.

This should be similarly created for <code>index.options</code>. This is an appropriate options file for a DIALS distribution built after 20th Feb:

<pre>
cat > index.options << EOF
unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills
EOF
</pre>

Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills</code>

The DIALS scripts can now be run as follows:

<pre>
cppxfel.run_dials shot*.pickle
</pre>

== Assessing the output of DIALS indexing ==

The output from each DIALS find_spots and indexing event are stored in the <code>*find_spots.log</code> and <code>*index.log</code> files for each image name. The strong spots found in each individual image are stored in <code>_*_strong.pickle</code> for each respective image name. Any successfully indexed images follow the format <code>_*_experiments.json</code>. The number of indexed images can be roughly counted as so:

<pre>
find _*_experiments.json | wc -l
</pre>

== Converting the results of DIALS indexing for ''cppxfel''==

The images and matrices indexed by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:

<pre>
cppxfel.input_gen
</pre>

This runs on the number of cores specified by the <code>NSLOTS</code> environment variable to extract the appropriate image data, and also generates several input files: <code>panels.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code>, <code>merge.txt</code>, which will be explained in the following tutorials. The indexed images are extracted to <code>.img</code> files. This also produces a <code>matrices.dat</code> file which contains all the indexing solutions generated by DIALS.

Next step: [[Cppxfel Initial Orientation Matrix Refinement]]

Cppxfel Indexing with DIALS - Revision history

Helenginn: Starting "indexing with DIALS" page