Difference between revisions of "Cppxfel Initial Orientation Matrix Refinement"

From cctbx_xfel
Jump to: navigation, search
Line 115: Line 115:
 
== Final integration summary table ==
 
== Final integration summary table ==
  
The initial orientation matrix refinement commands finish with a summary table detailing the main results for each crystal and each image.
+
The initial orientation matrix refinement commands finish with a summary table detailing the main results for each crystal and each image. It will start with something like this:
 +
 
 +
<pre>
 +
Filename Refl num Score Horiz rot Vert rot Beam rot Stdev Wavelength Distance UnitCellA UnitCellB UnitCellC
 +
shot-s00-20130316165156302.img 111 8.91892 0 0 0 0.0260573 1.45691 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165043053.img 80 6.33333 -1 1 0 0.0240737 1.45647 91.002 105.918 105.918 105.918
 +
shot-s01-20130316165128672.img 14 4.28571 -1.11914 -1.11523 0 0.0170082 1.45962 91.0018 105.918 105.918 105.918
 +
shot-s01-20130316165047878.img 87 1.2069 -0.625 0.320312 0 0.0105548 1.45852 91.0018 105.918 105.918 105.918
 +
shot-s01-20130316165035212.img 21 9.47368 -1.06836 0.908203 0 0.0209798 1.45496 91.0018 105.918 105.918 105.918
 +
shot-s01-20130316165033837.img 37 6.77419 -0.00390625 0.533203 0 0.0239777 1.45621 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165336174.img 188 1.45161 -0.365234 0.371094 0 0.0104029 1.45705 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165131305.img 171 1.06509 0.212891 0.621094 0 0.0107561 1.4579 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165041011.img 26 6.11111 -0.445312 -0.298828 0 0.0223847 1.45643 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165126972.img 211 0.972222 -0.00585938 0.609375 0 0.00877267 1.45685 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165126972.img 47 7.875 0.183594 1.97266 0 0.0240756 1.45685 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165138553.img 41 9.28571 -0.365234 0.625 0 0.0253017 1.45673 91.0018 105.918 105.918 105.918
 +
shot-s01-20130316165050712.img 19 6.75 0.0214844 2.17578 0 0.0230937 1.45894 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165215513.img 146 2.02128 -0.498047 0.240234 0 0.010419 1.45674 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165133179.img 143 2.36301 -0.376953 0.283203 0 0.0129344 1.45528 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165143844.img 142 1.99301 -0.132812 0.554688 0 0.0117068 1.45789 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165231472.img 6 12.5 1 1.875 0 0.013229 1.45868 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165052512.img 67 8.23944 -0.365234 0.482422 0 0.0251099 1.45578 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165052512.img 168 1.86335 -0.458984 0.542969 0 0.01226 1.45578 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165125972.img 200 1.6129 -0.388672 0.638672 0 0.010696 1.456 91.0018 105.918 105.918 105.918
 +
shot-s00-20130316165052178.img 217 1.70616 -0.1875 0.6875 0 0.0116042 1.45657 91.0018 105.918 105.918 105.918
 +
</pre>
 +
 
 +
The "score" column is the result of the minimisation function used to refine the horizontal and vertical rotations, which are listed in the following two columns. In general, a score of 3 or less is good, though will be higher for low resolution data.
  
 
== Output files ==
 
== Output files ==

Revision as of 17:44, 24 November 2015

Initial orientation matrix refinement covers the first stage of the dedicated C++ code in cppxfel and uses an input file which is called integrate.txt by default. This should have been generated during the previous stage.

The contents of integrate.txt

One can look at the contents of integrate.txt by executing:

cat integrate.txt

This shows an input script along the following lines. This input script is divided into a number of parameters, followed by the line COMMANDS, after which the task(s) for cppxfel to complete are listed. In this case, cppxfel should INTEGRATE the data.

ORIENTATION_MATRIX_LIST matrices.dat
NEW_MATRIX_LIST orientations.dat

SPACE_GROUP 197
UNIT_CELL 105.917757854 105.917757854 105.917757854 90.0 90.0 90.0
FIX_UNIT_CELL ON

MM_PER_PIXEL 0.11
BEAM_CENTRE 881.755 881.5075
DETECTOR_DISTANCE 91.0018
INTEGRATION_WAVELENGTH 1.45738121124 

PANEL_LIST panels.txt
METROLOGY_SEARCH_SIZE 1

SHOEBOX_FOREGROUND_RADIUS 1
SHOEBOX_NEITHER_RADIUS 2
SHOEBOX_BACKGROUND_RADIUS 3

INTENSITY_THRESHOLD 12
ABSOLUTE_INTENSITY OFF
REFINE_ORIENTATIONS ON

VERBOSITY_LEVEL 1

COMMANDS

INTEGRATE
The matrix information from indexing is passed through using the generated file
matrices.dat
under the keyword ORIENTATION_MATRIX_LIST which was produced in the previous stage. One can see the contents of matrices.dat:
head -20 matrices.dat
image shot-s00-20130316165156302
wavelength 1.45690748941
distance 91.0018
centre 881.755 881.5075
unitcell 0.00945122403519 0.0 0.0 -5.78701450245e-19 0.00945122403519 0.0 -5.78701450245e-19 -1.51989185671e-18 0.00945122403519 
rotation 0.209524357745 -0.976621953811 0.0480531252429 -0.360236467579 -0.122785692599 -0.924745024923 0.909026529282 0.176446119331 -0.377541436183 
image shot-s01-20130316165035212
wavelength 1.45496194705
distance 91.0018
centre 881.755 881.5075
unitcell 0.00944171966033 0.0 0.0 7.80781215281e-18 0.00944171966033 0.0 3.61484632923e-18 2.67460240614e-18 0.00944171966033 
rotation -0.351867835054 0.0121912864248 0.935970298241 -0.852741351745 -0.416546104403 -0.315153184867 0.386032658825 -0.909032846184 0.156965190022 
image shot-s00-20130316165041011
wavelength 1.45643207192
distance 91.0018
centre 881.755 881.5075
unitcell 0.00943218298265 0.0 0.0 1.51682978341e-18 0.00943218298265 0.0 3.61119512738e-18 5.77535560549e-19 0.00943218298265 
rotation -0.21213292526 0.320476950034 0.92319778299 -0.956128815937 -0.263369266933 -0.128274380022 0.202032941263 -0.909907222596 0.362286539789 
image shot-s00-20130316165215513
wavelength 1.45673961831
distance 91.0018

The image tag has the name of the image file minus the extension. The values for the detector distance, beam centre and wavelength are over-ridden by the values in the integrate.txt file if they are present, but the over-riding values can be removed if the values vary from one image to another. The orientation matrix is constructed from the unit cell matrix and the rotation matrix listed.

Executing integrate.txt

This integration input file can be executed through cppxfel using the following command:

cppxfel.run -i integrate.txt

Wavelength histograms

The output tracks individual images which execute on different threads as their orientation matrices are refined. An example of a wavelength histogram created is shown below. These wavelength histograms were first described in Ginn et al, Nat Comms 2015.

Wavelength histogram for shot-s00-20130316165134721.img
1.41322	.
1.41759	
1.42196	
1.42633	.
1.4307	.
1.43508	..
1.43945	
1.44382	..
1.44819	.....
1.45256	.........
1.45693	..................
1.4613	......................................................................
1.46567	...............................................
1.47004	.....
1.47441	.......
1.47878	...
1.48315	.
1.48752	.
1.4919	..
1.49627	

These integration peaks should be as long and narrow as possible, depending on the extent of over-prediction as determined by the parameter OVER_PRED_BANDWIDTH.

Final integration summary table

The initial orientation matrix refinement commands finish with a summary table detailing the main results for each crystal and each image. It will start with something like this:

Filename	Refl num	Score	Horiz rot	Vert rot	Beam rot	Stdev	Wavelength	Distance	UnitCellA	UnitCellB	UnitCellC
shot-s00-20130316165156302.img	111	8.91892	0	0	0	0.0260573	1.45691	91.0018	105.918	105.918	105.918
shot-s00-20130316165043053.img	80	6.33333	-1	1	0	0.0240737	1.45647	91.002	105.918	105.918	105.918
shot-s01-20130316165128672.img	14	4.28571	-1.11914	-1.11523	0	0.0170082	1.45962	91.0018	105.918	105.918	105.918
shot-s01-20130316165047878.img	87	1.2069	-0.625	0.320312	0	0.0105548	1.45852	91.0018	105.918	105.918	105.918
shot-s01-20130316165035212.img	21	9.47368	-1.06836	0.908203	0	0.0209798	1.45496	91.0018	105.918	105.918	105.918
shot-s01-20130316165033837.img	37	6.77419	-0.00390625	0.533203	0	0.0239777	1.45621	91.0018	105.918	105.918	105.918
shot-s00-20130316165336174.img	188	1.45161	-0.365234	0.371094	0	0.0104029	1.45705	91.0018	105.918	105.918	105.918
shot-s00-20130316165131305.img	171	1.06509	0.212891	0.621094	0	0.0107561	1.4579	91.0018	105.918	105.918	105.918
shot-s00-20130316165041011.img	26	6.11111	-0.445312	-0.298828	0	0.0223847	1.45643	91.0018	105.918	105.918	105.918
shot-s00-20130316165126972.img	211	0.972222	-0.00585938	0.609375	0	0.00877267	1.45685	91.0018	105.918	105.918	105.918
shot-s00-20130316165126972.img	47	7.875	0.183594	1.97266	0	0.0240756	1.45685	91.0018	105.918	105.918	105.918
shot-s00-20130316165138553.img	41	9.28571	-0.365234	0.625	0	0.0253017	1.45673	91.0018	105.918	105.918	105.918
shot-s01-20130316165050712.img	19	6.75	0.0214844	2.17578	0	0.0230937	1.45894	91.0018	105.918	105.918	105.918
shot-s00-20130316165215513.img	146	2.02128	-0.498047	0.240234	0	0.010419	1.45674	91.0018	105.918	105.918	105.918
shot-s00-20130316165133179.img	143	2.36301	-0.376953	0.283203	0	0.0129344	1.45528	91.0018	105.918	105.918	105.918
shot-s00-20130316165143844.img	142	1.99301	-0.132812	0.554688	0	0.0117068	1.45789	91.0018	105.918	105.918	105.918
shot-s00-20130316165231472.img	6	12.5	1	1.875	0	0.013229	1.45868	91.0018	105.918	105.918	105.918
shot-s00-20130316165052512.img	67	8.23944	-0.365234	0.482422	0	0.0251099	1.45578	91.0018	105.918	105.918	105.918
shot-s00-20130316165052512.img	168	1.86335	-0.458984	0.542969	0	0.01226	1.45578	91.0018	105.918	105.918	105.918
shot-s00-20130316165125972.img	200	1.6129	-0.388672	0.638672	0	0.010696	1.456	91.0018	105.918	105.918	105.918
shot-s00-20130316165052178.img	217	1.70616	-0.1875	0.6875	0	0.0116042	1.45657	91.0018	105.918	105.918	105.918

The "score" column is the result of the minimisation function used to refine the horizontal and vertical rotations, which are listed in the following two columns. In general, a score of 3 or less is good, though will be higher for low resolution data.

Output files

The important files created by running integrate.txt are:

  • A new set of orientation matrices which have been refined under three file names:
  • * integrate-orientations.dat - this file is to be provided to input files which use the INTEGRATE command, such as integrate.txt generated by cppxfel.input_gen
  • * refine-orientations.dat - this file is to be provided to input files which use the REFINE_PARTIALITY command, such as refine.txt generated by cppxfel.input_gen
  • * merge-orientations.dat - this file is to be provided to input files which use the MERGE command, such as merge.txt generated by cppxfel.input_gen.
  • MTZ files following the format img*.mtz, which contain over-predicted reflection intensities. The first crystal will end in _0.mtz, the second with _1.mtz and so on for the number of lattices indexed in one image.
  • shot*.dat files, containing tab-delimited information on the coordinates and intensities of each reflection.

Once these files are generated, it is time to do Cppxfel Post-refinement.