http://viper.lbl.gov:8080/cctbx.xfel/api.php?action=feedcontributions&user=Helenginn&feedformat=atomcctbx_xfel - User contributions [en]2024-03-29T15:51:55ZUser contributionsMediaWiki 1.23.1http://viper.lbl.gov:8080/cctbx.xfel/index.php/CppxfelCppxfel2016-06-08T12:23:02Z<p>Helenginn: published paper link</p>
<hr />
<div>This page is under development.<br />
<br />
''cppxfel'' is a software suite primarily designed to showcase some methods which would benefit from being incorporated into larger XFEL-specific software suites. This suite currently covers the part of the XFEL data processing pipeline which stems from refining indexing solutions to generating merged MTZ files for use in structure refinement. These wiki files cover installation, step-by-step tutorials and a reference manual for each of the commands. This is meant to be read in conjunction with a paper ([http://scripts.iucr.org/cgi-bin/paper?s1600576716006981 Ginn et al J Appl Cryst 2016]).<br />
<br />
== Installation ==<br />
<br />
Download and installation instructions for cppxfel can be found [[Cppxfel Installation | here]]. ''cppxfel'' relies on a distribution of the ''cctbx'' library and Diffraction Integration for Advanced Light Sources (''DIALS''). In case you wish to update your current version, please follow the [[Updating cppxfel | updating instructions]]. Any major changes added to the cppxfel distribution is documented in this [[Cppxfel change log | change log]].<br />
<br />
== Step-by-step tutorials ==<br />
<br />
Step-by-step tutorials are provided for a 1000-image downloadable data set for CPV17 (structure solution described in [http://dx.doi.org/10.1038/ncomms7435 Ginn et al Nat. Comms. 2015]). The tutorials are divided into several stages:<br />
<br />
* [[Cppxfel Indexing with DIALS | Indexing with DIALS]] - cppxfel uses scripts to call on DIALS indexing programs to index individual images.<br />
* [[Cppxfel Indexing | Indexing with ''cppxfel'']] - cppxfel's own indexing algorithm, TakeTwo<br />
* [[Cppxfel Initial Orientation Matrix Refinement]] - performing initial refinement to produce high quality orientation matrices before post-refinement.<br />
* [[Cppxfel Post-refinement]] - post-refinement of crystals against a reference data set generated from the previous stage<br />
* [[Cppxfel Final Merge]] - producing a final MTZ file for use in structure solution.<br />
* [[Cppxfel Statistics]] - how to assess the quality of the XFEL data processing.<br />
<br />
== Detailed tutorials ==<br />
<br />
These are some more detailed tutorials on how to make decisions in the ''cppxfel'' pipeline.<br />
<br />
* [[Cppxfel spot finding | Spot-finding parameters with DIALS ]] - how to choose the spot-finding parameters which give you the best spots for indexing.<br />
<br />
== Reference manual ==<br />
<br />
The associated commands with the tutorials, in addition to several commands not used in the tutorials, are all documented in a [[Cppxfel Reference Manual | reference manual]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-05-04T14:26:12Z<p>Helenginn: </p>
<hr />
<div>This page is under development!<br />
<br />
The original version of cppxfel indexed using DIALS, but if the unit cell dimensions are known, cppxfel itself can also be used to index images using the TakeTwo algorithm. This page details how to index using cppxfel.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS spot-finding on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. This is to generate the spot-finding parameters for good indexing rates. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> for spotfinding. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=2 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=2 global_threshold=100</code>.<br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle index=no<br />
</pre><br />
<br />
The term <code>index=no</code> prevents DIALS from running indexing on these images, as we plan to index using cppxfel.<br />
<br />
The spot-finding results made by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.gen shot*.pickle<br />
</pre><br />
<br />
This will generate a very simple <code>_XXX_strong.list</code> text file for every <code>_XXX_strong.pickle</code> file, as well as prepare the images for ''cppxfel''. For an easy analysis of the number of spots per image, you can use the command<br />
<br />
<pre><br />
wc -l *strong.list<br />
</pre><br />
<br />
This will also create shells of <code>index.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code> and <code>merge.txt</code>.<br />
<br />
== Preparing for TakeTwo ==<br />
<br />
Have a look at the contents of the <code>index.txt</code> shell.<br />
<br />
<pre><br />
cat index.txt<br />
</pre><br />
<br />
The output will look something like this:<br />
<br />
<pre><br />
ORIENTATION_MATRIX_LIST matrices.dat<br />
NEW_MATRIX_LIST indexed.dat<br />
# Be sure to set the UNIT_CELL and SPACE_GROUP for indexing. cppxfel cannot index without this knowledge.<br />
SPACE_GROUP 0<br />
UNIT_CELL 0 0 0 0 0 0 <br />
<br />
MM_PER_PIXEL 0.11<br />
BEAM_CENTRE 881.755 881.5075<br />
DETECTOR_DISTANCE 90.9988<br />
INTEGRATION_WAVELENGTH 1.45825667181 <br />
<br />
PANEL_LIST panels.txt<br />
METROLOGY_SEARCH_SIZE 2<br />
<br />
# If your crystal is highly mosaic or the detector is quite far back you may need to increase the padding values.<br />
SHOEBOX_FOREGROUND_PADDING 1<br />
SHOEBOX_NEITHER_PADDING 2<br />
SHOEBOX_BACKGROUND_PADDING 3<br />
<br />
# If you see too many spots, increase the intensity threshold.<br />
INTENSITY_THRESHOLD 12<br />
ABSOLUTE_INTENSITY OFF<br />
<br />
OVER_PRED_BANDWIDTH 0.07<br />
<br />
REFINE_ORIENTATIONS ON<br />
ROUGH_CALCULATION ON<br />
<br />
# Specifies maximum multiple lattices to index in total<br />
SOLUTION_ATTEMPTS 1<br />
<br />
# Maximum reciprocal distance from spot to spot to consider for analysis.<br />
# A maximum reciprocal distance of 0.1 would be equivalent separation<br />
# between the beam centre and the 10 Angstrom resolution ring.<br />
MAX_RECIPROCAL_DISTANCE 0.15<br />
<br />
# Initial rlp size: used to determine the tolerances for the vector lengths in the crystal.<br />
# For a 1 micron crystal with no mosaicity, the initial rlp size is 0.0001 Ang^-1 (i.e.,<br />
# 1 / 10000 Ang). To be more strict for indexing, lower this number; to be less strict increase it.<br />
INITIAL_RLP_SIZE 0.0001<br />
<br />
# If you wish to see more verbose output, change to 1 (moderate), or 2 (debug, usually too much).<br />
VERBOSITY_LEVEL 0<br />
<br />
COMMANDS<br />
<br />
INDEX<br />
</pre><br />
<br />
Note that some parameters have not been initialised. Edit these lines in order to supply the correct information. The edited lines are shown below, but check the entire input. The space group and unit cell are essential for ''cppxfel'' indexing: it cannot currently index without a known unit cell and space group.<br />
<br />
<pre><br />
SPACE_GROUP 197<br />
UNIT_CELL 106.1 106.1 106.1 90 90 90<br />
</pre><br />
<br />
When the <code>index.txt</code> file is ready, you may run indexing on the data:<br />
<br />
<pre><br />
cppxfel.run -i index.txt<br />
</pre><br />
<br />
Wavelength histograms should appear every time an image is successfully indexed:<br />
<br />
<pre><br />
Wavelength histogram for shot-s00-20130316164947655.img<br />
1.356 <br />
1.366 <br />
1.377 <br />
1.387 ...<br />
1.397 ....<br />
1.407 ......<br />
1.417 ..<br />
1.428 ....<br />
1.438 ..<br />
1.448 .....<br />
1.458 ................................................................<br />
1.468 ................................................................<br />
1.479 ........<br />
1.489 ..<br />
1.499 ....<br />
1.509 ....<br />
1.52 ...<br />
1.53 <br />
1.54 <br />
1.55 <br />
1.56 <br />
</pre><br />
<br />
At the end of the run, it should create a file called <code>integrate-indexed.dat</code> which can be fed into integration.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/CppxfelCppxfel2016-05-04T01:57:36Z<p>Helenginn: Adding another tutorial</p>
<hr />
<div>This page is under development.<br />
<br />
''cppxfel'' is a software suite primarily designed to showcase some methods which would benefit from being incorporated into larger XFEL-specific software suites. This suite currently covers the part of the XFEL data processing pipeline which stems from refining indexing solutions to generating merged MTZ files for use in structure refinement. These wiki files cover installation, step-by-step tutorials and a reference manual for each of the commands. This is meant to be read in conjunction with an unpublished paper.<br />
<br />
== Installation ==<br />
<br />
Download and installation instructions for cppxfel can be found [[Cppxfel Installation | here]]. ''cppxfel'' relies on a distribution of the ''cctbx'' library and Diffraction Integration for Advanced Light Sources (''DIALS''). In case you wish to update your current version, please follow the [[Updating cppxfel | updating instructions]]. Any major changes added to the cppxfel distribution is documented in this [[Cppxfel change log | change log]].<br />
<br />
== Step-by-step tutorials ==<br />
<br />
Step-by-step tutorials are provided for a 1000-image downloadable data set for CPV17 (structure solution described in [http://dx.doi.org/10.1038/ncomms7435 Ginn et al Nat. Comms. 2015]). The tutorials are divided into several stages:<br />
<br />
* [[Cppxfel Indexing with DIALS | Indexing with DIALS]] - cppxfel uses scripts to call on DIALS indexing programs to index individual images.<br />
* [[Cppxfel Indexing | Indexing with ''cppxfel'']] - cppxfel's own indexing algorithm, TakeTwo<br />
* [[Cppxfel Initial Orientation Matrix Refinement]] - performing initial refinement to produce high quality orientation matrices before post-refinement.<br />
* [[Cppxfel Post-refinement]] - post-refinement of crystals against a reference data set generated from the previous stage<br />
* [[Cppxfel Final Merge]] - producing a final MTZ file for use in structure solution.<br />
* [[Cppxfel Statistics]] - how to assess the quality of the XFEL data processing.<br />
<br />
== Detailed tutorials ==<br />
<br />
These are some more detailed tutorials on how to make decisions in the ''cppxfel'' pipeline.<br />
<br />
* [[Cppxfel spot finding | Spot-finding parameters with DIALS ]] - how to choose the spot-finding parameters which give you the best spots for indexing.<br />
<br />
== Reference manual ==<br />
<br />
The associated commands with the tutorials, in addition to several commands not used in the tutorials, are all documented in a [[Cppxfel Reference Manual | reference manual]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_spot_findingCppxfel spot finding2016-05-04T01:55:47Z<p>Helenginn: </p>
<hr />
<div>This page is under development!<br />
<br />
Cppxfel currently uses DIALS to find spots on individual images. This tutorial explains how to determine a suitable set of parameters to put into the <code>find_spots.options</code> file (mentioned during the tutorial on [Cppxfel Indexing with DIALS])) using the DIALS GUI.<br />
<br />
We will choose an image to focus on: <code>shot-s00-20130316171711930.pickle</code>.<br />
<br />
== Loading the image into DIALS ==<br />
<br />
To load the image into the DIALS GUI, you can type:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle<br />
</pre><br />
<br />
== Adjusting the brightness ==<br />
<br />
This opens up a window which displays the image and a number of controls. The first task is to adjust the brightness until the background of the image is significantly lighter than the signal of the spots. This is adjusted using the "brightness" bar. A brightness of 20 works well for the image above.<br />
<br />
== Viewing the threshold image ==<br />
<br />
DIALS spot-finding occurs in a series of stages which finishes with a "threshold" image which can be viewed by pressing the "threshold" button in the bottom right hand corner. Black/grey pixels will be counted as spots if the number of neighbouring pixels (plus itself) exceeds the min_spot_size parameter. Using the default parameters, the CSPAD image does not look good, counting most of the background pixels as spots.<br />
<br />
[[File:cppxfel_dials_threshold_before.png|300px]]<br />
<br />
(example of threshold image before playing around with DIALS spot-finding parameters)<br />
<br />
== Adjusting the parameters ==<br />
<br />
The parameters need adjusting to pick up the correct spots on the image. The CSPAD has a gain of 14, so this should be set first. Type "14" into the gain box and press "enter". For reference, the SACLA detector (MPCCD) is usually set to a gain of 10.<br />
<br />
The next parameter to adjust is the global threshold. Try a global threshold of 100, 200 or 300.<br />
<br />
[[File:cppxfel_dials_spot_finding_parameters.png|300px]]<br />
<br />
These are the parameters you want to change.<br />
<br />
[[File:cppxfel_dials_after_threshold.png|300px]]<br />
<br />
These are the spots you want to see on the "threshold" image, with the minimum noise possible while still picking up as many spots as possible.<br />
<br />
After fiddling with the parameters, one can create a <code>find_spots.options</code> file from the results:<br />
<br />
<pre><br />
gain=14 global_threshold=200 min_spot_size=2<br />
</pre><br />
<br />
Sometimes one may wish to include a minimum/maximum resolution cutoff although it is not necessary for the test data set, e.g.<br />
<br />
<pre><br />
filter.d_min = 1.6<br />
filter.d_max = 30<br />
</pre><br />
<br />
== Running a trial image ==<br />
<br />
You can run DIALS using the cppxfel spot-finding scripts on a single image:<br />
<br />
<pre><br />
cppxfel.run_dials shot-s00-20130316171711930.pickle index=no<br />
</pre><br />
<br />
This will produce a <code>_shot-s00-20130316171711930_strong.pickle</code> file which you can check by loading back into <code>dials.image_viewer</code>:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle _shot-s00-20130316171711930_strong.pickle<br />
</pre><br />
<br />
This way, you can check your spot-finding parameters on one or a number of other images, before scaling up.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_spot_findingCppxfel spot finding2016-05-04T01:54:55Z<p>Helenginn: image widths</p>
<hr />
<div>This page is under development!<br />
<br />
Cppxfel currently uses DIALS to find spots on individual images. This tutorial explains how to determine a suitable set of parameters to put into the <code>find_spots.options</code> file (mentioned during the tutorial on [Cppxfel Indexing with DIALS])) using the DIALS GUI.<br />
<br />
We will choose an image to focus on: <code>shot-s00-20130316171711930.pickle</code>.<br />
<br />
== Loading the image into DIALS ==<br />
<br />
To load the image into the DIALS GUI, you can type:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle<br />
</pre><br />
<br />
== Adjusting the brightness ==<br />
<br />
This opens up a window which displays the image and a number of controls. The first task is to adjust the brightness until the background of the image is significantly lighter than the signal of the spots. This is adjusted using the "brightness" bar. A brightness of 20 works well for the image above.<br />
<br />
== Viewing the threshold image ==<br />
<br />
DIALS spot-finding occurs in a series of stages which finishes with a "threshold" image which can be viewed by pressing the "threshold" button in the bottom right hand corner. Black/grey pixels will be counted as spots if the number of neighbouring pixels (plus itself) exceeds the min_spot_size parameter. Using the default parameters, the CSPAD image does not look good, counting most of the background pixels as spots.<br />
<br />
[[File:cppxfel_dials_threshold_before.png|300px]]<br />
(example of threshold image before playing around with DIALS spot-finding parameters)<br />
<br />
== Adjusting the parameters ==<br />
<br />
The parameters need adjusting to pick up the correct spots on the image. The CSPAD has a gain of 14, so this should be set first. Type "14" into the gain box and press "enter". For reference, the SACLA detector (MPCCD) is usually set to a gain of 10.<br />
<br />
The next parameter to adjust is the global threshold. Try a global threshold of 100, 200 or 300.<br />
<br />
[[File:cppxfel_dials_spot_finding_parameters.png|300px]]<br />
These are the parameters you want to change.<br />
<br />
[[File:cppxfel_dials_after_threshold.png|300px]]<br />
These are the spots you want to see on the "threshold" image, with the minimum noise possible while still picking up as many spots as possible.<br />
<br />
After fiddling with the parameters, one can create a <code>find_spots.options</code> file from the results:<br />
<br />
<pre><br />
gain=14 global_threshold=200 min_spot_size=2<br />
</pre><br />
<br />
Sometimes one may wish to include a minimum/maximum resolution cutoff although it is not necessary for the test data set, e.g.<br />
<br />
<pre><br />
filter.d_min = 1.6<br />
filter.d_max = 30<br />
</pre><br />
<br />
== Running a trial image ==<br />
<br />
You can run DIALS using the cppxfel spot-finding scripts on a single image:<br />
<br />
<pre><br />
cppxfel.run_dials shot-s00-20130316171711930.pickle index=no<br />
</pre><br />
<br />
This will produce a <code>_shot-s00-20130316171711930_strong.pickle</code> file which you can check by loading back into <code>dials.image_viewer</code>:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle _shot-s00-20130316171711930_strong.pickle<br />
</pre><br />
<br />
This way, you can check your spot-finding parameters on one or a number of other images, before scaling up.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_spot_findingCppxfel spot finding2016-05-04T01:52:10Z<p>Helenginn: adding images</p>
<hr />
<div>This page is under development!<br />
<br />
Cppxfel currently uses DIALS to find spots on individual images. This tutorial explains how to determine a suitable set of parameters to put into the <code>find_spots.options</code> file (mentioned during the tutorial on [Cppxfel Indexing with DIALS])) using the DIALS GUI.<br />
<br />
We will choose an image to focus on: <code>shot-s00-20130316171711930.pickle</code>.<br />
<br />
== Loading the image into DIALS ==<br />
<br />
To load the image into the DIALS GUI, you can type:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle<br />
</pre><br />
<br />
== Adjusting the brightness ==<br />
<br />
This opens up a window which displays the image and a number of controls. The first task is to adjust the brightness until the background of the image is significantly lighter than the signal of the spots. This is adjusted using the "brightness" bar. A brightness of 20 works well for the image above.<br />
<br />
== Viewing the threshold image ==<br />
<br />
DIALS spot-finding occurs in a series of stages which finishes with a "threshold" image which can be viewed by pressing the "threshold" button in the bottom right hand corner. Black/grey pixels will be counted as spots if the number of neighbouring pixels (plus itself) exceeds the min_spot_size parameter. Using the default parameters, the CSPAD image does not look good, counting most of the background pixels as spots.<br />
<br />
[[File:cppxfel_dials_threshold_before.png]]<br />
(example of threshold image before playing around with DIALS spot-finding parameters)<br />
<br />
== Adjusting the parameters ==<br />
<br />
The parameters need adjusting to pick up the correct spots on the image. The CSPAD has a gain of 14, so this should be set first. Type "14" into the gain box and press "enter". For reference, the SACLA detector (MPCCD) is usually set to a gain of 10.<br />
<br />
The next parameter to adjust is the global threshold. Try a global threshold of 100, 200 or 300.<br />
<br />
[[File:cppxfel_dials_spot_finding_parameters.png]]<br />
These are the parameters you want to change.<br />
<br />
[[File:cppxfel_dials_threshold_after.png]]<br />
These are the spots you want to see on the "threshold" image, with the minimum noise possible while still picking up as many spots as possible.<br />
<br />
After fiddling with the parameters, one can create a <code>find_spots.options</code> file from the results:<br />
<br />
<pre><br />
gain=14 global_threshold=200 min_spot_size=2<br />
</pre><br />
<br />
Sometimes one may wish to include a minimum/maximum resolution cutoff although it is not necessary for the test data set, e.g.<br />
<br />
<pre><br />
filter.d_min = 1.6<br />
filter.d_max = 30<br />
</pre><br />
<br />
== Running a trial image ==<br />
<br />
You can run DIALS using the cppxfel spot-finding scripts on a single image:<br />
<br />
<pre><br />
cppxfel.run_dials shot-s00-20130316171711930.pickle index=no<br />
</pre><br />
<br />
This will produce a <code>_shot-s00-20130316171711930_strong.pickle</code> file which you can check by loading back into <code>dials.image_viewer</code>:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle _shot-s00-20130316171711930_strong.pickle<br />
</pre><br />
<br />
This way, you can check your spot-finding parameters on one or a number of other images, before scaling up.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/File:Cppxfel_dials_after_threshold.pngFile:Cppxfel dials after threshold.png2016-05-04T01:51:00Z<p>Helenginn: The ideal spot-finding situation for CPV17 image shot-s00-20130316171711930.pickle. This was used using a gain of 14 and a global_threshold of 200.</p>
<hr />
<div>The ideal spot-finding situation for CPV17 image shot-s00-20130316171711930.pickle. This was used using a gain of 14 and a global_threshold of 200.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/File:Cppxfel_dials_spot_finding_parameters.pngFile:Cppxfel dials spot finding parameters.png2016-05-04T01:49:47Z<p>Helenginn: Business area for DIALS spot-finding parameter estimation GUI.</p>
<hr />
<div>Business area for DIALS spot-finding parameter estimation GUI.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/File:Cppxfel_dials_threshold_before.pngFile:Cppxfel dials threshold before.png2016-05-04T01:46:28Z<p>Helenginn: Default parameters attempting spot-finding on a CSPAD detector. To remedy this, set the gain to 14 and change the global_threshold.</p>
<hr />
<div>Default parameters attempting spot-finding on a CSPAD detector. To remedy this, set the gain to 14 and change the global_threshold.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_spot_findingCppxfel spot finding2016-05-04T01:44:32Z<p>Helenginn: New page</p>
<hr />
<div>This page is under development!<br />
<br />
Cppxfel currently uses DIALS to find spots on individual images. This tutorial explains how to determine a suitable set of parameters to put into the <code>find_spots.options</code> file (mentioned during the tutorial on [Cppxfel Indexing with DIALS])) using the DIALS GUI.<br />
<br />
We will choose an image to focus on: <code>shot-s00-20130316171711930.pickle</code>.<br />
<br />
== Loading the image into DIALS ==<br />
<br />
To load the image into the DIALS GUI, you can type:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle<br />
</pre><br />
<br />
== Adjusting the brightness ==<br />
<br />
This opens up a window which displays the image and a number of controls. The first task is to adjust the brightness until the background of the image is significantly lighter than the signal of the spots. This is adjusted using the "brightness" bar. A brightness of 20 works well for the image above.<br />
<br />
== Viewing the threshold image ==<br />
<br />
DIALS spot-finding occurs in a series of stages which finishes with a "threshold" image which can be viewed by pressing the "threshold" button in the bottom right hand corner. Black/grey pixels will be counted as spots if the number of neighbouring pixels (plus itself) exceeds the min_spot_size parameter. Using the default parameters, the CSPAD image does not look good, counting most of the background pixels as spots.<br />
<br />
== Adjusting the parameters ==<br />
<br />
The parameters need adjusting to pick up the correct spots on the image. The CSPAD has a gain of 14, so this should be set first. Type "14" into the gain box and press "enter". For reference, the SACLA detector (MPCCD) is usually set to a gain of 10.<br />
<br />
The next parameter to adjust is the global threshold. Try a global threshold of 100, 200 or 300.<br />
<br />
After fiddling with the parameters, one can create a <code>find_spots.options</code> file from the results:<br />
<br />
<pre><br />
gain=14 global_threshold=200 min_spot_size=2<br />
</pre><br />
<br />
Sometimes one may wish to include a minimum/maximum resolution cutoff although it is not necessary for the test data set, e.g.<br />
<br />
<pre><br />
filter.d_min = 1.6<br />
filter.d_max = 30<br />
</pre><br />
<br />
== Running a trial image ==<br />
<br />
You can run DIALS using the cppxfel spot-finding scripts on a single image:<br />
<br />
<pre><br />
cppxfel.run_dials shot-s00-20130316171711930.pickle index=no<br />
</pre><br />
<br />
This will produce a <code>_shot-s00-20130316171711930_strong.pickle</code> file which you can check by loading back into <code>dials.image_viewer</code>:<br />
<br />
<pre><br />
dials.image_viewer shot-s00-20130316171711930.pickle _shot-s00-20130316171711930_strong.pickle<br />
</pre><br />
<br />
This way, you can check your spot-finding parameters on one or a number of other images, before scaling up.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-05-04T01:23:04Z<p>Helenginn: </p>
<hr />
<div>This page is under development!<br />
<br />
The original version of cppxfel indexed using DIALS, but if the unit cell dimensions are known, cppxfel itself can also be used to index images using the TakeTwo algorithm. This page details how to index using cppxfel.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS spot-finding on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. This is to generate the spot-finding parameters for good indexing rates. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> for spotfinding. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=2 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=1 global_threshold=100</code>.<br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle index=no<br />
</pre><br />
<br />
The term <code>index=no</code> prevents DIALS from running indexing on these images, as we plan to index using DIALS.<br />
<br />
The spot-finding results made by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.gen shot*.pickle<br />
</pre><br />
<br />
This will generate a very simple <code>_XXX_strong.list</code> text file for every <code>_XXX_strong.pickle</code> file, as well as prepare the images for ''cppxfel''. For an easy analysis of the number of spots per image, you can use the command<br />
<br />
<pre><br />
wc -l *strong.list<br />
</pre><br />
<br />
This will also create shells of <code>index.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code> and <code>merge.txt</code>.<br />
<br />
== Preparing for TakeTwo ==<br />
<br />
Have a look at the contents of the <code>index.txt</code> shell.<br />
<br />
<pre><br />
cat index.txt<br />
</pre><br />
<br />
The output will look something like this:<br />
<br />
<pre><br />
ORIENTATION_MATRIX_LIST matrices.dat<br />
NEW_MATRIX_LIST indexed.dat<br />
# Be sure to set the UNIT_CELL and SPACE_GROUP for indexing. cppxfel cannot index without this knowledge.<br />
SPACE_GROUP 0<br />
UNIT_CELL 0 0 0 0 0 0 <br />
<br />
MM_PER_PIXEL 0.11<br />
BEAM_CENTRE 881.755 881.5075<br />
DETECTOR_DISTANCE 90.9988<br />
INTEGRATION_WAVELENGTH 1.45825667181 <br />
<br />
PANEL_LIST panels.txt<br />
METROLOGY_SEARCH_SIZE 2<br />
<br />
# If your crystal is highly mosaic or the detector is quite far back you may need to increase the padding values.<br />
SHOEBOX_FOREGROUND_PADDING 1<br />
SHOEBOX_NEITHER_PADDING 2<br />
SHOEBOX_BACKGROUND_PADDING 3<br />
<br />
# If you see too many spots, increase the intensity threshold.<br />
INTENSITY_THRESHOLD 12<br />
ABSOLUTE_INTENSITY OFF<br />
<br />
OVER_PRED_BANDWIDTH 0.07<br />
<br />
REFINE_ORIENTATIONS ON<br />
ROUGH_CALCULATION ON<br />
<br />
# Specifies maximum multiple lattices to index in total<br />
SOLUTION_ATTEMPTS 1<br />
<br />
# Maximum reciprocal distance from spot to spot to consider for analysis.<br />
# A maximum reciprocal distance of 0.1 would be equivalent separation<br />
# between the beam centre and the 10 Angstrom resolution ring.<br />
MAX_RECIPROCAL_DISTANCE 0.15<br />
<br />
# Initial rlp size: used to determine the tolerances for the vector lengths in the crystal.<br />
# For a 1 micron crystal with no mosaicity, the initial rlp size is 0.0001 Ang^-1 (i.e.,<br />
# 1 / 10000 Ang). To be more strict for indexing, lower this number; to be less strict increase it.<br />
INITIAL_RLP_SIZE 0.0001<br />
<br />
# If you wish to see more verbose output, change to 1 (moderate), or 2 (debug, usually too much).<br />
VERBOSITY_LEVEL 0<br />
<br />
COMMANDS<br />
<br />
INDEX<br />
</pre><br />
<br />
Note that some parameters have not been initialised. Edit these lines in order to supply the correct information. The edited lines are shown below, but check the entire input. The space group and unit cell are essential for ''cppxfel'' indexing: it cannot currently index without a known unit cell and space group.<br />
<br />
<pre><br />
SPACE_GROUP 197<br />
UNIT_CELL 106.1 106.1 106.1 90 90 90<br />
</pre><br />
<br />
When the <code>index.txt</code> file is ready, you may run indexing on the data:<br />
<br />
<pre><br />
cppxfel.run -i index.txt<br />
</pre><br />
<br />
Wavelength histograms should appear every time an image is successfully indexed:<br />
<br />
<pre><br />
Wavelength histogram for shot-s00-20130316164947655.img<br />
1.356 <br />
1.366 <br />
1.377 <br />
1.387 ...<br />
1.397 ....<br />
1.407 ......<br />
1.417 ..<br />
1.428 ....<br />
1.438 ..<br />
1.448 .....<br />
1.458 ................................................................<br />
1.468 ................................................................<br />
1.479 ........<br />
1.489 ..<br />
1.499 ....<br />
1.509 ....<br />
1.52 ...<br />
1.53 <br />
1.54 <br />
1.55 <br />
1.56 <br />
</pre><br />
<br />
At the end of the run, it should create a file called <code>integrate-indexed.dat</code> which can be fed into integration.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/CppxfelCppxfel2016-05-04T01:22:40Z<p>Helenginn: adding extra tutorial page on cppxfel indexing</p>
<hr />
<div>This page is under development.<br />
<br />
''cppxfel'' is a software suite primarily designed to showcase some methods which would benefit from being incorporated into larger XFEL-specific software suites. This suite currently covers the part of the XFEL data processing pipeline which stems from refining indexing solutions to generating merged MTZ files for use in structure refinement. These wiki files cover installation, step-by-step tutorials and a reference manual for each of the commands. This is meant to be read in conjunction with an unpublished paper.<br />
<br />
== Installation ==<br />
<br />
Download and installation instructions for cppxfel can be found [[Cppxfel Installation | here]]. ''cppxfel'' relies on a distribution of the ''cctbx'' library and Diffraction Integration for Advanced Light Sources (''DIALS''). In case you wish to update your current version, please follow the [[Updating cppxfel | updating instructions]]. Any major changes added to the cppxfel distribution is documented in this [[Cppxfel change log | change log]].<br />
<br />
== Step-by-step tutorials ==<br />
<br />
Step-by-step tutorials are provided for a 1000-image downloadable data set for CPV17 (structure solution described in [http://dx.doi.org/10.1038/ncomms7435 Ginn et al Nat. Comms. 2015]). The tutorials are divided into several stages:<br />
<br />
* [[Cppxfel Indexing with DIALS | Indexing with DIALS]] - cppxfel uses scripts to call on DIALS indexing programs to index individual images.<br />
* [[Cppxfel Indexing | Indexing with ''cppxfel'']] - cppxfel's own indexing algorithm, TakeTwo<br />
* [[Cppxfel Initial Orientation Matrix Refinement]] - performing initial refinement to produce high quality orientation matrices before post-refinement.<br />
* [[Cppxfel Post-refinement]] - post-refinement of crystals against a reference data set generated from the previous stage<br />
* [[Cppxfel Final Merge]] - producing a final MTZ file for use in structure solution.<br />
* [[Cppxfel Statistics]] - how to assess the quality of the XFEL data processing.<br />
<br />
== Reference manual ==<br />
<br />
The associated commands with the tutorials, in addition to several commands not used in the tutorials, are all documented in a [[Cppxfel Reference Manual | reference manual]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-05-04T01:21:22Z<p>Helenginn: Editing cppxfel indexing to be actual cppxfel indexing</p>
<hr />
<div>This page is under development!<br />
<br />
The original version of cppxfel indexed using DIALS, but if the unit cell dimensions are known, cppxfel itself can also be used to index images. This page details how to index using cppxfel.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS spot-finding on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. This is to generate the spot-finding parameters for good indexing rates. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> for spotfinding. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=2 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=1 global_threshold=100</code>.<br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle index=no<br />
</pre><br />
<br />
The term <code>index=no</code> prevents DIALS from running indexing on these images, as we plan to index using DIALS.<br />
<br />
The spot-finding results made by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.gen shot*.pickle<br />
</pre><br />
<br />
This will generate a very simple <code>_XXX_strong.list</code> text file for every <code>_XXX_strong.pickle</code> file, as well as prepare the images for ''cppxfel''. For an easy analysis of the number of spots per image, you can use the command<br />
<br />
<pre><br />
wc -l *strong.list<br />
</pre><br />
<br />
This will also create shells of <code>index.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code> and <code>merge.txt</code>.<br />
<br />
== Preparing and running indexing with cppxfel ==<br />
<br />
Have a look at the contents of the <code>index.txt</code> shell.<br />
<br />
<pre><br />
cat index.txt<br />
</pre><br />
<br />
The output will look something like this:<br />
<br />
<pre><br />
ORIENTATION_MATRIX_LIST matrices.dat<br />
NEW_MATRIX_LIST indexed.dat<br />
# Be sure to set the UNIT_CELL and SPACE_GROUP for indexing. cppxfel cannot index without this knowledge.<br />
SPACE_GROUP 0<br />
UNIT_CELL 0 0 0 0 0 0 <br />
<br />
MM_PER_PIXEL 0.11<br />
BEAM_CENTRE 881.755 881.5075<br />
DETECTOR_DISTANCE 90.9988<br />
INTEGRATION_WAVELENGTH 1.45825667181 <br />
<br />
PANEL_LIST panels.txt<br />
METROLOGY_SEARCH_SIZE 2<br />
<br />
# If your crystal is highly mosaic or the detector is quite far back you may need to increase the padding values.<br />
SHOEBOX_FOREGROUND_PADDING 1<br />
SHOEBOX_NEITHER_PADDING 2<br />
SHOEBOX_BACKGROUND_PADDING 3<br />
<br />
# If you see too many spots, increase the intensity threshold.<br />
INTENSITY_THRESHOLD 12<br />
ABSOLUTE_INTENSITY OFF<br />
<br />
OVER_PRED_BANDWIDTH 0.07<br />
<br />
REFINE_ORIENTATIONS ON<br />
ROUGH_CALCULATION ON<br />
<br />
# Specifies maximum multiple lattices to index in total<br />
SOLUTION_ATTEMPTS 1<br />
<br />
# Maximum reciprocal distance from spot to spot to consider for analysis.<br />
# A maximum reciprocal distance of 0.1 would be equivalent separation<br />
# between the beam centre and the 10 Angstrom resolution ring.<br />
MAX_RECIPROCAL_DISTANCE 0.15<br />
<br />
# Initial rlp size: used to determine the tolerances for the vector lengths in the crystal.<br />
# For a 1 micron crystal with no mosaicity, the initial rlp size is 0.0001 Ang^-1 (i.e.,<br />
# 1 / 10000 Ang). To be more strict for indexing, lower this number; to be less strict increase it.<br />
INITIAL_RLP_SIZE 0.0001<br />
<br />
# If you wish to see more verbose output, change to 1 (moderate), or 2 (debug, usually too much).<br />
VERBOSITY_LEVEL 0<br />
<br />
COMMANDS<br />
<br />
INDEX<br />
</pre><br />
<br />
Note that some parameters have not been initialised. Edit these lines in order to supply the correct information. The edited lines are shown below, but check the entire input. The space group and unit cell are essential for ''cppxfel'' indexing: it cannot currently index without a known unit cell and space group.<br />
<br />
<pre><br />
SPACE_GROUP 197<br />
UNIT_CELL 106.1 106.1 106.1 90 90 90<br />
</pre><br />
<br />
When the <code>index.txt</code> file is ready, you may run indexing on the data:<br />
<br />
<pre><br />
cppxfel.run -i index.txt<br />
</pre><br />
<br />
Wavelength histograms should appear every time an image is successfully indexed:<br />
<br />
<pre><br />
Wavelength histogram for shot-s00-20130316164947655.img<br />
1.356 <br />
1.366 <br />
1.377 <br />
1.387 ...<br />
1.397 ....<br />
1.407 ......<br />
1.417 ..<br />
1.428 ....<br />
1.438 ..<br />
1.448 .....<br />
1.458 ................................................................<br />
1.468 ................................................................<br />
1.479 ........<br />
1.489 ..<br />
1.499 ....<br />
1.509 ....<br />
1.52 ...<br />
1.53 <br />
1.54 <br />
1.55 <br />
1.56 <br />
</pre><br />
<br />
At the end of the run, it should create a file called <code>integrate-indexed.dat</code> which can be fed into integration.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-05-04T01:13:21Z<p>Helenginn: minor edits</p>
<hr />
<div>This covers the generation of indexing solutions using DIALS within the [[Cppxfel|''cppxfel'' distribution]]. This assumes a successful [[Cppxfel Installation | installation of cppxfel]] including the DIALS dependency.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> and <code>index.options</code> for spotfinding/indexing the data. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=1 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=1 global_threshold=100</code>.<br />
<br />
This should be similarly created for <code>index.options</code>. This is an appropriate options file for a DIALS distribution built after 20th Feb:<br />
<br />
<pre><br />
cat > index.options << EOF<br />
unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills</code><br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle<br />
</pre><br />
<br />
== Assessing the output of DIALS indexing ==<br />
<br />
The output from each DIALS find_spots and indexing event are stored in the <code>*find_spots.log</code> and <code>*index.log</code> files for each image name. The strong spots found in each individual image are stored in <code>_*_strong.pickle</code> for each respective image name. Any successfully indexed images follow the format <code>_*_experiments.json</code>. The number of indexed images can be roughly counted as so:<br />
<br />
<pre><br />
find _*_experiments.json | wc -l<br />
</pre><br />
<br />
== Converting the results of DIALS indexing for ''cppxfel''==<br />
<br />
The images and matrices indexed by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.gen shot*.pickle<br />
</pre><br />
<br />
This runs on the number of cores specified by the <code>NSLOTS</code> environment variable to extract the appropriate image data, and also generates several input files: <code>panels.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code>, <code>merge.txt</code>, which will be explained in the following tutorials. The indexed images are extracted to <code>.img</code> files. This also produces a <code>matrices.dat</code> file which contains all the indexing solutions generated by DIALS.<br />
<br />
Next step: [[Cppxfel Initial Orientation Matrix Refinement]]</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-05-04T01:05:24Z<p>Helenginn: minor edits</p>
<hr />
<div>This covers the generation of indexing solutions using DIALS within the [[Cppxfel|''cppxfel'' distribution]]. This assumes a successful [[Cppxfel Installation | installation of cppxfel]] including the DIALS dependency.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> and <code>index.options</code> for spotfinding/indexing the data. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=1 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=1 global_threshold=100</code>.<br />
<br />
This should be similarly created for <code>index.options</code>. This is an appropriate options file for a DIALS distribution built after 20th Feb:<br />
<br />
<pre><br />
cat > index.options << EOF<br />
unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills</code><br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle<br />
</pre><br />
<br />
== Assessing the output of DIALS indexing ==<br />
<br />
The output from each DIALS find_spots and indexing event are stored in the <code>*find_spots.log</code> and <code>*index.log</code> files for each image name. The strong spots found in each individual image are stored in <code>_*_strong.pickle</code> for each respective image name. Any successfully indexed images follow the format <code>_*_experiments.json</code>. The number of indexed images can be roughly counted as so:<br />
<br />
<pre><br />
find _*_experiments.json | wc -l<br />
</pre><br />
<br />
== Converting the results of DIALS indexing for ''cppxfel''==<br />
<br />
The images and matrices indexed by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.gen<br />
</pre><br />
<br />
This runs on the number of cores specified by the <code>NSLOTS</code> environment variable to extract the appropriate image data, and also generates several input files: <code>panels.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code>, <code>merge.txt</code>, which will be explained in the following tutorials. The indexed images are extracted to <code>.img</code> files. This also produces a <code>matrices.dat</code> file which contains all the indexing solutions generated by DIALS.<br />
<br />
Next step: [[Cppxfel Initial Orientation Matrix Refinement]]</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/CppxfelCppxfel2016-05-04T00:59:19Z<p>Helenginn: changing link</p>
<hr />
<div>This page is under development.<br />
<br />
''cppxfel'' is a software suite primarily designed to showcase some methods which would benefit from being incorporated into larger XFEL-specific software suites. This suite currently covers the part of the XFEL data processing pipeline which stems from refining indexing solutions to generating merged MTZ files for use in structure refinement. These wiki files cover installation, step-by-step tutorials and a reference manual for each of the commands. This is meant to be read in conjunction with an unpublished paper.<br />
<br />
== Installation ==<br />
<br />
Download and installation instructions for cppxfel can be found [[Cppxfel Installation | here]]. ''cppxfel'' relies on a distribution of the ''cctbx'' library and Diffraction Integration for Advanced Light Sources (''DIALS''). In case you wish to update your current version, please follow the [[Updating cppxfel | updating instructions]]. Any major changes added to the cppxfel distribution is documented in this [[Cppxfel change log | change log]].<br />
<br />
== Step-by-step tutorials ==<br />
<br />
Step-by-step tutorials are provided for a 1000-image downloadable data set for CPV17 (structure solution described in [http://dx.doi.org/10.1038/ncomms7435 Ginn et al Nat. Comms. 2015]). The tutorials are divided into several stages:<br />
<br />
* [[Cppxfel Indexing with DIALS | Indexing with DIALS]] - cppxfel uses scripts to call on DIALS indexing programs to index individual images.<br />
* [[Cppxfel Initial Orientation Matrix Refinement]] - performing initial refinement to produce high quality orientation matrices before post-refinement.<br />
* [[Cppxfel Post-refinement]] - post-refinement of crystals against a reference data set generated from the previous stage<br />
* [[Cppxfel Final Merge]] - producing a final MTZ file for use in structure solution.<br />
* [[Cppxfel Statistics]] - how to assess the quality of the XFEL data processing.<br />
<br />
== Reference manual ==<br />
<br />
The associated commands with the tutorials, in addition to several commands not used in the tutorials, are all documented in a [[Cppxfel Reference Manual | reference manual]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_Indexing_with_DIALSCppxfel Indexing with DIALS2016-05-04T00:58:54Z<p>Helenginn: Starting "indexing with DIALS" page</p>
<hr />
<div>This covers the generation of indexing solutions using DIALS within the [[Cppxfel|''cppxfel'' distribution]]. This assumes a successful [[Cppxfel Installation | installation of cppxfel]] including the DIALS dependency.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> and <code>index.options</code> for spotfinding/indexing the data. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=1 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=1 global_threshold=100</code>.<br />
<br />
This should be similarly created for <code>index.options</code>. This is an appropriate options file for a DIALS distribution built after 20th Feb:<br />
<br />
<pre><br />
cat > index.options << EOF<br />
unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills</code><br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle<br />
</pre><br />
<br />
== Assessing the output of DIALS indexing ==<br />
<br />
The output from each DIALS find_spots and indexing event are stored in the <code>*find_spots.log</code> and <code>*index.log</code> files for each image name. The strong spots found in each individual image are stored in <code>_*_strong.pickle</code> for each respective image name. Any successfully indexed images follow the format <code>_*_experiments.json</code>. The number of indexed images can be roughly counted as so:<br />
<br />
<pre><br />
find _*_experiments.json | wc -l<br />
</pre><br />
<br />
== Converting the results of DIALS indexing for ''cppxfel''==<br />
<br />
The images and matrices indexed by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.input_gen<br />
</pre><br />
<br />
This runs on the number of cores specified by the <code>NSLOTS</code> environment variable to extract the appropriate image data, and also generates several input files: <code>panels.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code>, <code>merge.txt</code>, which will be explained in the following tutorials. The indexed images are extracted to <code>.img</code> files. This also produces a <code>matrices.dat</code> file which contains all the indexing solutions generated by DIALS.<br />
<br />
Next step: [[Cppxfel Initial Orientation Matrix Refinement]]</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-03-22T21:14:54Z<p>Helenginn: Tcsh shell warning</p>
<hr />
<div>This covers the generation of indexing solutions using DIALS within the [[Cppxfel|''cppxfel'' distribution]]. This assumes a successful [[Cppxfel Installation | installation of cppxfel]] including the DIALS dependency.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> and <code>index.options</code> for spotfinding/indexing the data. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=1 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>gain=14.0 min_spot_size=1 global_threshold=100</code>.<br />
<br />
This should be similarly created for <code>index.options</code>. This is an appropriate options file for a DIALS distribution built after 20th Feb:<br />
<br />
<pre><br />
cat > index.options << EOF<br />
unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills<br />
EOF<br />
</pre><br />
<br />
Note, in the tcsh shell, this is not a valid way of creating a text file. In this case, create an appropriate file in your favourite text editor with the text <code>unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills</code><br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle<br />
</pre><br />
<br />
== Assessing the output of DIALS indexing ==<br />
<br />
The output from each DIALS find_spots and indexing event are stored in the <code>*find_spots.log</code> and <code>*index.log</code> files for each image name. The strong spots found in each individual image are stored in <code>_*_strong.pickle</code> for each respective image name. Any successfully indexed images follow the format <code>_*_experiments.json</code>. The number of indexed images can be roughly counted as so:<br />
<br />
<pre><br />
find _*_experiments.json | wc -l<br />
</pre><br />
<br />
== Converting the results of DIALS indexing for ''cppxfel''==<br />
<br />
The images and matrices indexed by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.input_gen<br />
</pre><br />
<br />
This runs on the number of cores specified by the <code>NSLOTS</code> environment variable to extract the appropriate image data, and also generates several input files: <code>panels.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code>, <code>merge.txt</code>, which will be explained in the following tutorials. The indexed images are extracted to <code>.img</code> files. This also produces a <code>matrices.dat</code> file which contains all the indexing solutions generated by DIALS.<br />
<br />
Next step: [[Cppxfel Initial Orientation Matrix Refinement]]</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-03-06T16:53:39Z<p>Helenginn: Updating indexing options for recent DIALS distribution.</p>
<hr />
<div>This covers the generation of indexing solutions using DIALS within the [[Cppxfel|''cppxfel'' distribution]]. This assumes a successful [[Cppxfel Installation | installation of cppxfel]] including the DIALS dependency.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> and <code>index.options</code> for spotfinding/indexing the data. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=1 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
This should be similarly created for <code>index.options</code>. This is an appropriate options file for a DIALS distribution built after 20th Feb:<br />
<br />
<pre><br />
cat > index.options << EOF<br />
unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all recycle_unindexed_reflections=1 refinement.reflections.outlier.algorithm=null refinement_protocol.n_macro_cycles=1 refinement.reflections.weighting_strategy.override=stills<br />
EOF<br />
</pre><br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle<br />
</pre><br />
<br />
== Assessing the output of DIALS indexing ==<br />
<br />
The output from each DIALS find_spots and indexing event are stored in the <code>*find_spots.log</code> and <code>*index.log</code> files for each image name. The strong spots found in each individual image are stored in <code>_*_strong.pickle</code> for each respective image name. Any successfully indexed images follow the format <code>_*_experiments.json</code>. The number of indexed images can be roughly counted as so:<br />
<br />
<pre><br />
find _*_experiments.json | wc -l<br />
</pre><br />
<br />
== Converting the results of DIALS indexing for ''cppxfel''==<br />
<br />
The images and matrices indexed by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.input_gen<br />
</pre><br />
<br />
This runs on the number of cores specified by the <code>NSLOTS</code> environment variable to extract the appropriate image data, and also generates several input files: <code>panels.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code>, <code>merge.txt</code>, which will be explained in the following tutorials. The indexed images are extracted to <code>.img</code> files. This also produces a <code>matrices.dat</code> file which contains all the indexing solutions generated by DIALS.<br />
<br />
Next step: [[Cppxfel Initial Orientation Matrix Refinement]]</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/CppxfelCppxfel2016-03-01T12:21:14Z<p>Helenginn: </p>
<hr />
<div>This page is under development.<br />
<br />
''cppxfel'' is a software suite primarily designed to showcase some methods which would benefit from being incorporated into larger XFEL-specific software suites. This suite currently covers the part of the XFEL data processing pipeline which stems from refining indexing solutions to generating merged MTZ files for use in structure refinement. These wiki files cover installation, step-by-step tutorials and a reference manual for each of the commands. This is meant to be read in conjunction with an unpublished paper.<br />
<br />
== Installation ==<br />
<br />
Download and installation instructions for cppxfel can be found [[Cppxfel Installation | here]]. ''cppxfel'' relies on a distribution of the ''cctbx'' library and Diffraction Integration for Advanced Light Sources (''DIALS''). In case you wish to update your current version, please follow the [[Updating cppxfel | updating instructions]]. Any major changes added to the cppxfel distribution is documented in this [[Cppxfel change log | change log]].<br />
<br />
== Step-by-step tutorials ==<br />
<br />
Step-by-step tutorials are provided for a 1000-image downloadable data set for CPV17 (structure solution described in [http://dx.doi.org/10.1038/ncomms7435 Ginn et al Nat. Comms. 2015]). The tutorials are divided into several stages:<br />
<br />
* [[Cppxfel Indexing | Indexing with DIALS]] - cppxfel uses scripts to call on DIALS indexing programs to index individual images.<br />
* [[Cppxfel Initial Orientation Matrix Refinement]] - performing initial refinement to produce high quality orientation matrices before post-refinement.<br />
* [[Cppxfel Post-refinement]] - post-refinement of crystals against a reference data set generated from the previous stage<br />
* [[Cppxfel Final Merge]] - producing a final MTZ file for use in structure solution.<br />
* [[Cppxfel Statistics]] - how to assess the quality of the XFEL data processing.<br />
<br />
== Reference manual ==<br />
<br />
The associated commands with the tutorials, in addition to several commands not used in the tutorials, are all documented in a [[Cppxfel Reference Manual | reference manual]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_IndexingCppxfel Indexing2016-02-26T15:26:54Z<p>Helenginn: Removed reference to recycle_unindexed_reflections which has changed in the DIALS API.</p>
<hr />
<div>This covers the generation of indexing solutions using DIALS within the [[Cppxfel|''cppxfel'' distribution]]. This assumes a successful [[Cppxfel Installation | installation of cppxfel]] including the DIALS dependency.<br />
<br />
== Downloading test data ==<br />
<br />
Data can be downloaded as a [[https://dials.lbl.gov/datasets/ginn_jac_cpv17.tgz zip file]] from the DIALS website and should be extracted to a new folder.<br />
<br />
<pre><br />
tar zxvf ginn_jac_cpv17.tgz<br />
</pre><br />
<br />
== Running DIALS on the set of 1000 images ==<br />
<br />
The scripts included in ''cppxfel'' to run DIALS uses four cores by default. If you have more cores available on the current machine, this can be edited by setting the environment variable NSLOTS:<br />
<br />
<pre><br />
export NSLOTS=16<br />
</pre><br />
<br />
DIALS uses the options found in the two files <code>find_spots.options</code> and <code>index.options</code> for spotfinding/indexing the data. These should be generated for <code>find_spots.options</code> with these parameters:<br />
<br />
<pre><br />
cat > find_spots.options << EOF<br />
gain=14.0 min_spot_size=1 global_threshold=100<br />
EOF<br />
</pre><br />
<br />
This should be similarly created for <code>index.options</code>:<br />
<br />
<pre><br />
cat > index.options << EOF<br />
outlier.algorithm=null indexing.method=fft3d unit_cell=106.1,106.1,106.1 space_group=I23 minimum_number_of_reflections=20 detector.fix=all beam.fix=all<br />
EOF<br />
</pre><br />
<br />
The DIALS scripts can now be run as follows:<br />
<br />
<pre><br />
cppxfel.run_dials shot*.pickle<br />
</pre><br />
<br />
== Assessing the output of DIALS indexing ==<br />
<br />
The output from each DIALS find_spots and indexing event are stored in the <code>*find_spots.log</code> and <code>*index.log</code> files for each image name. The strong spots found in each individual image are stored in <code>_*_strong.pickle</code> for each respective image name. Any successfully indexed images follow the format <code>_*_experiments.json</code>. The number of indexed images can be roughly counted as so:<br />
<br />
<pre><br />
find _*_experiments.json | wc -l<br />
</pre><br />
<br />
== Converting the results of DIALS indexing for ''cppxfel''==<br />
<br />
The images and matrices indexed by DIALS should be converted to a new format for reading into ''cppxfel''. This is achieved by running the command:<br />
<br />
<pre><br />
cppxfel.input_gen<br />
</pre><br />
<br />
This runs on the number of cores specified by the <code>NSLOTS</code> environment variable to extract the appropriate image data, and also generates several input files: <code>panels.txt</code>, <code>integrate.txt</code>, <code>refine.txt</code>, <code>merge.txt</code>, which will be explained in the following tutorials. The indexed images are extracted to <code>.img</code> files. This also produces a <code>matrices.dat</code> file which contains all the indexing solutions generated by DIALS.<br />
<br />
Next step: [[Cppxfel Initial Orientation Matrix Refinement]]</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/CppxfelCppxfel2016-02-06T16:18:34Z<p>Helenginn: Adding change log link</p>
<hr />
<div>This page is under development.<br />
<br />
''cppxfel'' is a software suite primarily designed to showcase some methods which would benefit from being incorporated into larger XFEL-specific software suites. This suite currently covers the part of the XFEL data processing pipeline which stems from refining indexing solutions to generating merged MTZ files for use in structure refinement. These wiki files cover installation, step-by-step tutorials and a reference manual for each of the commands. This is meant to be read in conjunction with an unpublished paper.<br />
<br />
== Installation ==<br />
<br />
Download and installation instructions for cppxfel can be found [[Cppxfel Installation | here]]. ''cppxfel'' relies on a distribution of the ''cctbx'' library and Diffraction Integration for Advanced Light Sources (''DIALS''). In case you wish to update your current version, please follow the [[Updating cppxfel | updating instructions]]. Any changes added to the cppxfel distribution is documented in this [[Cppxfel change log | change log]].<br />
<br />
== Step-by-step tutorials ==<br />
<br />
Step-by-step tutorials are provided for a 1000-image downloadable data set for CPV17 (structure solution described in [http://dx.doi.org/10.1038/ncomms7435 Ginn et al Nat. Comms. 2015]). The tutorials are divided into several stages:<br />
<br />
* [[Cppxfel Indexing | Indexing with DIALS]] - cppxfel uses scripts to call on DIALS indexing programs to index individual images.<br />
* [[Cppxfel Initial Orientation Matrix Refinement]] - performing initial refinement to produce high quality orientation matrices before post-refinement.<br />
* [[Cppxfel Post-refinement]] - post-refinement of crystals against a reference data set generated from the previous stage<br />
* [[Cppxfel Final Merge]] - producing a final MTZ file for use in structure solution.<br />
* [[Cppxfel Statistics]] - how to assess the quality of the XFEL data processing.<br />
<br />
== Reference manual ==<br />
<br />
The associated commands with the tutorials, in addition to several commands not used in the tutorials, are all documented in a [[Cppxfel Reference Manual | reference manual]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_change_logCppxfel change log2016-02-06T16:16:45Z<p>Helenginn: New page</p>
<hr />
<div>Any changes to the cppxfel repository are listed here. Some features may be present in the cppxfel code which are currently undocumented if they are subject to change. If you are very keen to use these undocumented features, please email Helen Ginn (my first name @ strubi dot ox dot ac dot uk) and we can arrange something.<br />
<br />
=== 6th Feb 2016 ===<br />
<br />
<code>cppxfel.run_dials</code> now interfaces with the <code>dials.import</code> module correctly as the DIALS module API had been recently changed. If this breaks your installation, you need to make sure your copy of DIALS is up to date as well.<br />
<br />
=== 28th Jan 2016 ===<br />
<br />
Several undocumented features are now present. However, this includes a new command <code>ROUGH_CALCULATION</code> which can be included in a typical <code>integrate.txt</code> input file. This greatly increases the speed of orientation matrix refinement if you have set <code>REFINE_ORIENTATIONS</code> to <code>ON</code>.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/CppxfelCppxfel2016-01-28T13:35:14Z<p>Helenginn: </p>
<hr />
<div>This page is under development.<br />
<br />
''cppxfel'' is a software suite primarily designed to showcase some methods which would benefit from being incorporated into larger XFEL-specific software suites. This suite currently covers the part of the XFEL data processing pipeline which stems from refining indexing solutions to generating merged MTZ files for use in structure refinement. These wiki files cover installation, step-by-step tutorials and a reference manual for each of the commands. This is meant to be read in conjunction with an unpublished paper.<br />
<br />
== Installation ==<br />
<br />
Installation instructions for cppxfel can be found [[Cppxfel Installation | here]]. ''cppxfel'' relies on a distribution of the ''cctbx'' library and Diffraction Integration for Advanced Light Sources (''DIALS''). In case you wish to update your current version, please follow the [[Updating cppxfel | updating instructions]].<br />
<br />
== Step-by-step tutorials ==<br />
<br />
Step-by-step tutorials are provided for a 1000-image downloadable data set for CPV17 (structure solution described in [http://dx.doi.org/10.1038/ncomms7435 Ginn et al Nat. Comms. 2015]). The tutorials are divided into several stages:<br />
<br />
* [[Cppxfel Indexing | Indexing with DIALS]] - cppxfel uses scripts to call on DIALS indexing programs to index individual images.<br />
* [[Cppxfel Initial Orientation Matrix Refinement]] - performing initial refinement to produce high quality orientation matrices before post-refinement.<br />
* [[Cppxfel Post-refinement]] - post-refinement of crystals against a reference data set generated from the previous stage<br />
* [[Cppxfel Final Merge]] - producing a final MTZ file for use in structure solution.<br />
* [[Cppxfel Statistics]] - how to assess the quality of the XFEL data processing.<br />
<br />
== Reference manual ==<br />
<br />
The associated commands with the tutorials, in addition to several commands not used in the tutorials, are all documented in a [[Cppxfel Reference Manual | reference manual]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_Reference_ManualCppxfel Reference Manual2016-01-28T13:19:19Z<p>Helenginn: </p>
<hr />
<div>Here is a list of commands used in cppxfel and their effects on initial orientation matrix refinement or post-refinement.<br />
<br />
= List of commands =<br />
<br />
== General ==<br />
<br />
'''VERBOSITY_LEVEL x'''<br />
Integer x – number representing quantity of information sent to the standard output.<br />
Possible values:<br />
0 Normal (default)<br />
1 Detailed<br />
2 Debug<br />
<br />
'''MAX_THREADS x'''<br />
Integer x – number of threads to spread calculations over, but is preferentially overridden by setting the environment variable NSLOTS.<br />
<br />
'''MATRIX_LIST_VERSION x'''<br />
Double x – default 2.0. It is unlikely that you should ever need to use v1.0.<br />
<br />
'''INITIAL_MTZ'''<br />
String x - MTZ used as a starting reference data set in place of an initial merge. Useful for small numbers of images or to break the indexing ambiguity. No default.<br />
<br />
'''IMAGE_LIMIT'''<br />
Integer x - Only attempt to load x images from the list of matrices specified in ORIENTATION_MATRIX_LIST. Default 0 (no limit).<br />
<br />
'''NEW_MATRIX_LIST'''<br />
String x - Filename for the set of matrix lists (refine-x, integrate-x and merge-x) which may be produced from a round of integration or post-refinement. No default.<br />
<br />
'''SPACE_GROUP'''<br />
Int x – space group number for crystal according to the International Tables. No default.<br />
<br />
== Post-refinement ==<br />
<br />
'''MINIMUM_CYCLES x'''<br />
Integer x – minimum number of cycles of post-refinement to exceed before finishing (but may perform more cycles if not converged). Default 6.<br />
<br />
'''MAXIMUM_CYCLES x'''<br />
Integer x – maximum number of cycles of post-refinement to execute even if not converged. Default 0 (no maximum).<br />
<br />
'''STOP_REFINEMENT x'''<br />
Boolean x – if set to OFF, post-refinement will continue indefinitely.<br />
<br />
'''MINIMIZATION_METHOD x'''<br />
Integer x<br />
Possible values:<br />
0 Step search algorithm<br />
1 Nelder-Mead algorithm (default)<br />
<br />
'''NELDER_MEAD_CYCLES x'''<br />
Integer x – number of Nelder-Mead cycles to perform before finishing post-refinement of a single image per macrocycle. Default 50.<br />
<br />
'''MEDIAN_WAVELENGTH'''<br />
Boolean x – calculate starting X-ray beam wavelength for post-refinement of an image using the median excitation wavelength of all strong reflections. Otherwise a mean average is used. Default OFF.<br />
<br />
'''WAVELENGTH_RANGE x y'''<br />
Double x, y – start and end for range of wavelengths to consider when calculating the starting X-ray beam wavelength. Can ignore extreme outliers. Default 0 0 (not applied).<br />
<br />
'''REFINEMENT_INTENSITY_THRESHOLD x'''<br />
Double x - Intensity threshold x in absolute terms to define ‘strong’ reflections in initial wavelength determination. Default 200.<br />
<br />
'''ALLOW_TRUST x'''<br />
Boolean x - If an image correlates well with the reference data set, fix the indexing ambiguity chosen in the future to reduce computation time. Default ON<br />
<br />
'''TRUST_INDEXING_SOLUTION x'''<br />
Boolean x - Do not attempt to check alternative indexing solutions if set to ON. Good if the indexing ambiguity has been resolved by some other means. Default OFF.<br />
<br />
'''PARTIALITY_CUTOFF x'''<br />
'''Double x - If reflections are calculated with a cutoff below a certain partiality they are not included in target function calculation or merging. Default 0.2.'''<br />
<br />
'''SCALING_STRATEGY'''<br />
Int x – number representing the strategy for scaling individual crystals on each merging cycle. Default 1. Possible values:<br />
0 Average intensity set to 1000<br />
1 Scale to match the reference data set<br />
3 Minimise R merge as target function <br />
<br />
'''MINIMUM_REFLECTION_CUTOFF'''<br />
Int x – if a crystal refines to have fewer than x reflections then it is not included in the final merge. Default 100.<br />
<br />
'''DEFAULT_TARGET_FUNCTION'''<br />
Integer x - Target function used for post-refinement minimisation. Default 0.<br />
Possible values:<br />
0 R factor with reference data set<br />
1 Correlation with reference data set<br />
<br />
'''TARGET_FUNCTIONS x, y, z…'''<br />
Integers x, y, z… - Further target functions to occur during a single macrocycle after the default target function. For each additional function the calculation will be reperformed on the same image. Default: no extra target functions.<br />
<br />
'''RLP_MODEL x'''<br />
Integer x – model for reciprocal lattice point<br />
Possible values:<br />
0 Uniform sphere<br />
1 Gaussian decay from the centre<br />
<br />
'''CORRELATION_THRESHOLD x'''<br />
Double x – threshold for merging images; image which correlate less than x with reference will not be included in refinement. Also see INITIAL_CORRELATION_THRESHOLD. Default 0.9.<br />
<br />
'''INITIAL_CORRELATION_THRESHOLD x'''<br />
Double x – for a certain number of rounds of refinement determined by THRESHOLD_SWAP, a lower correlation threshold of x will be used. Also see CORRELATION_THRESHOLD. Default 0.8.<br />
<br />
'''THRESHOLD_SWAP x'''<br />
Integer x – this specifies the number of rounds of refinement before swapping from INITIAL_CORRELATION_THRESHOLD to CORRELATION_THRESHOLD. Default 2.<br />
<br />
'''PARTIALITY_CORRELATION_THRESHOLD x'''<br />
Double x – threshold for merging images, image whose partialities correlate less than x with the proportion of merged intensities of the reference will not be included in refinement. Default 0.<br />
<br />
'''R_FACTOR_THRESHOLD x'''<br />
Double x – in addition to correlation threshold, images with an R factor of more than x with the reference data set are rejected during merging. Default is 0 (not in use).<br />
<br />
'''MAX_RESOLUTION_ALL x'''<br />
Double x - do not use reflections in post-refinement beyond x Ã… resolution (but these will be included in the merge). Default 1.4.<br />
<br />
'''MIN_REFINED_RESOLUTION x'''<br />
Double x – do not refine using reflections below x Å resolution (but these will be included in the merge). Default 0 (no minimum).<br />
<br />
'''OUTLIER_REJECTION x'''<br />
Boolean x – master switch for rejection of outliers based on standard deviations from mean. Default ON.<br />
<br />
'''OUTLIER_REJECTION_SIGMA x'''<br />
Double x – number of standard deviations away from the mean intensity of a merged reflection beyond which an observation is rejected during merging. Default 1.8.<br />
<br />
'''CORRELATION_REJECTION x'''<br />
Boolean x – rejection on a per image basis if individual reflections correlate poorly with the image. Good for unindexed multiple lattices or bad-pixel detectors. Default ON.<br />
<br />
'''POLARISATION_CORRECTION x'''<br />
Boolean x – if switched on, polarisation factor is applied. Default OFF.<br />
<br />
'''POLARISATION_FACTOR x'''<br />
Double x – 0 for fully horizontal polarisation, 1 for fully vertical polarisation.<br />
<br />
'''REFINE_B_FACTOR x'''<br />
Boolean x - Only works when step search is enabled; a B factor is refined per image against the reference data set.<br />
<br />
'''PARTIALITY_SLICES x'''<br />
Integer x - Number of slices to make for individual reflections when calculating partiality higher than the resolution set by CAREFUL_RESOLUTION. Fewer slices can be made for smaller rlp sizes, increasing computation speed but running the risk of overfitting. Default 8, may need to be increased for high rlp size/mosaicity systems.<br />
<br />
'''MAX_SLICES x'''<br />
Integer x – Number of slices to make for reflections with a resolution lower than CAREFUL_RESOLUTION. May need to be increased for high rlp size/mosaicity systems. Default 100.<br />
<br />
'''CAREFUL_RESOLUTION x'''<br />
Double x – resolution at which reflections switch from using MAX_SLICES to PARTIALITY_SLICES for partiality calculation. Default 8 Å.<br />
<br />
'''REPLACE_REFERENCE'''<br />
If ON, reference is updated on each merging cycle (recommended). Otherwise reference is not replaced. Only generally used to measure degree of overfitting from refining a small number of images. Default ON.<br />
<br />
'''INITIAL_WAVELENGTH'''<br />
Does not need to be set, but if set, wavelength is always set to this value and not calculated per image. Possibly useful for very low resolution structures where individual spots deviate far from the true wavelength. Default 0 (not set).<br />
<br />
'''INITIAL_BANDWIDTH'''<br />
Initial bandwidth standard deviation for beam model. Multiply this by the wavelength to get the standard deviation in Ã…. Default 0.0013 (calibrated for a SASE pulse).<br />
<br />
'''INITIAL_MOSAICITY'''<br />
Initial mosaicity for rlp size determining how much the rlp increases with resolution in degrees. Defined according to Rossmann et al 1979 in degrees. Default 0º.<br />
<br />
'''INITIAL_EXPONENT'''<br />
Initial exponent for super-Gaussian distribution of beam wavelengths. Default 1.5.<br />
<br />
'''INITIAL_RLP_SIZE'''<br />
Size of theoretical Miller index (0, 0, 0) in reciprocal space distance (Ã…<sup>-1</sup>). Default 0.0001 Ã…<sup>-1</sup>.<br />
<br />
'''STEP_SIZE_WAVELENGTH x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for wavelength. Default 0.005 Ã….<br />
<br />
'''STEP_SIZE_BANDWIDTH x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for bandwidth. Default 0.0003.<br />
<br />
'''STEP_SIZE_MOSAICITY x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for mosaicity. Default 0.001.<br />
<br />
'''STEP_SIZE_EXPONENT x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for the super-Gaussian exponent. Default 0.1.<br />
<br />
'''STEP_SIZE_ORIENTATION x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for horizontal/vertical rotation displacement (degrees). Default 0.06º.<br />
<br />
'''STEP_SIZE_RLP_SIZE x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for base rlp size. Default 2 × 10<sup>-5</sup> Å<sup>-1</sup>.<br />
<br />
'''STEP_SIZE_UNIT_CELL_A x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for unit cell dimension A. Default 0.2 Ã….<br />
<br />
'''STEP_SIZE_UNIT_CELL_B x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for unit cell dimension B. Default 0.2 Ã….<br />
<br />
'''STEP_SIZE_UNIT_CELL_C x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for unit cell dimension C. Default 0.2 Ã….<br />
<br />
'''TOLERANCE_WAVELENGTH x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; wavelength. Default 1 × 10<sup>-5</sup> Å.<br />
<br />
'''TOLERANCE_BANDWIDTH x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; bandwidth. Default 1 × 10<sup>-5</sup>.<br />
<br />
'''TOLERANCE_MOSAICITY x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; mosaicity. Default 1 × 10<sup>-3</sup>º.<br />
<br />
'''TOLERANCE_EXPONENT x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; super-Gaussian exponent. Default 5 × 10<sup>-3</sup>.<br />
<br />
'''TOLERANCE_ORIENTATION x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; horizontal/vertical orientation displacement (degrees). Default 1 × 10<sup>-4</sup>º.<br />
<br />
'''TOLERANCE_RLP_SIZE x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; base rlp size. Default 1 × 10<sup>-5</sup> Å<sup>-1</sup>.<br />
<br />
'''OPTIMISING_WAVELENGTH x'''<br />
Boolean x – sets whether wavelength is a refined parameter. Default ON.<br />
<br />
'''OPTIMISING_BANDWIDTH x'''<br />
Boolean x – sets whether bandwidth is a refined parameter. Default OFF.<br />
<br />
'''OPTIMISING_MOSAICITY x'''<br />
Boolean x – sets whether mosaicity is a refined parameter. Default OFF.<br />
<br />
'''OPTIMISING_EXPONENT x'''<br />
Boolean x – sets whether exponent is a refined parameter. Default OFF.<br />
<br />
'''OPTIMISING_ORIENTATION x'''<br />
Boolean x – sets whether horizontal/vertical orientation displacement is a refined parameter. Default ON<br />
<br />
'''OPTIMISING_RLP_SIZE x'''<br />
Boolean x – sets whether rlp size is a refined parameter. Default ON.<br />
<br />
'''OPTIMISING_UNIT_CELL_A x'''<br />
Boolean x – sets whether unit cell dimension A is a refined parameter. Default OFF, should be turned on for certain space groups.<br />
<br />
'''OPTIMISING_UNIT_CELL_B x'''<br />
Boolean x – sets whether unit cell dimension B is a refined parameter. Default OFF, should be turned on for certain space groups.<br />
<br />
'''OPTIMISING_UNIT_CELL_C x'''<br />
Boolean x – sets whether unit cell dimension C is a refined parameter. Default OFF, should be turned on for certain space groups.<br />
<br />
'''ORIENTATION_MATRIX_LIST x'''<br />
String x - path to list of orientation matrices generated using cppxfel.input_gen. No default.<br />
<br />
== Integration ==<br />
<br />
'''INTEGRATION_WAVELENGTH'''<br />
Double x – wavelength of X-ray beam in Å. No default.<br />
<br />
'''DETECTOR_DISTANCE'''<br />
Double x – detector distance from crystal in mm. No default.<br />
<br />
'''BEAM_CENTRE'''<br />
Double x, y – coordinate position at which the beam hits the image in pixels. Default 882.0, 882.0 px to match CSPAD detector.<br />
<br />
'''MM_PER_PIXEL'''<br />
Double x – height/width of a pixel on the detector in mm. Default 0.11 mm to match CSPAD detector.<br />
<br />
'''DETECTOR_SIZE'''<br />
Int x, y – width and height of the detector. Default 1765, 1765 to match CSPAD detector.<br />
<br />
'''OVER_PRED_BANDWIDTH'''<br />
Double x – integration requires an over-estimated bandwidth to ‘catch’ all the reflections and more. This represents the % bandwidth to either side of the mean wavelength which should be used. Default 3%.<br />
<br />
'''OVER_PRED_RLP_SIZE'''<br />
Double x – reciprocal lattice point radius in Å-1 at the Miller index position (0, 0, 0) (i.e. without consideration of mosaicity). Default 2 × 10<sup>-4</sup> Å<sup>-1</sup>.<br />
<br />
'''REFINE_ORIENTATIONS'''<br />
Boolean x – if set to ON, using the INTEGRATE command will also refine orientation matrices according to the protocol in Ginn et al Nat Comms 2015. Default ON.<br />
<br />
'''INDEXING_ORIENTATION_TOLERANCE'''<br />
Double x – Initial orientation matrix refinement will finish when the step search angle decreases below x degrees. Default 1 × 10<sup>-3</sup>º.<br />
<br />
'''INTENSITY_THRESHOLD'''<br />
Double x - Absolute (counts) or signal-to-noise (I/sigI) threshold at which a spot is considered to be a strong spot for initial orientation matrix refinement. Default 12 (assuming a detector gain of 1.0) and also dependent on ABSOLUTE_INTENSITY.<br />
<br />
'''ABSOLUTE_INTENSITY'''<br />
Boolean x – if set to ON, intensity threshold is measured in counts, whereas if set to OFF this is a I/sigI threshold for spots to be considered strong spots for initial orientation matrix refinement. Default OFF.<br />
<br />
'''METROLOGY_SEARCH_SIZE'''<br />
Int x – metrology search size can be set to an integer number of pixels from which the predicted spot position may deviate in order to find the highest pixel value. Integration will be performed centred on this highest pixel value. Default 0 but often needs changing.<br />
<br />
'''SHOEBOX_FOREGROUND_PADDING'''<br />
Int x – for a simple shoebox centred on a coordinate, this represents the extra number of pixels in both axes which will be integrated as part of the signal foreground. This overrides the background and neither flags. Default 1.<br />
<br />
'''SHOEBOX_NEITHER_PADDING'''<br />
Int x - – for a simple shoebox centred on a coordinate, this represents the extra number of pixels from the centre in both axes for which pixels will not be considered as part of foreground or background signal. This overrides the background flags. Default 2.<br />
<br />
'''SHOEBOX_BACKGROUND_PADDING'''<br />
Int x - – for a simple shoebox centred on a coordinate, this represents the extra number of pixels from the centre in both axes for which pixels will be used to estimate the background. Default 3.<br />
<br />
'''COMPLEX_SHOEBOX'''<br />
Boolean x – if this is set, a complex shoebox will be generated to attempt to take into account the shape of the spot at high resolution, which may be distorted by the wavelength spread of the beam.<br />
<br />
'''MAX_INTEGRATED_RESOLUTION'''<br />
Double x – if set, integration will only occur up to the specified resolution. Default 1.4 Å.<br />
<br />
'''UNIT_CELL'''<br />
Double a, b, c, alpha, beta, gamma – unit cell parameters for the crystal of interest.<br />
<br />
'''FIX_UNIT_CELL'''<br />
Boolean x – if set to ON, the unit cell found in the ORIENTATION_MATRIX_LIST file will have its unit cell reset to the values specified in UNIT_CELL, preserving orientation, before any further integration or post-refinement.<br />
<br />
'''INITIAL_ORIENTATION_STEP'''<br />
Double x – for initial orientation matrix refinement, this is the angle in degrees that the algorithm steps in order to search for a new minimum. Step search algorithm is used. Default 1.0º.<br />
<br />
'''ORIENTATION_SCORE'''<br />
Int x – target function during initial orientation matrix refinement. Default 0 (weighted combination of total reflections and standard deviation). Values of interest:<br />
0 weighted combination of total reflections within bandwidth range and standard deviation<br />
1 total reflections within bandwidth range only<br />
<br />
'''ORIENTATION_CORRECTION'''<br />
Double x,y – sometimes the indexing solutions from DIALS are corrected by a systematic error in the horizontal and vertical rotations due to a component of the experiment which is modelled in DIALS but not by cppxfel. If this systematic shift is known, it can be supplied in degrees (x in the horizontal axis and y in the vertical axis). These values may be deduced from a table of corrections produced at the end of an INTEGRATE command. Default 0,0.<br />
<br />
'''IMAGE_MASKED_VALUE'''<br />
Int x – if there is a pixel value used as a masking value in the images provided, this can be set using this command. If IMAGE_MASKED_VALUE is not set, the default is not to mask any pixel value.<br />
<br />
'''SPHERE_THICKNESS'''<br />
Double x - This is the thickness in Ã…<sup>-1</sup> around the mean Ewald radius to consider integrating. The subset of reflections which are chosen for integration are then determined by OVER_PRED_RLP_SIZE, so SPHERE_THICKNESS is a prior scanning step. Default 0.02 Ã…<sup>-1</sup>. Increasing this will improve spot prediction at low resolution but will reduce the overall speed of the program.<br />
<br />
'''PIXEL_COUNT_CUTOFF'''<br />
Int x - If, during integration, a pixel exceeds the value x, this reflection is rejected. Can be used to protect against non-linear gain of the detector at high pixel counts. Default is not to use.<br />
<br />
'''REFINE_UNIT_CELL_A'''<br />
Boolean x – if set to ON, this unit cell a dimension is refined in either integration or post-refinement, using the step size specified by STEP_UNIT_CELL_A. Default OFF.<br />
<br />
'''REFINE_UNIT_CELL_B'''<br />
Boolean x – if set to ON, this unit cell b dimension is refined in either initial orientation matrix refinement or post-refinement, using the step size specified by STEP_UNIT_CELL_B. Default OFF.<br />
<br />
'''REFINE_UNIT_CELL_C'''<br />
Boolean x – if set to ON, this unit cell c dimension is refined in either initial orientation matrix refinement or post-refinement, using the step size specified by STEP_UNIT_CELL_C. Default OFF.<br />
<br />
'''STEP_UNIT_CELL_A'''<br />
Double x – this determines the step size used during either initial orientation matrix refinement or post-refinement for unit cell A. Default 0.2 Å.<br />
<br />
'''STEP_UNIT_CELL_B'''<br />
Double x – this determines the step size used during either initial orientation matrix refinement or post-refinement for unit cell B. Default 0.2 Å.<br />
<br />
'''STEP_UNIT_CELL_C'''<br />
Double x – this determines the step size used during either initial orientation matrix refinement or post-refinement for unit cell C. Default 0.2 Å.<br />
<br />
'''FROM_DIALS'''<br />
Boolean x – should be set to ON if matrices were calculated by DIALS. Unlikely to be any other value. Default ON.<br />
<br />
'''DO_NOT_REJECT_REFLECTIONS'''<br />
Boolean x – usually, if there are multiple lattices on the same image then overlapping shoeboxes lead to reflections being rejected. This also applies to overlapping reflections from the same lattice, if the over-prediction is too wide. This may not always be a desired feature, so setting this value to OFF will disable it. Default ON.<br />
<br />
'''REFINE_IN_PLANE_OF_DETECTOR'''<br />
Boolean x – in some cases, the rotation of the orientation matrix around the beam axis is not exact. Setting this value to ON will refine the orientation matrix around this axis. Default OFF.<br />
<br />
'''FIT_BACKGROUND_AS_PLANE'''<br />
Boolean x – to fit the background pixels as a plane rather than simple summation, switch this to ON. Individual pixels may also be rejected as outliers. Default OFF.<br />
<br />
'''PANEL_LIST'''<br />
String x – list of panels generated by cppxfel.input_gen or modified by REFINE_METROLOGY command. Individual panels in this file are specified by the keyword PANEL followed by the upper left x,y and bottom right x,y integer coordinates separated by single spaces. Required for initial orientation matrix refinement but not post-refinement. No default.<br />
<br />
'''ROUGH_CALCULATION''' - v1.1<br />
Bool x - if switched to ON, some recalculations are skipped during integration, which vastly increases the speed of integration. Comes with a mild warning that the integration solutions may be compromised - in practice this appears to be fine. Default OFF.<br />
<br />
== Merging ==<br />
<br />
'''RECALCULATE_SIGMA'''<br />
Boolean x - To be used on a final merge of MTZ files: an estimate of the standard deviation will be made from the distribution of the observation intensities for each reflection. Default OFF.<br />
<br />
'''MERGE_ANOMALOUS'''<br />
Boolean x – to be used on a final merge of MTZ files, this flag will merge anomalous data sets rather than straight intensities. Anomalous data will be written to anomalous_diff.mtz. Default OFF.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Updating_cppxfelUpdating cppxfel2016-01-28T13:13:44Z<p>Helenginn: Created page with "In the case of updating the cppxfel installation, open a terminal and navigate to the <code>cppxfel</code> directory. <pre>cd /absolute/path/to/dials/dials-dev-508/modules/cp..."</p>
<hr />
<div>In the case of updating the cppxfel installation, open a terminal and navigate to the <code>cppxfel</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/dials-dev-508/modules/cppxfel</pre><br />
<br />
Then, pull the latest code from the repository.<br />
<br />
<pre>git pull</pre><br />
<br />
If necessary, some command line tools may need to be configured. To do this, type<br />
<br />
<pre>libtbx.configure cppxfel</pre><br />
<br />
Then, the C++ code will need recompiling.<br />
<br />
<pre><br />
cd ../../build<br />
make<br />
</pre><br />
<br />
The new distribution of cppxfel will be available to use.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T16:30:57Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
These installation instructions have been tested on CentOS 7.<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]) from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November 2015 or later for Boost thread support. Extract the dials-installer-dev to a new location. Navigate into this directory, e.g.:<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory. Navigate to this directory, e.g.<br />
<br />
<pre>cd /absolute/path/to/dials/dials-dev-508/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/dials-dev-508/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/dials-dev-508/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/dials-dev-508/build/setpaths.sh && \<br />
. /absolute/path/to/dials/dials-dev-508/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/dials-dev-508/build/setpaths.csh && \<br />
source /absolute/path/to/dials/dials-dev-508/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:50:09Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
These installation instructions have been tested on CentOS 7.<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]) from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November 2015 or later for Boost thread support. Extract the dials-installer-dev to a new location. Navigate into this directory, e.g.:<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory. Navigate to this directory, e.g.<br />
<br />
<pre>cd /absolute/path/to/dials/dials-dev-508/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/dials-dev-508/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/dials-dev-508/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/dials-dev-508/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/dials-dev-508/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:49:29Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
These installation instructions have been tested on CentOS 7.<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]) from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November 2015 or later for Boost thread support. Extract the dials-installer-dev to a new location. Navigate into this directory, e.g.:<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory. Navigate to this directory, e.g.<br />
<br />
<pre>cd /absolute/path/to/dials/dials-dev-508/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:43:19Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
These installation instructions have been tested on CentOS 7.<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]) from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November 2015 or later for Boost thread support. Extract the dials-installer-dev to a new location. Navigate into this directory, e.g.:<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:10:00Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
These installation instructions have been tested on CentOS 7.<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]) from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November or later for Boost thread support. Extract the dials-installer-dev to a new location. Navigate into this directory, e.g.:<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:09:02Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
'''These installation instructions are not finished and will not currently result in a working version of cppxfel.'''<br />
<br />
<!--<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
--><br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November or later (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]). Extract the dials-installer-dev to a new location. Navigate into this directory, e.g.:<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:08:36Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
'''These installation instructions are not finished and will not currently result in a working version of cppxfel.'''<br />
<br />
<!--<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
--><br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November or later (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]). Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:07:54Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
'''These installation instructions are not finished and will not currently result in a working version of cppxfel.'''<br />
<br />
<!--<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
--><br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November or later (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]). Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library. Consider adding these links to your <code>~/.bashrc</code> or <code>~/.cshrc</code> files to be executed on the generation of every new shell.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:06:37Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
'''These installation instructions are not finished and will not currently result in a working version of cppxfel.'''<br />
<br />
<!--<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
--><br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November or later (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]). Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.<br />
<br />
== Check the installation ==<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-30T15:05:50Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
'''These installation instructions are not finished and will not currently result in a working version of cppxfel.'''<br />
<br />
<!--<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
--><br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Ensure the distribution downloaded was compiled on 29th November or later (see [[http://cci.lbl.gov/dials/installers/ list of latest builds]]). Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The build directory must be removed and recreated.<br />
<br />
<pre><br />
cd ..<br />
rm -r build<br />
mkdir build<br />
cd build<br />
</pre><br />
<br />
The DIALS distribution must be configured to compile the xfel extension, and then the distribution must be compiled.<br />
<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 xfel<br />
make<br />
</pre><br />
<br />
The same must be done for the cppxfel code.<br />
<pre><br />
../base/bin/python ../modules/cctbx_project/libtbx/configure.py --enable_boost_threads=1 cppxfel<br />
make<br />
</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-26T00:20:37Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
'''These installation instructions are not finished and will not currently result in a working version of cppxfel.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-26T00:19:45Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. ''cppxfel'' relies on three components:<br />
* Boost libraries (''thread'' and ''system'') - these allow C++ to access threading over multiple cores.<br />
* DIALS distribution - provides the ''cctbx'' libraries for L-BFGS refinement and some crystallographic functions, and the indexing algorithms from DIALS itself.<br />
* ''cppxfel'' distribution - primarily written in C++, but some Python scripts to automate DIALS indexing and prepare the results for input into ''cppxfel''.<br />
<br />
'''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-26T00:15:52Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
You can use a package distributor to install the appropriate Boost package according to your OS flavour:<br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-26T00:15:09Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
Download the Boost source code from the Boost website. For example, [[http://www.boost.org/users/history/version_1_59_0.html Version 1.59.0]]. Follow the instructions in Section 5 in Boost's [[http://www.boost.org/doc/libs/1_59_0/more/getting_started/unix-variants.html Getting Started - Unix Variants]] page to compile the boost libraries required by ''cppxfel''. Install these files to a suitable location by specifying an appropriate prefix that you have write privileges to.<br />
<br />
Because you do not have user privileges and cannot install the libraries to a standard place, you will have to set the environment variable LD_LIBRARY_PATH to the path of the library, and you will also have to set the environment variable BOOST_LOCATION in order that ''cppxfel'' can find your boost libraries. You may want to append these to your .bashrc or .cshrc files.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/path/to/boost/lib/"</code><br />
| <code>setenv BOOST_LOCATION /path/to/boost/lib/</code><br />
|-<br />
|<code>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH":/path/to/boost/lib/"</code><br />
|<code>setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:/path/to/boost/lib/</code><br />
|-<br />
| Enter last line into ~/.bashrc<br />
| Enter last line into ~/.cshrc<br />
|}<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_Reference_ManualCppxfel Reference Manual2015-11-25T23:13:40Z<p>Helenginn: </p>
<hr />
<div>Here is a list of commands used in cppxfel and their effects on initial orientation matrix refinement or post-refinement.<br />
<br />
= List of commands =<br />
<br />
== General ==<br />
<br />
'''VERBOSITY_LEVEL x'''<br />
Integer x – number representing quantity of information sent to the standard output.<br />
Possible values:<br />
0 Normal (default)<br />
1 Detailed<br />
2 Debug<br />
<br />
'''MAX_THREADS x'''<br />
Integer x – number of threads to spread calculations over, but is preferentially overridden by setting the environment variable NSLOTS.<br />
<br />
'''MATRIX_LIST_VERSION x'''<br />
Double x – default 2.0. It is unlikely that you should ever need to use v1.0.<br />
<br />
'''INITIAL_MTZ'''<br />
String x - MTZ used as a starting reference data set in place of an initial merge. Useful for small numbers of images or to break the indexing ambiguity. No default.<br />
<br />
'''IMAGE_LIMIT'''<br />
Integer x - Only attempt to load x images from the list of matrices specified in ORIENTATION_MATRIX_LIST. Default 0 (no limit).<br />
<br />
'''NEW_MATRIX_LIST'''<br />
String x - Filename for the set of matrix lists (refine-x, integrate-x and merge-x) which may be produced from a round of integration or post-refinement. No default.<br />
<br />
'''SPACE_GROUP'''<br />
Int x – space group number for crystal according to the International Tables. No default.<br />
<br />
== Post-refinement ==<br />
<br />
'''MINIMUM_CYCLES x'''<br />
Integer x – minimum number of cycles of post-refinement to exceed before finishing (but may perform more cycles if not converged). Default 6.<br />
<br />
'''MAXIMUM_CYCLES x'''<br />
Integer x – maximum number of cycles of post-refinement to execute even if not converged. Default 0 (no maximum).<br />
<br />
'''STOP_REFINEMENT x'''<br />
Boolean x – if set to OFF, post-refinement will continue indefinitely.<br />
<br />
'''MINIMIZATION_METHOD x'''<br />
Integer x<br />
Possible values:<br />
0 Step search algorithm<br />
1 Nelder-Mead algorithm (default)<br />
<br />
'''NELDER_MEAD_CYCLES x'''<br />
Integer x – number of Nelder-Mead cycles to perform before finishing post-refinement of a single image per macrocycle. Default 50.<br />
<br />
'''MEDIAN_WAVELENGTH'''<br />
Boolean x – calculate starting X-ray beam wavelength for post-refinement of an image using the median excitation wavelength of all strong reflections. Otherwise a mean average is used. Default OFF.<br />
<br />
'''WAVELENGTH_RANGE x y'''<br />
Double x, y – start and end for range of wavelengths to consider when calculating the starting X-ray beam wavelength. Can ignore extreme outliers. Default 0 0 (not applied).<br />
<br />
'''REFINEMENT_INTENSITY_THRESHOLD x'''<br />
Double x - Intensity threshold x in absolute terms to define ‘strong’ reflections in initial wavelength determination. Default 200.<br />
<br />
'''ALLOW_TRUST x'''<br />
Boolean x - If an image correlates well with the reference data set, fix the indexing ambiguity chosen in the future to reduce computation time. Default ON<br />
<br />
'''TRUST_INDEXING_SOLUTION x'''<br />
Boolean x - Do not attempt to check alternative indexing solutions if set to ON. Good if the indexing ambiguity has been resolved by some other means. Default OFF.<br />
<br />
'''PARTIALITY_CUTOFF x'''<br />
'''Double x - If reflections are calculated with a cutoff below a certain partiality they are not included in target function calculation or merging. Default 0.2.'''<br />
<br />
'''SCALING_STRATEGY'''<br />
Int x – number representing the strategy for scaling individual crystals on each merging cycle. Default 1. Possible values:<br />
0 Average intensity set to 1000<br />
1 Scale to match the reference data set<br />
3 Minimise R merge as target function <br />
<br />
'''MINIMUM_REFLECTION_CUTOFF'''<br />
Int x – if a crystal refines to have fewer than x reflections then it is not included in the final merge. Default 100.<br />
<br />
'''DEFAULT_TARGET_FUNCTION'''<br />
Integer x - Target function used for post-refinement minimisation. Default 0.<br />
Possible values:<br />
0 R factor with reference data set<br />
1 Correlation with reference data set<br />
<br />
'''TARGET_FUNCTIONS x, y, z…'''<br />
Integers x, y, z… - Further target functions to occur during a single macrocycle after the default target function. For each additional function the calculation will be reperformed on the same image. Default: no extra target functions.<br />
<br />
'''RLP_MODEL x'''<br />
Integer x – model for reciprocal lattice point<br />
Possible values:<br />
0 Uniform sphere<br />
1 Gaussian decay from the centre<br />
<br />
'''CORRELATION_THRESHOLD x'''<br />
Double x – threshold for merging images; image which correlate less than x with reference will not be included in refinement. Also see INITIAL_CORRELATION_THRESHOLD. Default 0.9.<br />
<br />
'''INITIAL_CORRELATION_THRESHOLD x'''<br />
Double x – for a certain number of rounds of refinement determined by THRESHOLD_SWAP, a lower correlation threshold of x will be used. Also see CORRELATION_THRESHOLD. Default 0.8.<br />
<br />
'''THRESHOLD_SWAP x'''<br />
Integer x – this specifies the number of rounds of refinement before swapping from INITIAL_CORRELATION_THRESHOLD to CORRELATION_THRESHOLD. Default 2.<br />
<br />
'''PARTIALITY_CORRELATION_THRESHOLD x'''<br />
Double x – threshold for merging images, image whose partialities correlate less than x with the proportion of merged intensities of the reference will not be included in refinement. Default 0.<br />
<br />
'''R_FACTOR_THRESHOLD x'''<br />
Double x – in addition to correlation threshold, images with an R factor of more than x with the reference data set are rejected during merging. Default is 0 (not in use).<br />
<br />
'''MAX_RESOLUTION_ALL x'''<br />
Double x - do not use reflections in post-refinement beyond x Ã… resolution (but these will be included in the merge). Default 1.4.<br />
<br />
'''MIN_REFINED_RESOLUTION x'''<br />
Double x – do not refine using reflections below x Å resolution (but these will be included in the merge). Default 0 (no minimum).<br />
<br />
'''OUTLIER_REJECTION x'''<br />
Boolean x – master switch for rejection of outliers based on standard deviations from mean. Default ON.<br />
<br />
'''OUTLIER_REJECTION_SIGMA x'''<br />
Double x – number of standard deviations away from the mean intensity of a merged reflection beyond which an observation is rejected during merging. Default 1.8.<br />
<br />
'''CORRELATION_REJECTION x'''<br />
Boolean x – rejection on a per image basis if individual reflections correlate poorly with the image. Good for unindexed multiple lattices or bad-pixel detectors. Default ON.<br />
<br />
'''POLARISATION_CORRECTION x'''<br />
Boolean x – if switched on, polarisation factor is applied. Default OFF.<br />
<br />
'''POLARISATION_FACTOR x'''<br />
Double x – 0 for fully horizontal polarisation, 1 for fully vertical polarisation.<br />
<br />
'''REFINE_B_FACTOR x'''<br />
Boolean x - Only works when step search is enabled; a B factor is refined per image against the reference data set.<br />
<br />
'''PARTIALITY_SLICES x'''<br />
Integer x - Number of slices to make for individual reflections when calculating partiality higher than the resolution set by CAREFUL_RESOLUTION. Fewer slices can be made for smaller rlp sizes, increasing computation speed but running the risk of overfitting. Default 8, may need to be increased for high rlp size/mosaicity systems.<br />
<br />
'''MAX_SLICES x'''<br />
Integer x – Number of slices to make for reflections with a resolution lower than CAREFUL_RESOLUTION. May need to be increased for high rlp size/mosaicity systems. Default 100.<br />
<br />
'''CAREFUL_RESOLUTION x'''<br />
Double x – resolution at which reflections switch from using MAX_SLICES to PARTIALITY_SLICES for partiality calculation. Default 8 Å.<br />
<br />
'''REPLACE_REFERENCE'''<br />
If ON, reference is updated on each merging cycle (recommended). Otherwise reference is not replaced. Only generally used to measure degree of overfitting from refining a small number of images. Default ON.<br />
<br />
'''INITIAL_WAVELENGTH'''<br />
Does not need to be set, but if set, wavelength is always set to this value and not calculated per image. Possibly useful for very low resolution structures where individual spots deviate far from the true wavelength. Default 0 (not set).<br />
<br />
'''INITIAL_BANDWIDTH'''<br />
Initial bandwidth standard deviation for beam model. Multiply this by the wavelength to get the standard deviation in Ã…. Default 0.0013 (calibrated for a SASE pulse).<br />
<br />
'''INITIAL_MOSAICITY'''<br />
Initial mosaicity for rlp size determining how much the rlp increases with resolution in degrees. Defined according to Rossmann et al 1979 in degrees. Default 0º.<br />
<br />
'''INITIAL_EXPONENT'''<br />
Initial exponent for super-Gaussian distribution of beam wavelengths. Default 1.5.<br />
<br />
'''INITIAL_RLP_SIZE'''<br />
Size of theoretical Miller index (0, 0, 0) in reciprocal space distance (Ã…<sup>-1</sup>). Default 0.0001 Ã…<sup>-1</sup>.<br />
<br />
'''STEP_SIZE_WAVELENGTH x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for wavelength. Default 0.005 Ã….<br />
<br />
'''STEP_SIZE_BANDWIDTH x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for bandwidth. Default 0.0003.<br />
<br />
'''STEP_SIZE_MOSAICITY x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for mosaicity. Default 0.001.<br />
<br />
'''STEP_SIZE_EXPONENT x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for the super-Gaussian exponent. Default 0.1.<br />
<br />
'''STEP_SIZE_ORIENTATION x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for horizontal/vertical rotation displacement (degrees). Default 0.06º.<br />
<br />
'''STEP_SIZE_RLP_SIZE x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for base rlp size. Default 2 × 10<sup>-5</sup> Å<sup>-1</sup>.<br />
<br />
'''STEP_SIZE_UNIT_CELL_A x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for unit cell dimension A. Default 0.2 Ã….<br />
<br />
'''STEP_SIZE_UNIT_CELL_B x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for unit cell dimension B. Default 0.2 Ã….<br />
<br />
'''STEP_SIZE_UNIT_CELL_C x'''<br />
Double x - Initial step size or initial displacement for step search or Nelder-Mead algorithm respectively for unit cell dimension C. Default 0.2 Ã….<br />
<br />
'''TOLERANCE_WAVELENGTH x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; wavelength. Default 1 × 10<sup>-5</sup> Å.<br />
<br />
'''TOLERANCE_BANDWIDTH x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; bandwidth. Default 1 × 10<sup>-5</sup>.<br />
<br />
'''TOLERANCE_MOSAICITY x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; mosaicity. Default 1 × 10<sup>-3</sup>º.<br />
<br />
'''TOLERANCE_EXPONENT x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; super-Gaussian exponent. Default 5 × 10<sup>-3</sup>.<br />
<br />
'''TOLERANCE_ORIENTATION x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; horizontal/vertical orientation displacement (degrees). Default 1 × 10<sup>-4</sup>º.<br />
<br />
'''TOLERANCE_RLP_SIZE x'''<br />
Double x – Step size search algorithm only – when step search size falls below this tolerance, stop parameter refinement; base rlp size. Default 1 × 10<sup>-5</sup> Å<sup>-1</sup>.<br />
<br />
'''OPTIMISING_WAVELENGTH x'''<br />
Boolean x – sets whether wavelength is a refined parameter. Default ON.<br />
<br />
'''OPTIMISING_BANDWIDTH x'''<br />
Boolean x – sets whether bandwidth is a refined parameter. Default OFF.<br />
<br />
'''OPTIMISING_MOSAICITY x'''<br />
Boolean x – sets whether mosaicity is a refined parameter. Default OFF.<br />
<br />
'''OPTIMISING_EXPONENT x'''<br />
Boolean x – sets whether exponent is a refined parameter. Default OFF.<br />
<br />
'''OPTIMISING_ORIENTATION x'''<br />
Boolean x – sets whether horizontal/vertical orientation displacement is a refined parameter. Default ON<br />
<br />
'''OPTIMISING_RLP_SIZE x'''<br />
Boolean x – sets whether rlp size is a refined parameter. Default ON.<br />
<br />
'''OPTIMISING_UNIT_CELL_A x'''<br />
Boolean x – sets whether unit cell dimension A is a refined parameter. Default OFF, should be turned on for certain space groups.<br />
<br />
'''OPTIMISING_UNIT_CELL_B x'''<br />
Boolean x – sets whether unit cell dimension B is a refined parameter. Default OFF, should be turned on for certain space groups.<br />
<br />
'''OPTIMISING_UNIT_CELL_C x'''<br />
Boolean x – sets whether unit cell dimension C is a refined parameter. Default OFF, should be turned on for certain space groups.<br />
<br />
'''ORIENTATION_MATRIX_LIST x'''<br />
String x - path to list of orientation matrices generated using cppxfel.input_gen. No default.<br />
<br />
== Integration ==<br />
<br />
'''INTEGRATION_WAVELENGTH'''<br />
Double x – wavelength of X-ray beam in Å. No default.<br />
<br />
'''DETECTOR_DISTANCE'''<br />
Double x – detector distance from crystal in mm. No default.<br />
<br />
'''BEAM_CENTRE'''<br />
Double x, y – coordinate position at which the beam hits the image in pixels. Default 882.0, 882.0 px to match CSPAD detector.<br />
<br />
'''MM_PER_PIXEL'''<br />
Double x – height/width of a pixel on the detector in mm. Default 0.11 mm to match CSPAD detector.<br />
<br />
'''DETECTOR_SIZE'''<br />
Int x, y – width and height of the detector. Default 1765, 1765 to match CSPAD detector.<br />
<br />
'''OVER_PRED_BANDWIDTH'''<br />
Double x – integration requires an over-estimated bandwidth to ‘catch’ all the reflections and more. This represents the % bandwidth to either side of the mean wavelength which should be used. Default 3%.<br />
<br />
'''OVER_PRED_RLP_SIZE'''<br />
Double x – reciprocal lattice point radius in Å-1 at the Miller index position (0, 0, 0) (i.e. without consideration of mosaicity). Default 2 × 10<sup>-4</sup> Å<sup>-1</sup>.<br />
<br />
'''REFINE_ORIENTATIONS'''<br />
Boolean x – if set to ON, using the INTEGRATE command will also refine orientation matrices according to the protocol in Ginn et al Nat Comms 2015. Default ON.<br />
<br />
'''INDEXING_ORIENTATION_TOLERANCE'''<br />
Double x – Initial orientation matrix refinement will finish when the step search angle decreases below x degrees. Default 1 × 10<sup>-3</sup>º.<br />
<br />
'''INTENSITY_THRESHOLD'''<br />
Double x - Absolute (counts) or signal-to-noise (I/sigI) threshold at which a spot is considered to be a strong spot for initial orientation matrix refinement. Default 12 (assuming a detector gain of 1.0) and also dependent on ABSOLUTE_INTENSITY.<br />
<br />
'''ABSOLUTE_INTENSITY'''<br />
Boolean x – if set to ON, intensity threshold is measured in counts, whereas if set to OFF this is a I/sigI threshold for spots to be considered strong spots for initial orientation matrix refinement. Default OFF.<br />
<br />
'''METROLOGY_SEARCH_SIZE'''<br />
Int x – metrology search size can be set to an integer number of pixels from which the predicted spot position may deviate in order to find the highest pixel value. Integration will be performed centred on this highest pixel value. Default 0 but often needs changing.<br />
<br />
'''SHOEBOX_FOREGROUND_PADDING'''<br />
Int x – for a simple shoebox centred on a coordinate, this represents the extra number of pixels in both axes which will be integrated as part of the signal foreground. This overrides the background and neither flags. Default 1.<br />
<br />
'''SHOEBOX_NEITHER_PADDING'''<br />
Int x - – for a simple shoebox centred on a coordinate, this represents the extra number of pixels from the centre in both axes for which pixels will not be considered as part of foreground or background signal. This overrides the background flags. Default 2.<br />
<br />
'''SHOEBOX_BACKGROUND_PADDING'''<br />
Int x - – for a simple shoebox centred on a coordinate, this represents the extra number of pixels from the centre in both axes for which pixels will be used to estimate the background. Default 3.<br />
<br />
'''COMPLEX_SHOEBOX'''<br />
Boolean x – if this is set, a complex shoebox will be generated to attempt to take into account the shape of the spot at high resolution, which may be distorted by the wavelength spread of the beam.<br />
<br />
'''MAX_INTEGRATED_RESOLUTION'''<br />
Double x – if set, integration will only occur up to the specified resolution. Default 1.4 Å.<br />
<br />
'''UNIT_CELL'''<br />
Double a, b, c, alpha, beta, gamma – unit cell parameters for the crystal of interest.<br />
<br />
'''FIX_UNIT_CELL'''<br />
Boolean x – if set to ON, the unit cell found in the ORIENTATION_MATRIX_LIST file will have its unit cell reset to the values specified in UNIT_CELL, preserving orientation, before any further integration or post-refinement.<br />
<br />
'''INITIAL_ORIENTATION_STEP'''<br />
Double x – for initial orientation matrix refinement, this is the angle in degrees that the algorithm steps in order to search for a new minimum. Step search algorithm is used. Default 1.0º.<br />
<br />
'''ORIENTATION_SCORE'''<br />
Int x – target function during initial orientation matrix refinement. Default 0 (weighted combination of total reflections and standard deviation). Values of interest:<br />
0 weighted combination of total reflections within bandwidth range and standard deviation<br />
1 total reflections within bandwidth range only<br />
<br />
'''ORIENTATION_CORRECTION'''<br />
Double x,y – sometimes the indexing solutions from DIALS are corrected by a systematic error in the horizontal and vertical rotations due to a component of the experiment which is modelled in DIALS but not by cppxfel. If this systematic shift is known, it can be supplied in degrees (x in the horizontal axis and y in the vertical axis). These values may be deduced from a table of corrections produced at the end of an INTEGRATE command. Default 0,0.<br />
<br />
'''IMAGE_MASKED_VALUE'''<br />
Int x – if there is a pixel value used as a masking value in the images provided, this can be set using this command. If IMAGE_MASKED_VALUE is not set, the default is not to mask any pixel value.<br />
<br />
'''SPHERE_THICKNESS'''<br />
Double x - This is the thickness in Ã…<sup>-1</sup> around the mean Ewald radius to consider integrating. The subset of reflections which are chosen for integration are then determined by OVER_PRED_RLP_SIZE, so SPHERE_THICKNESS is a prior scanning step. Default 0.02 Ã…<sup>-1</sup>. Increasing this will improve spot prediction at low resolution but will reduce the overall speed of the program.<br />
<br />
'''PIXEL_COUNT_CUTOFF'''<br />
Int x - If, during integration, a pixel exceeds the value x, this reflection is rejected. Can be used to protect against non-linear gain of the detector at high pixel counts. Default is not to use.<br />
<br />
'''REFINE_UNIT_CELL_A'''<br />
Boolean x – if set to ON, this unit cell a dimension is refined in either integration or post-refinement, using the step size specified by STEP_UNIT_CELL_A. Default OFF.<br />
<br />
'''REFINE_UNIT_CELL_B'''<br />
Boolean x – if set to ON, this unit cell b dimension is refined in either initial orientation matrix refinement or post-refinement, using the step size specified by STEP_UNIT_CELL_B. Default OFF.<br />
<br />
'''REFINE_UNIT_CELL_C'''<br />
Boolean x – if set to ON, this unit cell c dimension is refined in either initial orientation matrix refinement or post-refinement, using the step size specified by STEP_UNIT_CELL_C. Default OFF.<br />
<br />
'''STEP_UNIT_CELL_A'''<br />
Double x – this determines the step size used during either initial orientation matrix refinement or post-refinement for unit cell A. Default 0.2 Å.<br />
<br />
'''STEP_UNIT_CELL_B'''<br />
Double x – this determines the step size used during either initial orientation matrix refinement or post-refinement for unit cell B. Default 0.2 Å.<br />
<br />
'''STEP_UNIT_CELL_C'''<br />
Double x – this determines the step size used during either initial orientation matrix refinement or post-refinement for unit cell C. Default 0.2 Å.<br />
<br />
'''FROM_DIALS'''<br />
Boolean x – should be set to ON if matrices were calculated by DIALS. Unlikely to be any other value. Default ON.<br />
<br />
'''DO_NOT_REJECT_REFLECTIONS'''<br />
Boolean x – usually, if there are multiple lattices on the same image then overlapping shoeboxes lead to reflections being rejected. This also applies to overlapping reflections from the same lattice, if the over-prediction is too wide. This may not always be a desired feature, so setting this value to OFF will disable it. Default ON.<br />
<br />
'''REFINE_IN_PLANE_OF_DETECTOR'''<br />
Boolean x – in some cases, the rotation of the orientation matrix around the beam axis is not exact. Setting this value to ON will refine the orientation matrix around this axis. Default OFF.<br />
<br />
'''FIT_BACKGROUND_AS_PLANE'''<br />
Boolean x – to fit the background pixels as a plane rather than simple summation, switch this to ON. Individual pixels may also be rejected as outliers. Default OFF.<br />
<br />
'''PANEL_LIST'''<br />
String x – list of panels generated by cppxfel.input_gen or modified by REFINE_METROLOGY command. Individual panels in this file are specified by the keyword PANEL followed by the upper left x,y and bottom right x,y integer coordinates separated by single spaces. Required for initial orientation matrix refinement but not post-refinement. No default.<br />
<br />
== Merging ==<br />
<br />
'''RECALCULATE_SIGMA'''<br />
Boolean x - To be used on a final merge of MTZ files: an estimate of the standard deviation will be made from the distribution of the observation intensities for each reflection. Default OFF.<br />
<br />
'''MERGE_ANOMALOUS'''<br />
Boolean x – to be used on a final merge of MTZ files, this flag will merge anomalous data sets rather than straight intensities. Anomalous data will be written to anomalous_diff.mtz. Default OFF.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_Initial_Orientation_Matrix_RefinementCppxfel Initial Orientation Matrix Refinement2015-11-25T23:12:44Z<p>Helenginn: </p>
<hr />
<div>Initial orientation matrix refinement covers the first stage of the dedicated C++ code in [[Cppxfel|''cppxfel'']] and uses an input file which is called <code>integrate.txt</code> by default. This should have been generated during the [[Cppxfel Indexing | previous stage]].<br />
<br />
== The contents of integrate.txt ==<br />
<br />
One can look at the contents of integrate.txt by executing:<br />
<br />
<pre><br />
cat integrate.txt<br />
</pre><br />
<br />
This shows an input script along the following lines. This input script is divided into a number of parameters, followed by the line <code>COMMANDS</code>, after which the task(s) for ''cppxfel'' to complete are listed. In this case, ''cppxfel'' should <code>INTEGRATE</code> the data.<br />
<br />
<pre><br />
ORIENTATION_MATRIX_LIST matrices.dat<br />
NEW_MATRIX_LIST orientations.dat<br />
<br />
SPACE_GROUP 197<br />
UNIT_CELL 105.917757854 105.917757854 105.917757854 90.0 90.0 90.0<br />
FIX_UNIT_CELL ON<br />
<br />
MM_PER_PIXEL 0.11<br />
BEAM_CENTRE 881.755 881.5075<br />
DETECTOR_DISTANCE 91.0018<br />
INTEGRATION_WAVELENGTH 1.45738121124 <br />
<br />
PANEL_LIST panels.txt<br />
METROLOGY_SEARCH_SIZE 1<br />
<br />
SHOEBOX_FOREGROUND_PADDING 1<br />
SHOEBOX_NEITHER_PADDING 2<br />
SHOEBOX_BACKGROUND_PADDING 3<br />
<br />
INTENSITY_THRESHOLD 12<br />
ABSOLUTE_INTENSITY OFF<br />
REFINE_ORIENTATIONS ON<br />
<br />
VERBOSITY_LEVEL 1<br />
<br />
COMMANDS<br />
<br />
INTEGRATE<br />
</pre><br />
<br />
The matrix information from indexing is passed through using the generated file <pre>matrices.dat</pre> under the keyword <code>ORIENTATION_MATRIX_LIST</code> which was produced in the previous stage. One can see the contents of <code>matrices.dat</code>:<br />
<br />
<pre><br />
head -21 matrices.dat<br />
</pre><br />
<br />
<pre><br />
image shot-s00-20130316165156302<br />
wavelength 1.45690748941<br />
distance 91.0018<br />
centre 881.755 881.5075<br />
unitcell 0.00945122403519 0.0 0.0 -5.78701450245e-19 0.00945122403519 0.0 -5.78701450245e-19 -1.51989185671e-18 0.00945122403519 <br />
rotation 0.209524357745 -0.976621953811 0.0480531252429 -0.360236467579 -0.122785692599 -0.924745024923 0.909026529282 0.176446119331 -0.377541436183 <br />
image shot-s01-20130316165035212<br />
wavelength 1.45496194705<br />
distance 91.0018<br />
centre 881.755 881.5075<br />
unitcell 0.00944171966033 0.0 0.0 7.80781215281e-18 0.00944171966033 0.0 3.61484632923e-18 2.67460240614e-18 0.00944171966033 <br />
rotation -0.351867835054 0.0121912864248 0.935970298241 -0.852741351745 -0.416546104403 -0.315153184867 0.386032658825 -0.909032846184 0.156965190022 <br />
image shot-s00-20130316165041011<br />
wavelength 1.45643207192<br />
distance 91.0018<br />
centre 881.755 881.5075<br />
unitcell 0.00943218298265 0.0 0.0 1.51682978341e-18 0.00943218298265 0.0 3.61119512738e-18 5.77535560549e-19 0.00943218298265 <br />
rotation -0.21213292526 0.320476950034 0.92319778299 -0.956128815937 -0.263369266933 -0.128274380022 0.202032941263 -0.909907222596 0.362286539789 <br />
image shot-s00-20130316165215513<br />
wavelength 1.45673961831<br />
distance 91.0018<br />
</pre><br />
<br />
The image tag has the name of the image file minus the extension. The values for the detector distance, beam centre and wavelength are overridden by the values in the <code>integrate.txt</code> file if they are present, but the overriding values can be removed if the values vary from one image to another. The orientation matrix is constructed from the unit cell matrix and the rotation matrix listed. The unit cell matrix is the inverse of the real space unit cell matrix, and the unit cell matrix is multiplied by the rotation matrix before use (rotation × unit cell matrix).<br />
<br />
== Executing integrate.txt ==<br />
<br />
This integration input file can be executed through ''cppxfel'' using the following command:<br />
<br />
<pre><br />
cppxfel.run -i integrate.txt<br />
</pre><br />
<br />
== Wavelength histograms ==<br />
<br />
The output tracks individual images which execute on different threads as their orientation matrices are refined. An example of a wavelength histogram created is shown below. These wavelength histograms were first described in Ginn et al, Nat Comms 2015.<br />
<br />
<pre><br />
Wavelength histogram for shot-s00-20130316165134721.img<br />
1.41322 .<br />
1.41759 <br />
1.42196 <br />
1.42633 .<br />
1.4307 .<br />
1.43508 ..<br />
1.43945 <br />
1.44382 ..<br />
1.44819 .....<br />
1.45256 .........<br />
1.45693 ..................<br />
1.4613 ......................................................................<br />
1.46567 ...............................................<br />
1.47004 .....<br />
1.47441 .......<br />
1.47878 ...<br />
1.48315 .<br />
1.48752 .<br />
1.4919 ..<br />
1.49627 <br />
<br />
</pre><br />
<br />
These integration peaks should be as long and narrow as possible, depending on the extent of over-prediction as determined by the parameter <code>OVER_PRED_BANDWIDTH</code>.<br />
<br />
== Final integration summary table ==<br />
<br />
The initial orientation matrix refinement commands finish with a summary table detailing the main results for each crystal and each image. It will start with something like this:<br />
<br />
<pre><br />
Filename Refl num Score Horiz rot Vert rot Beam rot Stdev Wavelength Distance UnitCellA UnitCellB UnitCellC<br />
shot-s00-20130316165156302.img 111 8.91892 0 0 0 0.0260573 1.45691 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165043053.img 80 6.33333 -1 1 0 0.0240737 1.45647 91.002 105.918 105.918 105.918<br />
shot-s01-20130316165128672.img 14 4.28571 -1.11914 -1.11523 0 0.0170082 1.45962 91.0018 105.918 105.918 105.918<br />
shot-s01-20130316165047878.img 87 1.2069 -0.625 0.320312 0 0.0105548 1.45852 91.0018 105.918 105.918 105.918<br />
shot-s01-20130316165035212.img 21 9.47368 -1.06836 0.908203 0 0.0209798 1.45496 91.0018 105.918 105.918 105.918<br />
shot-s01-20130316165033837.img 37 6.77419 -0.00390625 0.533203 0 0.0239777 1.45621 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165336174.img 188 1.45161 -0.365234 0.371094 0 0.0104029 1.45705 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165131305.img 171 1.06509 0.212891 0.621094 0 0.0107561 1.4579 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165041011.img 26 6.11111 -0.445312 -0.298828 0 0.0223847 1.45643 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165126972.img 211 0.972222 -0.00585938 0.609375 0 0.00877267 1.45685 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165126972.img 47 7.875 0.183594 1.97266 0 0.0240756 1.45685 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165138553.img 41 9.28571 -0.365234 0.625 0 0.0253017 1.45673 91.0018 105.918 105.918 105.918<br />
shot-s01-20130316165050712.img 19 6.75 0.0214844 2.17578 0 0.0230937 1.45894 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165215513.img 146 2.02128 -0.498047 0.240234 0 0.010419 1.45674 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165133179.img 143 2.36301 -0.376953 0.283203 0 0.0129344 1.45528 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165143844.img 142 1.99301 -0.132812 0.554688 0 0.0117068 1.45789 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165231472.img 6 12.5 1 1.875 0 0.013229 1.45868 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165052512.img 67 8.23944 -0.365234 0.482422 0 0.0251099 1.45578 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165052512.img 168 1.86335 -0.458984 0.542969 0 0.01226 1.45578 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165125972.img 200 1.6129 -0.388672 0.638672 0 0.010696 1.456 91.0018 105.918 105.918 105.918<br />
shot-s00-20130316165052178.img 217 1.70616 -0.1875 0.6875 0 0.0116042 1.45657 91.0018 105.918 105.918 105.918<br />
</pre><br />
<br />
The "score" column is the result of the minimisation function used to refine the horizontal and vertical rotations, which are listed in the following two columns. This is a weighted combination of the total number of reflections and the standard deviation of the Ewald sphere wavelengths. In general, a score of 3 or less is good, though will be higher for low resolution data, using the standard setting for <code>ORIENTATION_SCORE</code>. The horizontal and vertical rotations show the rotations in degrees required to find the optimal solution.<br />
<br />
== Output files ==<br />
<br />
The important files created by running <code>integrate.txt</code> are:<br />
* A new set of orientation matrices which have been refined under three file names:<br />
* * <code>integrate-orientations.dat</code> - this file is to be provided to input files which use the <code>INTEGRATE</code> command, such as <code>integrate.txt</code> generated by <code>cppxfel.input_gen</code><br />
* * <code>refine-orientations.dat</code> - this file is to be provided to input files which use the <code>REFINE_PARTIALITY</code> command, such as <code>refine.txt</code> generated by <code>cppxfel.input_gen</code><br />
* * <code>merge-orientations.dat</code> - this file is to be provided to input files which use the <code>MERGE</code> command, such as <code>merge.txt</code> generated by <code>cppxfel.input_gen</code>.<br />
* MTZ files following the format <code>img*.mtz</code>, which contain over-predicted reflection intensities. The first crystal will end in <code>_0.mtz</code>, the second with <code>_1.mtz</code> and so on for the number of lattices indexed in one image.<br />
* <code>shot*.dat</code> files, containing tab-delimited information on the coordinates and intensities of each reflection.<br />
* <code>integration.csv</code>, which contains the information in the final integration summary table which can be imported into other graph-drawing software suites.<br />
<br />
Once these files are generated, it is time to do [[Cppxfel Post-refinement]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T23:06:22Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.<br />
<br />
== Next steps ==<br />
<br />
Try running the tutorial on a set of 1000 images from CPV, starting with [[Cppxfel Indexing|indexing]].</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T23:05:13Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==<br />
<br />
The ''cppxfel'' distribution requires the CCP4 symmetry libraries. This needs to be pointed to by the SYMINFO environment variable. To set this:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export SYMINFO="/absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib"</code><br />
| <code>setenv SYMINFO /absolute/path/to/cctbx/modules/ccp4io/libccp4/data/syminfo.lib</code><br />
|}<br />
<br />
This will prevent run-time errors from the CCP4 library functions not being able to find its symmetry information library.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T22:22:30Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html the DIALS website]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T22:21:48Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS from [[http://dials.github.io/installation.html]]. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T22:06:03Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed. A pointer to the CCP4 symmetry libraries is required.<br />
<br />
== CCP4 Symmetry libraries ==</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T22:01:25Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel<br />
cd ../build<br />
make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T22:00:49Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
== Installing the boost libraries ==<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
=== With user privileges ===<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
=== Without user privileges ===<br />
<br />
<br />
<br />
== DIALS distribution ==<br />
<br />
Download the DIALS distribution for your flavour of OS. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
== ''cppxfel'' distribution ==<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel</pre><br />
<pre>cd ../build</pre><br />
<pre>make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.</div>Helenginnhttp://viper.lbl.gov:8080/cctbx.xfel/index.php/Cppxfel_InstallationCppxfel Installation2015-11-25T22:00:10Z<p>Helenginn: </p>
<hr />
<div>This details the installation instructions for [[Cppxfel|''cppxfel'']] on Linux and Mac OS X. '''These installation instructions are not finished and will not currently result in a working version of ''cppxfel''.'''<br />
<br />
<br />
<br />
= Installing the boost libraries =<br />
<br />
The boost library is a dependency of ''cppxfel'' allowing it to run on multiple threads from within C++. Either download source and compile from boost website, or if you have user privileges, you can use a package manager.<br />
<br />
== With user privileges ==<br />
<br />
On CentOS:<br />
<br />
<pre>yum install boost boost-devel</pre><br />
<br />
On Ubuntu:<br />
<br />
<pre>apt-get install libboost-all-dev</pre><br />
<br />
On Mac OS X, install brew, and then:<br />
<br />
<pre>brew install boost</pre><br />
<br />
{| class="wikitable"<br />
|-<br />
! CentOS<br />
! Ubuntu<br />
! Mac OS X<br />
|-<br />
| <code>yum install boost boost-devel</code><br />
| <code>apt-get install libboost-all-dev</code><br />
| Install brew, then: <code>brew install boost</code><br />
|}<br />
<br />
We need to make sure the boost library is visible to the DIALS/cppxfel installation. To see where the boost library was installed:<br />
<br />
<pre>locate libboost_thread-mt.so</pre><br />
<br />
This will show a directory e.g.<br />
<br />
<pre>/usr/lib64/libboost_thread-mt.so</pre><br />
<br />
This should be set to the environment variable <code>BOOST_LOCATION</code>.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>export BOOST_LOCATION="/usr/lib64/"</code><br />
| <code>setenv BOOST_LOCATION /usr/lib64</code><br />
|}<br />
<br />
<pre></pre><br />
<br />
== Without user privileges ==<br />
<br />
<br />
<br />
= DIALS distribution =<br />
<br />
Download the DIALS distribution for your flavour of OS. Extract the dials-installer-dev to a new location. Navigate into this directory.<br />
<br />
<pre>cd dials-installer-dev</pre><br />
<br />
Start the install command to a new directory to which you have write access.<br />
<br />
<pre>./install --prefix=/absolute/path/to/dials</pre><br />
<br />
= ''cppxfel'' distribution =<br />
<br />
The ''cppxfel'' source code must be downloaded into the <code>modules</code> directory.<br />
<br />
<pre>cd /absolute/path/to/dials/modules</pre><br />
<br />
The repository can be cloned from the GitHub link for cppxfel.<br />
<br />
<pre>git clone https://github.com/cppxfel/cppxfel.git</pre><br />
<br />
The DIALS distribution must be configured to compile the cppxfel code, and then the distribution must be compiled.<br />
<br />
<pre>libtbx.configure cppxfel</pre><br />
<pre>cd ../build</pre><br />
<pre>make</pre><br />
<br />
Set the paths to the DIALS and cppxfel command line tools (bash):<br />
<br />
. /absolute/path/to/dials/build/setpaths.sh<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>source /absolute/path/to/dials/build/setpaths.csh</code><br />
|}<br />
<br />
This needs to be repeated every time a new shell is started. Alternatively, place this in your appropriate file (bash)<br />
<br />
{| class="wikitable"<br />
|-<br />
! Bash shell<br />
! Csh shell<br />
|-<br />
| <code>test -r /absolute/path/to/dials/build/setpaths.sh && \<br />
. /absolute/path/to/dials/build/setpaths.sh</code><br />
| <code>test -r /absolute/path/to/dials/build/setpaths.csh && \<br />
source /absolute/path/to/dials/build/setpaths.csh</code><br />
|<br />
|-<br />
| to <code>~/.bashrc</code><br />
| to <code>~/.cshrc</code><br />
|}<br />
<br />
Confirm that ''cppxfel'' runs:<br />
<br />
<code>cppxfel.run</code><br />
<br />
The output should look something like this:<br />
<br />
<pre><br />
Welcome to cppxfel version 1.0!<br />
Please refer to & cite paper in Journal of Applied Crystallography (unpublished)<br />
<br />
Command order for regular structure solution:<br />
cppxfel.run_dials shot*.pickle<br />
cppxfel.input_gen<br />
cppxfel.run -i integrate.txt<br />
cppxfel.run -i refine.txt<br />
cppxfel.run -i merge.txt<br />
<br />
Other functions for assessing data quality:<br />
<br />
Correlation between two MTZ files:<br />
<br />
cppxfel.run -cc firstFile.mtz secondFile.mtz [ambiguity] [lowRes] [highRes] [bins]<br />
<br />
ambiguity: 0, 1, 2 or 3 - will compare different indexing solutions where the Bravais lattice symmetry is higher than that of the point group for certain space groups. Default 0<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
<br />
Partiality CSV files:<br />
<br />
cppxfel.run -partiality reference.mtz ref-img-shot-number.mtz [highRes]<br />
<br />
highRes: 0, 1, 2 or 3 - highest resolution reflection to report results on. Default 1.4<br />
This outputs partiality_[m].csv where m is bin number, which can be imported into other graphing softwares such as R.<br />
<br />
Merging statistics:<br />
<br />
cppxfel.run -rpim unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmeas unmerged_file.mtz [lowRes] [highRes] [bins]<br />
cppxfel.run -rmerge unmerged_file.mtz [lowRes] [highRes] [bins]<br />
<br />
lowRes and highRes: set to resolution in Angstroms to bound the results, or set to 0 to take lowest/highest resolution data. Default 0, 0<br />
bins: number of bins to report correlation statistics. Default 20.<br />
</pre><br />
<br />
If you see this, ''cppxfel'' has been installed.</div>Helenginn