cctbx.xfel GUI/Workflows

Step-by-step recipes for the common things done with the cctbx.xfel GUI. Each step names the tab or dialog it happens in; see the tab pages for the details of every control.

Back to cctbx.xfel GUI.

Setting up a new experiment

1. Start cctbx.xfel. In the login dialog press DB Credentials... and enter the database server, database, user and password. If you have no server, fill those in anyway and press Start DB Server (see starting a local server).
2. Enter a new Experiment Tag for this experiment, choose the Facility and, at LCLS, the Experiment name; in standalone mode press Options... and set the folder to monitor and the file template.
3. Choose the Output folder (on a file system the compute nodes can write to).
4. Press Advanced Settings... and set the multiprocessing method, queue, number of processors and environment script for your cluster. Press OK.
5. Press Save Project As... and name the project, then OK. The main window opens.
6. On the toolbar press Watch for new runs. Runs appear in the Runs tab.

Processing runs as they are collected

1. Runs tab: press Manage Tags and create tags for your samples; press Manage Persistent Tags and tick the tag for the sample currently on the beam, so that every new run gets it. Change the persistent tags when the sample changes.
2. Trials tab: press New Trial. Enter the unit cell and space group if known, adjust spot finding if needed, and press OK. In the new trial press New Block: set the start run, leave Auto add runs selected, enter the detector address, beam centre and distance (LCLS), any mask or reference geometry, and press OK. Tick Active Trial.
3. On the toolbar press Auto-submit jobs. The job sentinel now submits a job for every run in the block and for each new run as it arrives.
4. Jobs tab: confirm the jobs reach RUN and then DONE. If one shows EXIT or S_FAIL, read its log (see Troubleshooting).
5. Run Stats tab: select the trial and press Auto plot last five runs. Leave this tab showing during collection.
6. Unit Cell tab: select the trial, add a tag set for the sample, and check that the cell histograms are single peaks at the expected values.

Tuning spot finding and indexing

1. Make a new trial (it copies the previous one) and change one thing: for example lower Sigma strong or set Min spot size. Use Copy runblocks from the previous trial so both trials cover the same runs, or give the new trial a block with an explicit end run covering a few representative runs.
2. Activate the new trial. Both trials can be active at once; jobs for the new trial appear in the Jobs tab.
3. In the Run Stats tab switch between the two trials and compare the strong spot counts, indexing rate and resolution for the same runs.
4. Deactivate the trial you will not use, so it stops consuming queue time on new runs.

Reprocessing after a change to a run block

Detector settings live in run blocks, and results are stored per block. To reprocess runs with a new geometry or mask:

1. Trials tab: click the block, change the setting (for example Reference geometry) and press OK. Because a setting other than the end run or comment changed, the old block is deactivated and a new block is created in the trial.
2. With the trial active and Auto-submit jobs on, jobs for the new block are submitted. The old block's results remain on disk under the old block id.

To instead re-run a single failed job unchanged, use Restart job in the Jobs tab.

Merging a dataset

1. Make sure the runs to merge carry a tag that identifies them (Runs tab) and that their indexing jobs are DONE.
2. Datasets tab: press New Dataset. Name it, choose the trial and the tag(s). Choose a reference model, or Unknown structure with the cell and space group. Keep Include ensemble refinement ticked unless time is short. Accept or change the cosym suggestion. Set the resolution limit. Finish.
3. Tick Active Dataset. With Auto-submit jobs on, ensemble refinement and scaling jobs are submitted per run, then a merging job once some runs are through. Follow them in the Jobs tab.
4. Merging stats tab: select the dataset. Version All shows CC1/2 and multiplicity growing as versions accumulate; a single version shows its statistics by resolution.
5. The merged MTZ is <output folder>/<dataset name>/v<NNN>/<name>_v<NNN>_all.mtz.

Merging without a reference structure

Choose Unknown structure in the wizard. The merge is a plain average without per-image scaling. Once it has produced an MTZ, create a second dataset with Known reference model pointing at that MTZ to get a properly scaled and post-refined merge. If the symmetry has an indexing ambiguity, tick cosym in both datasets; the first merge fixes an arbitrary but consistent indexing frame which the second then anchors to.

Merging one crystal form out of a mixture

1. Unit Cell tab: select the trial and the tag set, tick Plot clusters, and adjust Cluster epsilon until the forms separate. Note which component (0 is the largest) is the one you want. The cluster file is written to cluster/ in the output folder.
2. Datasets tab: open the dataset, and in the Scaling stage tick Filter by unit-cell cluster, choose the cluster file, enter the component number and, if needed, a tighter Mahalanobis cutoff. Press OK.
3. The next version merged (restart the scaling jobs from the Jobs tab, or wait for new runs) uses only lattices from that cluster.

Merging the same runs with different settings

Press the copy button on the dataset. The copy is inactive and shares the original's tasks. Open it, change what you want (resolution limit, reference model, a stage's parameters); when asked, choose Detach so the original is unaffected. Give the copy a comment saying what differs, then activate it.

Stopping and restarting jobs

• To stop submitting new jobs, turn off Auto-submit jobs or untick the trial or dataset's Active box.
• To stop a running job, select it in the Jobs tab and press Stop job.
• To re-run a job after fixing the cause of a failure, select it, press Stop job if it is still running, then Restart job. Its old results are deleted and the job sentinel resubmits it.
• To throw away a trial's results, delete its jobs in the Jobs tab (untick Only display jobs from active trials/blocks to see jobs of inactive trials), and leave the trial inactive. The trial itself cannot be deleted.

Watching an experiment from elsewhere

Start a second GUI with the same project (copy ~/.cctbx.xfel/settings_<name>.phil to the other machine, or set the same credentials by hand) and monitoring_mode=True:

cctbx.xfel monitoring_mode=True

The Run Stats, Unit Cell and Merging stats tabs work as usual; nothing can be submitted from it. Keep job submission to a single GUI instance.

Standalone mode with files from another source

1. Settings: facility Standalone; Options...: set the folder, Monitor for files or folders, the file template (for example *.h5) and tick Files are composite for HDF5 or NeXus files. For one-image-per-file data, use folders with one folder per run and untick composite.
2. Proceed as for a live experiment. Run blocks select runs by their order of discovery; the detector geometry comes from the image headers unless a reference geometry is set in the block.