Aaron: Created page with "{{DISPLAYTITLE:cctbx.xfel GUI/Workflows}} Step-by-step recipes for the common things done with the cctbx.xfel GUI. Each step names the tab or dialog it happens in; see the tab pages for the details of every control. Back to cctbx.xfel GUI. == Setting up a new experiment == # Start `cctbx.xfel`. In the login dialog press '''DB Credentials...''' and enter the database server, database, user and password. If you have no server, fill those in anyway and pres..."

2026-10-09T22:32:28Z

Created page with "{{DISPLAYTITLE:cctbx.xfel GUI/Workflows}} Step-by-step recipes for the common things done with the cctbx.xfel GUI. Each step names the tab or dialog it happens in; see the tab pages for the details of every control. Back to cctbx.xfel GUI. == Setting up a new experiment == # Start <code>cctbx.xfel</code>. In the login dialog press '''DB Credentials...''' and enter the database server, database, user and password. If you have no server, fill those in anyway and pres..."

New page

{{DISPLAYTITLE:cctbx.xfel GUI/Workflows}}
Step-by-step recipes for the common things done with the cctbx.xfel GUI. Each step names the tab or dialog it happens in; see the tab pages for the details of every control.

Back to [[cctbx.xfel GUI]].

== Setting up a new experiment ==
# Start <code>cctbx.xfel</code>. In the login dialog press '''DB Credentials...''' and enter the database server, database, user and password. If you have no server, fill those in anyway and press '''Start DB Server''' (see [[cctbx.xfel GUI/Settings#Starting a local database server|starting a local server]]).
# Enter a new '''Experiment Tag''' for this experiment, choose the '''Facility''' and, at LCLS, the '''Experiment''' name; in standalone mode press '''Options...''' and set the folder to monitor and the file template.
# Choose the '''Output''' folder (on a file system the compute nodes can write to).
# Press '''Advanced Settings...''' and set the multiprocessing method, queue, number of processors and environment script for your cluster. Press OK.
# Press '''Save Project As...''' and name the project, then '''OK'''. The main window opens.
# On the toolbar press '''Watch for new runs'''. Runs appear in the Runs tab.

== Processing runs as they are collected ==
# '''Runs tab''': press '''Manage Tags''' and create tags for your samples; press '''Manage Persistent Tags''' and tick the tag for the sample currently on the beam, so that every new run gets it. Change the persistent tags when the sample changes.
# '''Trials tab''': press '''New Trial'''. Enter the unit cell and space group if known, adjust spot finding if needed, and press OK. In the new trial press '''New Block''': set the start run, leave '''Auto add runs''' selected, enter the detector address, beam centre and distance (LCLS), any mask or reference geometry, and press OK. Tick '''Active Trial'''.
# On the toolbar press '''Auto-submit jobs'''. The job sentinel now submits a job for every run in the block and for each new run as it arrives.
# '''Jobs tab''': confirm the jobs reach <code>RUN</code> and then <code>DONE</code>. If one shows <code>EXIT</code> or <code>S_FAIL</code>, read its log (see [[cctbx.xfel GUI/Troubleshooting|Troubleshooting]]).
# '''Run Stats tab''': select the trial and press '''Auto plot last five runs'''. Leave this tab showing during collection.
# '''Unit Cell tab''': select the trial, add a tag set for the sample, and check that the cell histograms are single peaks at the expected values.

== Tuning spot finding and indexing ==
# Make a new trial (it copies the previous one) and change one thing: for example lower ''Sigma strong'' or set ''Min spot size''. Use '''Copy runblocks from''' the previous trial so both trials cover the same runs, or give the new trial a block with an explicit end run covering a few representative runs.
# Activate the new trial. Both trials can be active at once; jobs for the new trial appear in the Jobs tab.
# In the Run Stats tab switch between the two trials and compare the strong spot counts, indexing rate and resolution for the same runs.
# Deactivate the trial you will not use, so it stops consuming queue time on new runs.

== Reprocessing after a change to a run block ==
Detector settings live in run blocks, and results are stored per block. To reprocess runs with a new geometry or mask:
# Trials tab: click the block, change the setting (for example ''Reference geometry'') and press OK. Because a setting other than the end run or comment changed, the old block is deactivated and a new block is created in the trial.
# With the trial active and ''Auto-submit jobs'' on, jobs for the new block are submitted. The old block's results remain on disk under the old block id.
To instead re-run a single failed job unchanged, use '''Restart job''' in the Jobs tab.

== Merging a dataset ==
# Make sure the runs to merge carry a tag that identifies them (Runs tab) and that their indexing jobs are <code>DONE</code>.
# Datasets tab: press '''New Dataset'''. Name it, choose the trial and the tag(s). Choose a reference model, or ''Unknown structure'' with the cell and space group. Keep ''Include ensemble refinement'' ticked unless time is short. Accept or change the cosym suggestion. Set the resolution limit. Finish.
# Tick '''Active Dataset'''. With ''Auto-submit jobs'' on, ensemble refinement and scaling jobs are submitted per run, then a merging job once some runs are through. Follow them in the Jobs tab.
# Merging stats tab: select the dataset. Version ''All'' shows CC1/2 and multiplicity growing as versions accumulate; a single version shows its statistics by resolution.
# The merged MTZ is <code><output folder>/<dataset name>/v<NNN>/<name>_v<NNN>_all.mtz</code>.

=== Merging without a reference structure ===
Choose ''Unknown structure'' in the wizard. The merge is a plain average without per-image scaling. Once it has produced an MTZ, create a second dataset with ''Known reference model'' pointing at that MTZ to get a properly scaled and post-refined merge. If the symmetry has an indexing ambiguity, tick cosym in both datasets; the first merge fixes an arbitrary but consistent indexing frame which the second then anchors to.

=== Merging one crystal form out of a mixture ===
# Unit Cell tab: select the trial and the tag set, tick '''Plot clusters''', and adjust ''Cluster epsilon'' until the forms separate. Note which component (0 is the largest) is the one you want. The cluster file is written to <code>cluster/</code> in the output folder.
# Datasets tab: open the dataset, and in the '''Scaling''' stage tick '''Filter by unit-cell cluster''', choose the cluster file, enter the component number and, if needed, a tighter Mahalanobis cutoff. Press OK.
# The next version merged (restart the scaling jobs from the Jobs tab, or wait for new runs) uses only lattices from that cluster.

== Merging the same runs with different settings ==
Press the copy button on the dataset. The copy is inactive and shares the original's tasks. Open it, change what you want (resolution limit, reference model, a stage's parameters); when asked, choose '''Detach''' so the original is unaffected. Give the copy a comment saying what differs, then activate it.

== Stopping and restarting jobs ==
* To stop submitting new jobs, turn off '''Auto-submit jobs''' or untick the trial or dataset's ''Active'' box.
* To stop a running job, select it in the Jobs tab and press '''Stop job'''.
* To re-run a job after fixing the cause of a failure, select it, press '''Stop job''' if it is still running, then '''Restart job'''. Its old results are deleted and the job sentinel resubmits it.
* To throw away a trial's results, delete its jobs in the Jobs tab (untick ''Only display jobs from active trials/blocks'' to see jobs of inactive trials), and leave the trial inactive. The trial itself cannot be deleted.

== Watching an experiment from elsewhere ==
Start a second GUI with the same project (copy <code>~/.cctbx.xfel/settings_<name>.phil</code> to the other machine, or set the same credentials by hand) and <code>monitoring_mode=True</code>:
<pre>cctbx.xfel monitoring_mode=True</pre>
The Run Stats, Unit Cell and Merging stats tabs work as usual; nothing can be submitted from it. Keep job submission to a single GUI instance.

== Standalone mode with files from another source ==
# Settings: facility ''Standalone''; '''Options...''': set the folder, '''Monitor for''' ''files'' or ''folders'', the file template (for example <code>*.h5</code>) and tick ''Files are composite'' for HDF5 or NeXus files. For one-image-per-file data, use ''folders'' with one folder per run and untick ''composite''.
# Proceed as for a live experiment. Run blocks select runs by their order of discovery; the detector geometry comes from the image headers unless a reference geometry is set in the block.

Cctbx.xfel GUI/Workflows - Revision history