cctbx.xfel GUI/Settings

The Settings dialog is the first thing the GUI shows when it starts (titled CCTBX.XFEL Login) and is also opened by the Settings toolbar button. It collects everything the GUI needs to connect to an experiment: the database, the experiment tag, the facility, the output folder, and how jobs are submitted. All of it is saved as a project.

Back to cctbx.xfel GUI.

The Settings dialog

Field	Meaning
Experiment Tag	Prefix for every database table belonging to this experiment. Use a new tag for a new experiment, or for a clean restart of the same experiment. Locked while the GUI is connected.
DB Credentials...	Opens the Database credentials dialog. Locked while the GUI is connected.
Facility	LCLS uses the LCLS run database and XTC streams. Standalone monitors a folder for image files from any other source. Options... opens the facility-specific settings below.
Experiment	LCLS experiment name, for example cxid9114. Disabled for standalone.
Output	Folder that will receive all processing results. Browse... picks it with a folder chooser. See Where the results go.
Load Project...	Choose a saved project from a list (sorted by last modified). Loading a project fills in every field. While the GUI is connected, a project whose database connection or experiment tag differs from the current one is refused, since applying it would require a restart.
Save Project As...	Save the current settings under a name. The file is ~/.cctbx.xfel/settings_<name>.phil. The project most recently saved or loaded becomes the default for the next start.
Advanced Settings...	Opens the Advanced settings dialog: multiprocessing, queue and processing back end.
OK	At start-up: connects to the database, creates the tables if they do not exist, and opens the main window. If no project has been named yet, you are asked to name one first. From the running GUI: applies the changes and saves them to the current project.

Database credentials

Opened with DB Credentials.... The GUI needs a MySQL server it can create tables on.

Field	Meaning
DB Host name	Server host. The default is the LCLS user database host, psdb-user.slac.stanford.edu.
DB Port number	Usually 3306.
DB name	The database on that server.
DB user name / DB Password	Account with rights to create and modify tables in that database. The password is cached in plain text in the project file.
Delete and regenerate all tables	Drops every table with the current experiment tag and recreates them empty when you press OK in the Settings dialog. This destroys all records of the experiment (not the files on disk). You are asked to confirm.
XTC stream location	LCLS only. Where the XTC files are: SLAC, SRCF_FFB (fast feedback, active experiment only), SDF or NERSC.
Start DB Server	Launches your own MySQL server if you do not have one. See below.

Starting a local database server

Start DB Server submits a job (cctbx.xfel.ui_server) that runs a MySQL server using the queuing settings from Advanced settings. It asks for:

• DB Base Directory: where the server keeps its data files. Defaults to MySql inside the output folder. If the directory already contains a database, the server reuses it.
• DB Root Password: only needed when the base directory is new, to initialise the server. The database, user and password typed in the credentials dialog are created at that point, so fill those in first.

With the local multiprocessing method the server runs on this machine and the host is set to 127.0.0.1. With Slurm or Shifter the job runs on a compute node and the GUI fills in that node's host name once the job starts; the server lives as long as the job's wall time. The GUI waits for the server to accept connections before continuing and reports the server's error log if it fails. Make sure a server is not already running on the same base directory.

Facility options

LCLS options

Option	Meaning
Use ffb (fast feedback) file system	Read XTC data from the fast feedback system. Only for the active experiment on the priority queues.
Dump all images to disk	Write every image as CBF to the job's all/ folder, whether or not it indexed. Needed for the Strong images that didn't index feature of the Run Stats tab and useful when tuning spot finding. Uses a lot of disk space.
Require stream 80 / 81 (FEE spectrometer) before processing	Do not process a run until the FEE spectrometer stream is present, so that per-image wavelengths are available.

Standalone options

In standalone mode the run sentinel watches a folder for new data and turns each file or folder it finds into a run.

Option	Meaning
Folder to monitor	The directory new data appear in.
Monitor for	files: every file in the folder matching the template is a run. folders: every sub-folder is a run (or, with composite files, each matching file in each sub-folder is a run).
Run complete criteria	For folders. Status file: the folder is complete when it contains a Cheetah-style status.txt whose Status is Finished. Number of files: complete once it holds at least Number of files per run matching files.
Minimum time since last modified / Minimum file size	For files. A file is only picked up once it has been untouched for this many seconds and is at least this many bytes, so that files still being written are not processed.
File matching template	Glob pattern for data files, for example *.h5 or *.cbf.
Files are composite	Tick for formats where one file holds many images (HDF5, NeXus): each file becomes its own run. Untick for one-image-per-file formats, where the whole folder becomes a single run.

Run names in standalone mode are derived from the file or folder names (for example run_0012_data for a composite file data.h5 in folder run_0012). Run blocks in standalone mode therefore select runs by database id, in order of discovery, rather than by run number.

Advanced settings

Opened with Advanced Settings.... The controls shown depend on the chosen multiprocessing method.

Multiprocessing options

Option	Meaning
Multiprocessing	How jobs are run: local (on this machine, one job at a time), lsf, slurm, shifter (Slurm with a Shifter container, for NERSC), sge, pbs, htcondor or custom (a user-supplied submission template, set in the settings file).
Queue	Queue or partition to submit to. A drop-down of the LCLS queues for LSF at LCLS, a free text field otherwise.
Total number of processors	Processes per job (MPI ranks). With the LCLS queues this snaps to the core count of a node.
Total number of nodes / Max Walltime	Shifter only: nodes per job and the time limit in minutes.
Number of processors per node	Auto lets the queuing system decide; otherwise an explicit count. Together with the total number of processors this determines the node count.
MPI command	The command that launches MPI programs, for example mpirun or srun, with any extra arguments.
Environment setup script	A script that is sourced at the start of each job to set up the cctbx environment, when the compute nodes do not inherit yours.
Phenix setup script	Likewise for Phenix, used by Phenix tasks in datasets.
Nodes per job	Slurm, PBS and Shifter: a separate node count for Indexing, TDER (time-dependent ensemble refinement), Scaling and Merging jobs, since merging usually needs more nodes than indexing a single run.
Extra submission arguments	One per line; appended to the submission command (sbatch, bsub, ...). Use this for account names, memory requests and similar.
Shifter settings	Shifter image name, the sbatch and srun script templates, job name, NERSC project (-A), reservation, constraint, and whether to stage logs to the DataWarp burst buffer.
HTCondor settings	Path to the MPI executable script (openmpiscript or mp2script) and the shared file system domain.

One further parameter has no control: mp.max_queued in the settings file limits how many jobs may be running or queued at once; the job sentinel waits when the limit is reached.

Data analysis options

Processing back end chooses the program each indexing job runs:

• cctbx.xfel (cctbx.xfel.process): DIALS spot finding, indexing, refinement and integration with stills-specific defaults. The usual choice.
• Small cell (cctbx.xfel.small_cell_process): small-cell indexing (Brewster 2015) followed by DIALS refinement and integration.
• custom: any other program name. Trial parameters are validated against the chosen program's PHIL scope.

Changing the back end after trials exist is not recommended, since the trials' parameters were written for the previous program.