Aaron: Created page with "{{DISPLAYTITLE:cctbx.xfel GUI/Settings}} The '''Settings dialog''' is the first thing the GUI shows when it starts (titled ''CCTBX.XFEL Login'') and is also opened by the '''Settings''' toolbar button. It collects everything the GUI needs to connect to an experiment: the database, the experiment tag, the facility, the output folder, and how jobs are submitted. All of it is saved as a project. Back to cctbx.xfel GUI. == T..."

2026-10-09T20:58:36Z

Created page with "{{DISPLAYTITLE:cctbx.xfel GUI/Settings}} The '''Settings dialog''' is the first thing the GUI shows when it starts (titled ''CCTBX.XFEL Login'') and is also opened by the '''Settings''' toolbar button. It collects everything the GUI needs to connect to an experiment: the database, the experiment tag, the facility, the output folder, and how jobs are submitted. All of it is saved as a project. Back to cctbx.xfel GUI. == T..."

New page

{{DISPLAYTITLE:cctbx.xfel GUI/Settings}}
The '''Settings dialog''' is the first thing the GUI shows when it starts (titled ''CCTBX.XFEL Login'') and is also opened by the '''Settings''' toolbar button. It collects everything the GUI needs to connect to an experiment: the database, the experiment tag, the facility, the output folder, and how jobs are submitted. All of it is saved as a [[cctbx.xfel GUI#Projects (saved settings)|project]].

Back to [[cctbx.xfel GUI]].

== The Settings dialog ==
{| class="wikitable"
! Field !! Meaning
|-
| '''Experiment Tag''' || Prefix for every database table belonging to this experiment. Use a new tag for a new experiment, or for a clean restart of the same experiment. Locked while the GUI is connected.
|-
| '''DB Credentials...''' || Opens the [[#Database credentials|Database credentials]] dialog. Locked while the GUI is connected.
|-
| '''Facility''' || ''LCLS'' uses the LCLS run database and XTC streams. ''Standalone'' monitors a folder for image files from any other source. '''Options...''' opens the facility-specific settings below.
|-
| '''Experiment''' || LCLS experiment name, for example <code>cxid9114</code>. Disabled for standalone.
|-
| '''Output''' || Folder that will receive all processing results. '''Browse...''' picks it with a folder chooser. See [[cctbx.xfel GUI#Where the results go|Where the results go]].
|-
| '''Load Project...''' || Choose a saved project from a list (sorted by last modified). Loading a project fills in every field. While the GUI is connected, a project whose database connection or experiment tag differs from the current one is refused, since applying it would require a restart.
|-
| '''Save Project As...''' || Save the current settings under a name. The file is <code>~/.cctbx.xfel/settings_<name>.phil</code>. The project most recently saved or loaded becomes the default for the next start.
|-
| '''Advanced Settings...''' || Opens the [[#Advanced settings|Advanced settings]] dialog: multiprocessing, queue and processing back end.
|-
| '''OK''' || At start-up: connects to the database, creates the tables if they do not exist, and opens the main window. If no project has been named yet, you are asked to name one first. From the running GUI: applies the changes and saves them to the current project.
|}

== Database credentials ==
Opened with '''DB Credentials...'''. The GUI needs a MySQL server it can create tables on.

{| class="wikitable"
! Field !! Meaning
|-
| '''DB Host name''' || Server host. The default is the LCLS user database host, <code>psdb-user.slac.stanford.edu</code>.
|-
| '''DB Port number''' || Usually 3306.
|-
| '''DB name''' || The database on that server.
|-
| '''DB user name''' / '''DB Password''' || Account with rights to create and modify tables in that database. The password is cached in plain text in the project file.
|-
| '''Delete and regenerate all tables''' || Drops every table with the current experiment tag and recreates them empty when you press OK in the Settings dialog. This destroys all records of the experiment (not the files on disk). You are asked to confirm.
|-
| '''XTC stream location''' || LCLS only. Where the XTC files are: <code>SLAC</code>, <code>SRCF_FFB</code> (fast feedback, active experiment only), <code>SDF</code> or <code>NERSC</code>.
|-
| '''Start DB Server''' || Launches your own MySQL server if you do not have one. See below.
|}

=== Starting a local database server ===
'''Start DB Server''' submits a job (<code>cctbx.xfel.ui_server</code>) that runs a MySQL server using the queuing settings from Advanced settings. It asks for:

* '''DB Base Directory''': where the server keeps its data files. Defaults to <code>MySql</code> inside the output folder. If the directory already contains a database, the server reuses it.
* '''DB Root Password''': only needed when the base directory is new, to initialise the server. The database, user and password typed in the credentials dialog are created at that point, so fill those in first.

With the ''local'' multiprocessing method the server runs on this machine and the host is set to <code>127.0.0.1</code>. With Slurm or Shifter the job runs on a compute node and the GUI fills in that node's host name once the job starts; the server lives as long as the job's wall time. The GUI waits for the server to accept connections before continuing and reports the server's error log if it fails. Make sure a server is not already running on the same base directory.

== Facility options ==

=== LCLS options ===
{| class="wikitable"
! Option !! Meaning
|-
| '''Use ffb (fast feedback) file system''' || Read XTC data from the fast feedback system. Only for the active experiment on the priority queues.
|-
| '''Dump all images to disk''' || Write every image as CBF to the job's <code>all/</code> folder, whether or not it indexed. Needed for the ''Strong images that didn't index'' feature of the [[cctbx.xfel GUI/Run Stats tab|Run Stats tab]] and useful when tuning spot finding. Uses a lot of disk space.
|-
| '''Require stream 80 / 81 (FEE spectrometer) before processing''' || Do not process a run until the FEE spectrometer stream is present, so that per-image wavelengths are available.
|}

=== Standalone options ===
In standalone mode the run sentinel watches a folder for new data and turns each file or folder it finds into a run.

{| class="wikitable"
! Option !! Meaning
|-
| '''Folder to monitor''' || The directory new data appear in.
|-
| '''Monitor for''' || ''files'': every file in the folder matching the template is a run. ''folders'': every sub-folder is a run (or, with composite files, each matching file in each sub-folder is a run).
|-
| '''Run complete criteria''' || For folders. ''Status file'': the folder is complete when it contains a Cheetah-style <code>status.txt</code> whose <code>Status</code> is <code>Finished</code>. ''Number of files'': complete once it holds at least '''Number of files per run''' matching files.
|-
| '''Minimum time since last modified''' / '''Minimum file size''' || For files. A file is only picked up once it has been untouched for this many seconds and is at least this many bytes, so that files still being written are not processed.
|-
| '''File matching template''' || Glob pattern for data files, for example <code>*.h5</code> or <code>*.cbf</code>.
|-
| '''Files are composite''' || Tick for formats where one file holds many images (HDF5, NeXus): each file becomes its own run. Untick for one-image-per-file formats, where the whole folder becomes a single run.
|}

Run names in standalone mode are derived from the file or folder names (for example <code>run_0012_data</code> for a composite file <code>data.h5</code> in folder <code>run_0012</code>). Run blocks in standalone mode therefore select runs by database id, in order of discovery, rather than by run number.

== Advanced settings ==
Opened with '''Advanced Settings...'''. The controls shown depend on the chosen multiprocessing method.

=== Multiprocessing options ===
{| class="wikitable"
! Option !! Meaning
|-
| '''Multiprocessing''' || How jobs are run: ''local'' (on this machine, one job at a time), ''lsf'', ''slurm'', ''shifter'' (Slurm with a Shifter container, for NERSC), ''sge'', ''pbs'', ''htcondor'' or ''custom'' (a user-supplied submission template, set in the settings file).
|-
| '''Queue''' || Queue or partition to submit to. A drop-down of the LCLS queues for LSF at LCLS, a free text field otherwise.
|-
| '''Total number of processors''' || Processes per job (MPI ranks). With the LCLS queues this snaps to the core count of a node.
|-
| '''Total number of nodes''' / '''Max Walltime''' || Shifter only: nodes per job and the time limit in minutes.
|-
| '''Number of processors per node''' || ''Auto'' lets the queuing system decide; otherwise an explicit count. Together with the total number of processors this determines the node count.
|-
| '''MPI command''' || The command that launches MPI programs, for example <code>mpirun</code> or <code>srun</code>, with any extra arguments.
|-
| '''Environment setup script''' || A script that is sourced at the start of each job to set up the cctbx environment, when the compute nodes do not inherit yours.
|-
| '''Phenix setup script''' || Likewise for Phenix, used by Phenix tasks in datasets.
|-
| '''Nodes per job''' || Slurm, PBS and Shifter: a separate node count for ''Indexing'', ''TDER'' (time-dependent ensemble refinement), ''Scaling'' and ''Merging'' jobs, since merging usually needs more nodes than indexing a single run.
|-
| '''Extra submission arguments''' || One per line; appended to the submission command (<code>sbatch</code>, <code>bsub</code>, ...). Use this for account names, memory requests and similar.
|-
| '''Shifter settings''' || Shifter image name, the sbatch and srun script templates, job name, NERSC project (<code>-A</code>), reservation, constraint, and whether to stage logs to the DataWarp burst buffer.
|-
| '''HTCondor settings''' || Path to the MPI executable script (<code>openmpiscript</code> or <code>mp2script</code>) and the shared file system domain.
|}

One further parameter has no control: <code>mp.max_queued</code> in the settings file limits how many jobs may be running or queued at once; the job sentinel waits when the limit is reached.

=== Data analysis options ===
'''Processing back end''' chooses the program each indexing job runs:
* ''cctbx.xfel'' (<code>cctbx.xfel.process</code>): DIALS spot finding, indexing, refinement and integration with stills-specific defaults. The usual choice.
* ''Small cell'' (<code>cctbx.xfel.small_cell_process</code>): small-cell indexing (Brewster 2015) followed by DIALS refinement and integration.
* ''custom'': any other program name. Trial parameters are validated against the chosen program's PHIL scope.

Changing the back end after trials exist is not recommended, since the trials' parameters were written for the previous program.

Cctbx.xfel GUI/Settings - Revision history