Data Processing at SACLA - Revision history

Mlxd at 01:48, 18 November 2017

2017-11-18T01:48:08Z

← Older revision		Revision as of 01:48, 18 November 2017
Line 78:		Line 78:

	`self.distance = 100.0`		`self.distance = 100.0`

			== Submission of jobs ==
			Use the file '/work/jkern/2017B8085/xrd/run_scripts/test_pierre.sh'

			<code>
			echo -n "enter phil file to use:"
			read phil_inp
			echo $phil_inp

			for run in $(seq $1 $2);
			do

			for chunk in $(seq $3 $4)
			do

			cxi.mpi_submit input.data_dir=/work/jkern/2017B8085/xrd/data/${run}-${chunk} input.run_num=${run} \
			input.dispatcher=dials.stills_process output.output_dir=/work/jkern/2017B8085/xrd/results \
			input.target="/work/jkern/2017B8085/xrd/processing_phils/${phil_inp}" input.run_chunk=$chunk\
			mp.method=pbs mp.queue=bl2-occupancy mp.nnodes=1 mp.nproc_per_node=28 \
			mp.env_script=/home/jkern/xfel_env/setup_env.sh input.data_template="run%d-%d.h5"

			done
			done
			</code>

2017-11-18T01:41:07Z

← Older revision		Revision as of 01:41, 18 November 2017
Line 69:		Line 69:

	Phil file must have good parameters for data processing. Take one from previous users.		Phil file must have good parameters for data processing. Take one from previous users.

			== Converting SACLA pipeline geometry file to DIALS ==
			libtbx.python modules/cctbx_project/xfel/sacla/mpccd_geom2json.py <SACLA_GEOM> distance=<DETECTOR DISTANCE>

			== Modifications to detector distance ==
			Distance can be specified by modifying line 55 of
			/home/jkern/xfel_env/conda_install/modules/cctbx_project/dxtbx/format/FormatHDF5SaclaMPCCD.py

			`self.distance = 100.0`

2017-11-18T00:13:50Z

Obtaining metadata like detector position

2017-11-17T23:15:12Z

@@ Line 5: / Line 5: @@
 --> run sacla geom to json on *.geom to get equivalent for DIALS processing
-) h5_mpi_submit --> launches dials.stills_process with process.phil, and queueing options
+) h5_mpi_submit --> launches dials.stills_process with process.phil, and queueing options.
 Specify what runs (integers)
 == DIALS workflow ==

2017-11-17T22:25:24Z

Obtaining metadata like detector position

← Older revision		Revision as of 22:25, 17 November 2017
Line 4:		Line 4:
	--> update SACLA-provided *.geom file (CrystFEL format)		--> update SACLA-provided *.geom file (CrystFEL format)
	--> run sacla geom to json on *.geom to get equivalent for DIALS processing		--> run sacla geom to json on *.geom to get equivalent for DIALS processing

			2) h5_mpi_submit --> launches dials.stills_process with process.phil, and queueing options
			Specify what runs (integers)

	== DIALS workflow ==		== DIALS workflow ==

2017-11-17T22:21:53Z

← Older revision		Revision as of 22:21, 17 November 2017
Line 14:		Line 14:
	dials.index strong.pickle datablock.json		dials.index strong.pickle datablock.json
	--> indexed.pickle experiments.json		--> indexed.pickle experiments.json

			dials.refine
			--> refined_experiments.json refined_reflections.pickle

			dials.integrate

	== Aggregate processing at XFELS ==		== Aggregate processing at XFELS ==
	~~dials~~.~~stills_process~~ (~~substitutes for~~ the individual steps)		Need to submit a single job for each *.h5 file (manually, or write a script)
			Instead of running the individual steps:
			dials.stills_process *.h5 process.phil

			Phil file must have good parameters for data processing. Take one from previous users.

2017-11-17T22:18:52Z

Headline text

← Older revision		Revision as of 22:18, 17 November 2017
Line 1:		Line 1:

	== ~~Headline text~~ ==		== Obtaining metadata like detector position ==
	1) AgBeh (silver behenate). Determine detector distance and beam center		1) AgBeh (silver behenate). Determine detector distance and beam center
	--> update SACLA-provided *.geom file (CrystFEL format)		--> update SACLA-provided *.geom file (CrystFEL format)
	--> run sacla geom to json on *.geom to get equivalent for DIALS processing		--> run sacla geom to json on *.geom to get equivalent for DIALS processing


	== DIALS workflow ==		== DIALS workflow ==

2017-11-17T22:17:58Z

← Older revision		Revision as of 22:17, 17 November 2017
Line 1:		Line 1:

			== Headline text ==
	1) AgBeh (silver behenate). Determine detector distance and beam center		1) AgBeh (silver behenate). Determine detector distance and beam center
			--> update SACLA-provided *.geom file (CrystFEL format)
			--> run sacla geom to json on *.geom to get equivalent for DIALS processing


			== DIALS workflow ==
			dials.import file.h5 (the h5 will have 1000's of images in it)
			--> datablock.json. Has experimental models as abstracted from image header

			dials.find_spots datablock.json
			--> strong.pickle

			dials.index strong.pickle datablock.json
			--> indexed.pickle experiments.json

			== Aggregate processing at XFELS ==
			dials.stills_process (substitutes for the individual steps)

2017-11-17T22:12:23Z

Created page with "1) AgBeh (silver behenate). Determine detector distance and beam center"

New page

1) AgBeh (silver behenate). Determine detector distance and beam center