-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathssx_processing_guide.html
221 lines (189 loc) · 19.4 KB
/
ssx_processing_guide.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
<!DOCTYPE html>
<html lang="en" data-content_root="./">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>SSX processing guide — DIALS documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
<link rel="stylesheet" type="text/css" href="_static/basic.css?v=686e5160" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css?v=27fed22d" />
<link rel="stylesheet" href="_static/dials-styles.css" type="text/css" />
<script src="_static/documentation_options.js?v=5929fcd5"></script>
<script src="_static/doctools.js?v=9bcbadda"></script>
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
<link rel="icon" href="_static/favicon.ico"/>
<link rel="author" title="About these documents" href="about.html" />
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Workshops" href="workshops/index.html" />
<link rel="prev" title="User guide for scaling data with DIALS" href="dials_scale_user_guide.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
</head><body>
<div class="logoheader container">
<a href="index.html">
<img class="logoheader" alt="DIALS" src="_static/dials_header.png" />
</a>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="ssx-processing-guide">
<h1>SSX processing guide<a class="headerlink" href="#ssx-processing-guide" title="Link to this heading">¶</a></h1>
<p>This is a guide on how to process synchrotron serial crystallography (SSX) data
with DIALS, using stepwise command line programs in a similar manner to processing
rotation data.</p>
<section id="indexing-ssx-data-with-dials-ssx-index">
<h2>Indexing SSX data with dials.ssx_index<a class="headerlink" href="#indexing-ssx-data-with-dials-ssx-index" title="Link to this heading">¶</a></h2>
<p>A sequence of SSX images can be imported and spotfinding can be run in the same
way as for rotation data:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="kn">import</span><span class="w"> </span><span class="o">/</span><span class="n">data</span><span class="o">/</span><span class="n">images</span><span class="o">*</span><span class="n">cbf</span>
<span class="n">dials</span><span class="o">.</span><span class="n">find_spots</span> <span class="n">imported</span><span class="o">.</span><span class="n">expt</span>
</pre></div>
</div>
<p>Note that for simplicity in this example we are not using a reference geometry when importing.
These commands produce an <code class="samp docutils literal notranslate"><span class="pre">imported.expt</span></code> experiments file containing the experimental
metadata and a <code class="samp docutils literal notranslate"><span class="pre">strong.refl</span></code> reflections file containing the found spots.
Then to index the data, we can use:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="n">ssx_index</span> <span class="n">strong</span><span class="o">.</span><span class="n">refl</span> <span class="n">imported</span><span class="o">.</span><span class="n">expt</span>
</pre></div>
</div>
<p>This program wraps a call to the dials.index program with options suitable
for processing still images. As with indexing regular sweeps, if the unit cell
and/or space group are known, providing them as input to the program gives a
greater chance of successful indexing. These can be provided as additional options,
for example in this format:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="n">ssx_index</span> <span class="n">strong</span><span class="o">.</span><span class="n">refl</span> <span class="n">imported</span><span class="o">.</span><span class="n">expt</span> <span class="n">space_group</span><span class="o">=</span><span class="n">P4</span> <span class="n">unit_cell</span><span class="o">=</span><span class="mi">60</span><span class="p">,</span><span class="mi">60</span><span class="p">,</span><span class="mi">85</span><span class="p">,</span><span class="mi">90</span><span class="p">,</span><span class="mi">90</span><span class="p">,</span><span class="mi">90</span>
</pre></div>
</div>
<p>For <code class="samp docutils literal notranslate"><span class="pre">dials.ssx_index</span></code>, if the unit cell is given, then indexing will be attempted
on each image with the “fft1d” algorithm, followed by the “real space grid search”
algorithm if the fft1d indexing is not successful. If the unit cell is not given,
only the fft1d algorithm is used. To specify the use of only one particular method,
the <code class="samp docutils literal notranslate"><span class="pre">method</span></code> option can be set to just one of <code class="samp docutils literal notranslate"><span class="pre">fft1d</span> <span class="pre">real_space_grid_search</span></code>.
As each image may contain diffraction from more than one crystal, another useful
option is to set the <code class="samp docutils literal notranslate"><span class="pre">max_lattices</span></code> parameter to greater than one, to
enable multiple crystals to be found on each image.</p>
<p>As the indexing of each image is an independent process, the processing will
be split across the available computing cores, significantly speeding up the
processing when using computing clusters/high performance machines. After
indexing has been attempted on all images, unit cell clustering is performed and
reported. This is particularly useful if the unit cell is not currently known,
as it can be used as a guide for assessing the crystal parameters:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="mi">2</span> <span class="n">clusters</span><span class="p">:</span>
<span class="n">Cluster_id</span> <span class="n">N_xtals</span> <span class="n">Med_a</span> <span class="n">Med_b</span> <span class="n">Med_c</span> <span class="n">Med_alpha</span> <span class="n">Med_beta</span> <span class="n">Med_gamma</span> <span class="n">Delta</span><span class="p">(</span><span class="n">deg</span><span class="p">)</span>
<span class="mi">22</span> <span class="ow">in</span> <span class="n">P1</span><span class="o">.</span>
<span class="n">cluster_1</span> <span class="mi">22</span> <span class="mf">96.38</span> <span class="p">(</span><span class="mf">0.02</span> <span class="p">)</span> <span class="mf">96.39</span> <span class="p">(</span><span class="mf">0.02</span> <span class="p">)</span> <span class="mf">96.42</span> <span class="p">(</span><span class="mf">0.03</span> <span class="p">)</span> <span class="mf">90.02</span> <span class="p">(</span><span class="mf">0.02</span><span class="p">)</span> <span class="mf">90.02</span> <span class="p">(</span><span class="mf">0.03</span><span class="p">)</span> <span class="mf">90.02</span> <span class="p">(</span><span class="mf">0.03</span><span class="p">)</span>
<span class="n">P</span> <span class="n">m</span> <span class="o">-</span><span class="mi">3</span> <span class="n">m</span> <span class="p">(</span><span class="n">No</span><span class="o">.</span> <span class="mi">221</span><span class="p">)</span> <span class="mf">96.40</span> <span class="mf">96.40</span> <span class="mf">96.40</span> <span class="mf">90.00</span> <span class="mf">90.00</span> <span class="mf">90.00</span> <span class="mf">0.038</span>
<span class="mi">7</span> <span class="ow">in</span> <span class="n">P1</span><span class="o">.</span>
<span class="n">cluster_2</span> <span class="mi">7</span> <span class="mf">136.26</span><span class="p">(</span><span class="mf">0.10</span> <span class="p">)</span> <span class="mf">136.32</span><span class="p">(</span><span class="mf">0.04</span> <span class="p">)</span> <span class="mf">166.99</span><span class="p">(</span><span class="mf">0.13</span> <span class="p">)</span> <span class="mf">90.01</span> <span class="p">(</span><span class="mf">0.05</span><span class="p">)</span> <span class="mf">90.01</span> <span class="p">(</span><span class="mf">0.03</span><span class="p">)</span> <span class="mf">119.94</span><span class="p">(</span><span class="mf">29.64</span><span class="p">)</span>
<span class="n">P</span> <span class="mi">6</span><span class="o">/</span><span class="n">m</span> <span class="n">m</span> <span class="n">m</span> <span class="p">(</span><span class="n">No</span><span class="o">.</span> <span class="mi">191</span><span class="p">)</span> <span class="mf">136.33</span> <span class="mf">136.33</span> <span class="mf">166.99</span> <span class="mf">90.00</span> <span class="mf">90.00</span> <span class="mf">120.00</span> <span class="mf">0.077</span>
</pre></div>
</div>
<p>Repeating indexing with the unit cell/space group of the most populous cluster
can then improve the overall indexing result.</p>
<p>To help with the interpretation of the indexing results for a large number of
images, a <code class="samp docutils literal notranslate"><span class="pre">dials.ssx_index.html</span></code> report is generated which contains plots
of useful statistics such as the number of spots indexed on each image, the distribution
of rmsd values and unit cell clustering analysis. This data can also be output to
json format for further analysis, by providing a filename to the option <code class="samp docutils literal notranslate"><span class="pre">output.json</span></code>.</p>
<p>For weak/sparse serial collections, it may be the case that few images contain
a useful number of spots. To allow rapid assessment in such cases,
<code class="samp docutils literal notranslate"><span class="pre">dials.ssx_index</span></code> will skip attempted indexing of images which contain fewer
than <code class="samp docutils literal notranslate"><span class="pre">min_spots</span></code> strong spots (default value 10).</p>
<p>The log output of the program is minimal, however as with other DIALS programs,
this can be increased by setting the verbosity level. <code class="samp docutils literal notranslate"><span class="pre">-v</span></code> will add timestamps
to the log file, and <code class="samp docutils literal notranslate"><span class="pre">-vv</span></code> will show standard logging output from the underlying
<code class="samp docutils literal notranslate"><span class="pre">dials.index</span></code> code.</p>
<p>To summarise the main options (and their default values):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">space_group</span> <span class="o">=</span> <span class="kc">None</span> <span class="p">:</span> <span class="s1">'Index in this space group'</span>
<span class="n">unit_cell</span> <span class="o">=</span> <span class="kc">None</span> <span class="p">:</span> <span class="s1">'Index with this unit cell'</span>
<span class="n">max_lattices</span> <span class="o">=</span> <span class="mi">1</span> <span class="p">:</span> <span class="s1">'Max crystal lattices to search for per image'</span>
<span class="n">method</span> <span class="o">=</span> <span class="n">fft1d</span> <span class="n">real_space_grid_search</span> <span class="p">:</span> <span class="s1">'Indexing methods to try if suitable'</span>
<span class="n">min_spots</span> <span class="o">=</span> <span class="mi">10</span> <span class="p">:</span> <span class="s1">'Skip indexing of images with fewer than this number of spots'</span>
<span class="o">-</span><span class="n">vv</span> <span class="p">:</span> <span class="s1">'Output additional logging from dials.index code'</span>
<span class="n">output</span><span class="o">.</span><span class="n">html</span> <span class="o">=</span> <span class="n">dials</span><span class="o">.</span><span class="n">ssx_index</span><span class="o">.</span><span class="n">html</span> <span class="p">:</span> <span class="s1">'If not None, write a summary html report to this file'</span>
<span class="n">output</span><span class="o">.</span><span class="n">json</span> <span class="o">=</span> <span class="kc">None</span> <span class="p">:</span> <span class="s1">'If not None, write summary plots data to this file'</span>
</pre></div>
</div>
<p>To see the full list of options with descriptions, run <code class="samp docutils literal notranslate"><span class="pre">dials.ssx_index</span> <span class="pre">-ce2</span> <span class="pre">-a2</span></code></p>
</section>
<section id="integrating-ssx-data-with-dials-ssx-integrate">
<h2>Integrating SSX data with dials.ssx_integrate<a class="headerlink" href="#integrating-ssx-data-with-dials-ssx-integrate" title="Link to this heading">¶</a></h2>
<p>After indexing, the experimental models can be further refined with dials.refine,
or the indexing output can also be integrated directly.
To integrate the data, we can use:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="n">ssx_integrate</span> <span class="n">indexed</span><span class="o">.</span><span class="n">expt</span> <span class="n">indexed</span><span class="o">.</span><span class="n">refl</span>
</pre></div>
</div>
<p>This program wraps a call to parts of the <code class="samp docutils literal notranslate"><span class="pre">dials.integrate</span></code> program,
using either the <code class="samp docutils literal notranslate"><span class="pre">stills</span></code> integrator or the <code class="samp docutils literal notranslate"><span class="pre">ellipsoid</span></code> integration algorithm.
The stills integrator is the default algorithm used for integration in
<code class="samp docutils literal notranslate"><span class="pre">dials.stills_process</span></code>. The ellipsoid algorithm refines the unit cell,
orientation and a 3D ellipsoidal mosaicity parameterisation for each crystal,
by assessing the pixel-intensity distribution of the strong spots:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="n">ssx_integrate</span> <span class="n">indexed</span><span class="o">.</span><span class="n">refl</span> <span class="n">indexed</span><span class="o">.</span><span class="n">expt</span> <span class="n">algorithm</span><span class="o">=</span><span class="n">stills</span>
<span class="n">dials</span><span class="o">.</span><span class="n">ssx_integrate</span> <span class="n">indexed</span><span class="o">.</span><span class="n">refl</span> <span class="n">indexed</span><span class="o">.</span><span class="n">expt</span> <span class="n">algorithm</span><span class="o">=</span><span class="n">ellipsoid</span>
</pre></div>
</div>
<p>Processing will be split across the available computing cores for performance.
During processing, data files will be created after each batch of crystals has
been processed. The size of the batch for saving data can be set with the
<code class="samp docutils literal notranslate"><span class="pre">batch_size</span></code> option. This creates numbered output files such as
<code class="samp docutils literal notranslate"><span class="pre">integrated_0.refl,</span> <span class="pre">integrated_0.expt,</span> <span class="pre">integrated_1.refl,</span> <span class="pre">integrated_1.expt</span></code> etc.
After all images have been integrated, unit cell clustering is performed and
reported, as this will have changed compared to at the end of indexing if
using the ellipsoid integration algorithm.</p>
<p>To help with the interpretation of the integration results for a large number of
crystals, a <code class="samp docutils literal notranslate"><span class="pre">dials.ssx_integrate.html</span></code> report is generated which contains plots
of useful statistics such as the number of spots integrated on each image,
the modelled mosaicity values and unit cell clustering analysis. This data can
also be output to json format for further analysis, by providing a filename to
the option <code class="samp docutils literal notranslate"><span class="pre">output.json</span></code>.</p>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper"><!--<h3>Navigation</h3>-->
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="index.html">Home</a></li>
<li class="toctree-l1"><a class="reference internal" href="about.html">About</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="documentation/index.html">Documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="documentation/tutorials/index.html">Tutorials</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="howto.html">How-to</a></li>
<li class="toctree-l1"><a class="reference external" href="https://dials.github.io/kb">Knowledge Base</a></li>
<li class="toctree-l1"><a class="reference internal" href="workshops/index.html">Workshops</a></li>
<li class="toctree-l1"><a class="reference internal" href="national_resource.html">US National Resource</a></li>
<li class="toctree-l1"><a class="reference internal" href="publications.html">Publications</a></li>
<li class="toctree-l1"><a class="reference internal" href="projects.html">Projects</a></li>
<li class="toctree-l1"><a class="reference internal" href="license.html">License</a></li>
</ul>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer container">
<a href="https://www.diamond.ac.uk/"><img class="logofooter" alt="Diamond" src="_static/diamond_logo.png" /></a>
<a href="https://www.ccp4.ac.uk/"><img class="logofooter" alt="CCP4" src="_static/CCP4-logo-plain.png" /></a>
<a href="https://www.stfc.ac.uk/"><img class="logofooter" alt="STFC" src="https://dials.github.io/images/logos/ukri-stfc.png" /></a>
<a href="https://www.lbl.gov/"><img class="logofooter" alt="LBL" src="https://dials.github.io/images/logos/lbl.png" /></a>
<a href="http://smb.slac.stanford.edu/"><img class="logofooter" alt="SSRL/SMB" src="https://dials.github.io/images/logos/smbssrl.jpg" /></a>
</div>
<script type="text/javascript">
$(document).ready(function() {
$(".toggle > *").hide();
$(".toggle .header").show();
$(".toggle .header").click(function() {
$(this).parent().children().not(".header").toggle(400);
$(this).parent().children(".header").toggleClass("open");
})
});
</script>
<div class="footer">
©2025, Diamond Light Source, Lawrence Berkeley National Laboratory and STFC.
</div>
</body>
</html>