---

title: Additional Information


keywords: fastai
sidebar: home_sidebar

summary: "This Notebook contains information on use of _deepflash2_."
description: "This Notebook contains information on use of _deepflash2_."
nb_path: "nbs/add_information.ipynb"
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/add_information.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
        
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Required-Data-Structure-and-Naming">Required Data Structure and Naming<a class="anchor-link" href="#Required-Data-Structure-and-Naming"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Ground-Truth-Estimation">Ground Truth Estimation<a class="anchor-link" href="#Ground-Truth-Estimation"> </a></h3><ul>
<li><strong>One parent folder</strong></li>
<li><strong>One folder per expert</strong></li>
<li><strong>Identical names for segmentations</strong></li>
</ul>
<p><em>Examplary structure:</em></p>
<ul>
<li>[folder] parent_folder<ul>
<li>[folder] expert1<ul>
<li>[file] mask1.png</li>
<li>[file] mask2.png</li>
</ul>
</li>
<li>[folder] expert1<ul>
<li>[file] mask1.png</li>
<li>[file] mask2.png</li>
</ul>
</li>
</ul>
</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Training">Training<a class="anchor-link" href="#Training"> </a></h3><ul>
<li><strong>One folder for training images</strong><ul>
<li>Images must have unique name or ID</li>
<li>_0001.tif --&gt; name/ID: 0001; img_5.png --&gt; name/ID: img<em>5, ...</em> </li>
</ul>
</li>
<li><strong>One folder for segmentation masks</strong><ul>
<li>Corresponding masks must start with name or ID + a mask suffix__<ul>
<li>_0001 -&gt; 0001_mask.png (mask_suffix = "<em>mask.png")</em></li>
<li>_0001 -&gt; 0001.png (mask<em>suffix = ".png")</em></li>
<li>mask suffix is inferred automatically </li>
</ul>
</li>
</ul>
</li>
</ul>
<p><em>Examplary structure:</em></p>
<ul>
<li>[folder] images<ul>
<li>[file] 0001.tif</li>
<li>[file] 0002.tif</li>
</ul>
</li>
<li>[folder] masks<ul>
<li>[file] 0001_mask.png</li>
<li>[file] 0002_mask.png</li>
</ul>
</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Prediction">Prediction<a class="anchor-link" href="#Prediction"> </a></h3><ul>
<li><strong>One folder for training images</strong><ul>
<li>Images must have unique name or ID<ul>
<li>_0001.tif --&gt; name/ID: 0001; img_5.png --&gt; name/ID: img<em>5, ...</em> </li>
</ul>
</li>
</ul>
</li>
<li><strong>One folder containing trained models (ensemble)</strong><ul>
<li>Ensemble folder and models will be created during Training__<ul>
<li>Do not change the naming of the models</li>
<li>If you want to train different ensembles, simply rename the ensemble folder</li>
</ul>
</li>
</ul>
</li>
</ul>
<p><em>Examplary structure:</em></p>
<ul>
<li>[folder] images<ul>
<li>[file] 0001.tif</li>
<li>[file] 0002.tif</li>
</ul>
</li>
<li>[folder] ensemble<ul>
<li>[file] unext50_deepflash2_model-1.pth</li>
<li>[file] unext50_deepflash2_model-2.pth</li>
</ul>
</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Train-validation-split">Train-validation-split<a class="anchor-link" href="#Train-validation-split"> </a></h2><p>The train-validation-split is defined as _<a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html">k-fold cross validation</a>_ with <code>n_splits</code></p>
<ul>
<li><code>n_splits</code> is the minimum of: (number of files in dataset,  <code>max_splits</code> (default:5))</li>
<li>By default, the number of models per ensemble is limited to <code>n_splits</code></li>
</ul>
<p><em>Example for a dataset containing 15 images</em></p>
<ul>
<li><code>model_1</code> is trained on 12 images (3 validation images) </li>
<li><code>model_2</code> is trained on 12 images (3 different validation images) </li>
<li>...</li>
<li><code>model_5</code> is trained on 12 images (3 different validation images) </li>
</ul>
<p><em>Example for a dataset containing 2 images</em></p>
<ul>
<li><code>model_1</code> is trained on 1 image (1 validation image) </li>
<li><code>model_2</code> is trained on 1 images (1 different validation image) </li>
<li>Only two models per ensemble</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Training-Epochs-and-Iterations">Training Epochs and Iterations<a class="anchor-link" href="#Training-Epochs-and-Iterations"> </a></h2><p>To streamline the training process and allow an easier comparison across differently sized datasets, we decided to use the number of training <em>iterations</em> instead of <em>epochs</em> to define the lenght of a <a href="https://matjesg.github.io/deepflash2/utils.html#calc_iterations">training cycle</a>.</p>
<p>Some useful definitions (adapted from <a href="https://stackoverflow.com/questions/4752626/epoch-vs-iteration-when-training-neural-networks">stackoverflow</a>):</p>
<ul>
<li>Epoch: one training pass (forward pass and one backward pass) of all the training examples</li>
<li>Batch size: the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you'll need.</li>
<li>Iteration: One forward pass and one backward pass using [batch size] number of examples.</li>
</ul>
<p><em>Example:</em>
Your dataset comprises 20 images and you want to train for 1000 iterations given a batch size of 4. The <a href="https://matjesg.github.io/deepflash2/utils.html#calc_iterations">algorithm</a> calculates the minimum of epochs needed to train 1000 iterations):</p>
<p>$Epochs = \frac{iterations}{\frac{\#images}{batch size}} = \frac{1000}{\frac{20}{4}} = 200$</p>
<p>The number of epochs will be ceiled to the next integer.</p>

</div>
</div>
</div>
</div>