README.md 2.33 KB
Newer Older
Kevin Bronik's avatar
Kevin Bronik committed
1
# Performance Analysis Tool
Kevin Bronik's avatar
Kevin Bronik committed
2

Kevin Bronik's avatar
Kevin Bronik committed
3 4
This software  calculates all the following important performance metrics 
(can be used to evaluate algorithmic performance against a manual segmentation):
Kevin Bronik's avatar
Kevin Bronik committed
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
- Dice Similarity Index (SI) 
- Sensitivity  
- Precision  
- Specificity 
- Jaccardindex  
- Conformity 
- HausdorffDistance
- Overlapcoefficient 
- Tanimotocoefficient
- Matthewscorrelationcoefficient  
- Accuracy 
- Voxel-level False Discovery Rate (FDR) 
- Voxel-level False Negative Ratio (FNR)
- Cluster-level FDR 
- Cluster-level FNR 
- Detection Error Rate (DER) 
- Outline Error Rate (OER)
- Mean Total Area (MTA)
- Volume of segmentation
- Volume of manual mask

# Running the Program!

Use one of the following methods to run the program:

(1) Using shell script file "RunMe.sh" which can be run by the Unix shell

```sh
  - chmod 755 RunMe.sh  
Kevin Bronik's avatar
Kevin Bronik committed
34
  - ./RunMe.sh <lesionmask> <threshold> <manualmask> <foldershortname> <modelname> <distancefunction> <factor/opt> <symmetry> <weight> <p_norm> <saveoutput>
Kevin Bronik's avatar
Kevin Bronik committed
35
  or
Kevin Bronik's avatar
Kevin Bronik committed
36
  - bash  RunMe.sh <lesionmask> <threshold> <manualmask> <foldershortname> <modelname> <distancefunction> <factor/opt> <symmetry> <weight> <p_norm> <saveoutput>
Kevin Bronik's avatar
Kevin Bronik committed
37
```  
Kevin Bronik's avatar
Kevin Bronik committed
38 39 40
where
```sh
 - <lesionmask>: Segmented lesion mask using lesion prediction algorithm
Kevin Bronik's avatar
Kevin Bronik committed
41 42
 - <threshold>: Probability threshold that will be applied to  output before calculation of the overlap measures
 - <manualmask>: The manual segmentation 
Kevin Bronik's avatar
Kevin Bronik committed
43
 - <foldershortname>: The testing folders short name
Kevin Bronik's avatar
Kevin Bronik committed
44 45
 - <modelname>: The Model Name that has been used to  train the data 
 - <distancefunction>: Euclidean, minkowski, cityblock, cosine, hamming, seuclidean, sqeuclidean, correlation, jaccard, chebyshev, canberra, braycurtis, mahalanobis, yule, matching, dice, kulsinski, rogerstanimoto, russellrao, sokalmichener, sokalsneath, wminkowski 
Kevin Bronik's avatar
Kevin Bronik committed
46 47
 - <factor/opt>: Mean, direct 
 - <symmetry>: Symmetric, nonsymmetric 
Kevin Bronik's avatar
Kevin Bronik committed
48
 - <weight>: Needed to compute the weighted Minkowski distance
Kevin Bronik's avatar
Kevin Bronik committed
49 50
 - <p_norm>: The order of norm
 - <saveoutput>: 0, 1
Kevin Bronik's avatar
Kevin Bronik committed
51
```
Kevin Bronik's avatar
Kevin Bronik committed
52
(2) Using shell script file "RunMeWithGui.sh" which can be also run by the Unix shell
Kevin Bronik's avatar
Kevin Bronik committed
53

Kevin Bronik's avatar
Kevin Bronik committed
54 55 56 57 58 59 60
```sh
  - chmod 755 RunMeWithGui.sh  
  - ./RunMeWithGui.sh
  or
  - bash  RunMeWithGui.sh
``` 
  Here a graphical user interface (GUI) will allow the user easily to run the program
Kevin Bronik's avatar
Kevin Bronik committed
61 62 63 64
 
 <br>
 <img height="700" src="https://cmiclab.cs.ucl.ac.uk/kbronik/performance_analysis/blob/master/images/GUI.jpg" />
 <br>