README.md 2.89 KB
Newer Older
Kevin Bronik's avatar
Kevin Bronik committed
1
# Performance Analysis Tool
Kevin Bronik's avatar
Kevin Bronik committed
2

Kevin Bronik's avatar
Kevin Bronik committed
3 4
This software  calculates all the following important performance metrics 
(can be used to evaluate algorithmic performance against a manual segmentation):
Kevin Bronik's avatar
Kevin Bronik committed
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
- Dice Similarity Index (SI) 
- Sensitivity  
- Precision  
- Specificity 
- Jaccardindex  
- Conformity 
- HausdorffDistance
- Overlapcoefficient 
- Tanimotocoefficient
- Matthewscorrelationcoefficient  
- Accuracy 
- Voxel-level False Discovery Rate (FDR) 
- Voxel-level False Negative Ratio (FNR)
- Cluster-level FDR 
- Cluster-level FNR 
- Detection Error Rate (DER) 
- Outline Error Rate (OER)
- Mean Total Area (MTA)
- Volume of segmentation
- Volume of manual mask

# Running the Program!

Kevin Bronik's avatar
Kevin Bronik committed
28 29
First user will need to download and extract all the files at the root of folder where all testing folders previously placed, then user can use one of the following methods to run the program:

Kevin Bronik's avatar
Kevin Bronik committed
30 31 32 33 34

(1) Using shell script file "RunMe.sh" which can be run by the Unix shell

```sh
  - chmod 755 RunMe.sh  
Kevin Bronik's avatar
Kevin Bronik committed
35
  - ./RunMe.sh <lesionmask> <threshold> <manualmask> <foldershortname> <modelname> <distancefunction> <factor/opt> <symmetry> <weight> <p_norm> <saveoutput>
Kevin Bronik's avatar
Kevin Bronik committed
36
  or
Kevin Bronik's avatar
Kevin Bronik committed
37
  - bash  RunMe.sh <lesionmask> <threshold> <manualmask> <foldershortname> <modelname> <distancefunction> <factor/opt> <symmetry> <weight> <p_norm> <saveoutput>
Kevin Bronik's avatar
Kevin Bronik committed
38
```  
Kevin Bronik's avatar
Kevin Bronik committed
39 40 41
where
```sh
 - <lesionmask>: Segmented lesion mask using lesion prediction algorithm
Kevin Bronik's avatar
Kevin Bronik committed
42 43
 - <threshold>: Probability threshold that will be applied to  output before calculation of the overlap measures
 - <manualmask>: The manual segmentation 
Kevin Bronik's avatar
Kevin Bronik committed
44
 - <foldershortname>: The testing folders short name
Kevin Bronik's avatar
Kevin Bronik committed
45 46
 - <modelname>: The Model Name that has been used to  train the data 
 - <distancefunction>: Euclidean, minkowski, cityblock, cosine, hamming, seuclidean, sqeuclidean, correlation, jaccard, chebyshev, canberra, braycurtis, mahalanobis, yule, matching, dice, kulsinski, rogerstanimoto, russellrao, sokalmichener, sokalsneath, wminkowski 
Kevin Bronik's avatar
Kevin Bronik committed
47 48
 - <factor/opt>: Mean, direct 
 - <symmetry>: Symmetric, nonsymmetric 
Kevin Bronik's avatar
Kevin Bronik committed
49
 - <weight>: Needed to compute the weighted Minkowski distance
Kevin Bronik's avatar
Kevin Bronik committed
50 51
 - <p_norm>: The order of norm
 - <saveoutput>: 0, 1
Kevin Bronik's avatar
Kevin Bronik committed
52
```
Kevin Bronik's avatar
Kevin Bronik committed
53
(2) Using shell script file "RunMeWithGui.sh" which can be also run by the Unix shell
Kevin Bronik's avatar
Kevin Bronik committed
54

Kevin Bronik's avatar
Kevin Bronik committed
55 56 57 58 59 60
```sh
  - chmod 755 RunMeWithGui.sh  
  - ./RunMeWithGui.sh
  or
  - bash  RunMeWithGui.sh
``` 
Kevin Bronik's avatar
Kevin Bronik committed
61
  Here a graphical user interface (GUI) will allow user easily to run the program
Kevin Bronik's avatar
Kevin Bronik committed
62
 
Kevin Bronik's avatar
Kevin Bronik committed
63
 
Kevin Bronik's avatar
Kevin Bronik committed
64 65 66
 ![Screenshot](GUI.jpg)
 
 
Kevin Bronik's avatar
Kevin Bronik committed
67
 and after Help Button Clicked, user will see the following help window which can help user easily fill in the required fields
Kevin Bronik's avatar
Kevin Bronik committed
68 69 70
 
 
 ![Screenshot](GUI-with_help.jpg)
Kevin Bronik's avatar
Kevin Bronik committed
71 72
 
 
Kevin Bronik's avatar
Kevin Bronik committed
73 74 75 76 77 78
 and after user will fill the form with necessary information and click OK button, user will see the following message
 
 <br>
 <img height="300" src="AfterClickOK.jpg" />
 </br>

Kevin Bronik's avatar
Kevin Bronik committed
79
finally after All processes done!, user can see the result in a file "final_result.txt",  saved at the root of folder where user previously has extracted all the program files.
Kevin Bronik's avatar
Kevin Bronik committed
80