runComparison: Run the performance comparison between differential...

In compcodeR: RNAseq data simulation, differential expression analysis and performance comparison of differential expression methods

Description

The main function for performing comparisons among differential expression methods and generating a report in HTML format. It is assumed that all differential expression results have been generated in advance (using e.g. the function runDiffExp) and that the result compData object for each data set and each differential expression method is saved separately in files with the extension .rds. Note that the function can also be called via the runComparisonGUI function, which lets the user set parameters and select input files using a graphical user interface.

Usage

runComparison(
  file.table,
  parameters,
  output.directory,
  check.table = TRUE,
  out.width = NULL
)

Arguments

file.table	A data frame with at least a column input.files, potentially also columns named datasets, nbr.samples, repl and de.methods.
parameters	A list containing parameters for the comparison study. The following entries are supported, and used by different comparison methods: incl.nbr.samples An array with sample sizes (number of samples per condition) to consider in the comparison. If set to NULL, all sample sizes will be included. incl.dataset A dataset name (corresponding to the dataset slot of the results or data objects), indicating the dataset that will be used for the comparison. Only one dataset can be chosen. incl.replicates An array with replicate numbers to consider in the comparison. If set to NULL, all replicates will be included. incl.de.methods An array with differential expression methods to be compared. If set to NULL, all differential expression methods will be included. fdr.threshold The adjusted p-value threshold for FDR calculations. Default 0.05. tpr.threshold The adjusted p-value threshold for TPR calculations. Default 0.05. mcc.threshold The adjusted p-value threshold for MCC calculations. Default 0.05. typeI.threshold The nominal p-value threshold for type I error calculations. Default 0.05. fdc.maxvar The maximal number of variables to include in false discovery curve plots. Default 1500. overlap.threshold The adjusted p-value for overlap analysis. Default 0.05. fracsign.threshold The adjusted p-value for calculation of the fraction/number of genes called significant. Default 0.05. nbrtpfp.threshold The adjusted p-value for calculation of the number of TP, FP, TN, FN genes. Default 0.05. ma.threshold The adjusted p-value threshold for coloring genes in MA plots. Default 0.05. signal.measure Either 'mean' or 'snr', determining how to define the signal strength for a gene which is expressed in only one condition. upper.limits,lower.limits Lists that can be used to manually set the upper and lower plot limits for boxplots of fdr, tpr, auc, mcc, fracsign, nbrtpfp and typeIerror. comparisons Array containing the comparison methods to be applied. The entries must be chosen among the following abbreviations: "auc" - Compute the area under the ROC curve "mcc" - Compute Matthew's correlation coefficient "tpr" - Compute the true positive rate at a given adjusted p-value threshold (tpr.threshold) "fdr" - Compute the false discovery rate at a given adjusted p-value threshold (fdr.threshold) "fdrvsexpr" - Compute the false discovery rate as a function of the expression level. "typeIerror" - Compute the type I error rate at a given nominal p-value threshold (typeI.threshold) "fracsign" - Compute the fraction of genes called significant at a given adjusted p-value threshold (fracsign.threshold). "nbrsign" - Compute the number of genes called significant at a given adjusted p-value threshold (fracsign.threshold). "nbrtpfp" - Compute the number of true positives, false positives, true negatives and false negatives at a given adjusted p-value threshold (nbrtpfp.threshold). "maplot" - Construct MA plots, depicting the average expression level and the log fold change for the genes and indicating the genes called differential expressed at a given adjusted p-value threshold (ma.threshold). "fdcurvesall" - Construct false discovery curves for each of the included replicates. "fdcurvesone" - Construct false discovery curves for a single replicate only "rocall" - Construct ROC curves for each of the included replicates "rocone" - Construct ROC curves for a single replicate only "overlap" - Compute the overlap between collections of genes called differentially expressed by the different methods at a given adjusted p-value threshold (overlap.threshold) "sorensen" - Compute the Sorensen index, quantifying the overlap between collections of genes called differentially expressed by the different methods, at a given adjusted p-value threshold (overlap.threshold) "correlation" - Compute the Spearman correlation between gene scores assigned by different methods "scorevsoutlier" - Visualize the distribution of the gene scores as a function of the number of outlier counts introduced for the genes "scorevsexpr" - Visualize the gene scores as a function of the average expression level of the genes "scorevssignal" - Visualize the gene score as a function of the 'signal strength' (see the signal.measure parameter above) for genes that are expressed in only one condition
output.directory	The directory where the results should be written. The subdirectory structure will be created automatically. If the directory already exists, it will be overwritten.
check.table	Logical, should the input table be checked for consistency. Default TRUE.
out.width	The width of the figures in the final report. Will be passed on to knitr when the HTML is generated.

Details

The input to runComparison is a data frame with at least a column named input.files, containing paths to .rds files containing result objects (of the class compData), such as those generated by runDiffExp. Other columns that can be included in the data frame are datasets, nbr.samples, repl and de.methods. They have to match the information contained in the corresponding result objects. If these columns are not present, they will be added to the data frame automatically.

Value

The function will create a comparison report, named compcodeR_report<timestamp>.html, in the output.directory. It will also create subfolders named compcodeR_code and compcodeR_figure, where the code used to perform the differential expression analysis and the figures contained in the report, respectively, will be stored. Note that if these directories already exists, they will be overwritten.

Author(s)

Charlotte Soneson

Examples

tmpdir <- normalizePath(tempdir(), winslash = "/")
mydata.obj <- generateSyntheticData(dataset = "mydata", n.vars = 1000, 
                                    samples.per.cond = 5, n.diffexp = 100, 
                                    output.file = file.path(tmpdir, "mydata.rds"))
runDiffExp(data.file = file.path(tmpdir, "mydata.rds"), result.extent = "voom.limma", 
           Rmdfunction = "voom.limma.createRmd", output.directory = tmpdir, 
           norm.method = "TMM")
runDiffExp(data.file = file.path(tmpdir, "mydata.rds"), result.extent = "edgeR.exact", 
           Rmdfunction = "edgeR.exact.createRmd", output.directory = tmpdir, 
           norm.method = "TMM", 
           trend.method = "movingave", disp.type = "tagwise")
file.table <- data.frame(input.files = file.path(tmpdir, 
                         c("mydata_voom.limma.rds", "mydata_edgeR.exact.rds")), 
                         stringsAsFactors = FALSE)
parameters <- list(incl.nbr.samples = 5, incl.replicates = 1, incl.dataset = "mydata", 
                   incl.de.methods = NULL, 
                   fdr.threshold = 0.05, tpr.threshold = 0.05, typeI.threshold = 0.05,
                   ma.threshold = 0.05, fdc.maxvar = 1500, overlap.threshold = 0.05,
                   fracsign.threshold = 0.05, mcc.threshold = 0.05, 
                   nbrtpfp.threshold = 0.05, 
                   comparisons = c("auc", "fdr", "tpr", "ma", "correlation"))
if (interactive()) {
  runComparison(file.table = file.table, parameters = parameters, output.directory = tmpdir)
}

Example output

Loading required package: sm
Package 'sm', version 2.2-5.6: type help(sm) for summary information
Registered S3 method overwritten by 'gplots':
  method         from 
  reorder.factor gdata
Warning messages:
1: no DISPLAY variable so Tk is not available 
2: replacing previous import ‘gdata::reorder.factor’ by ‘gplots::reorder.factor’ when loading ‘compcodeR’ 


processing file: tempcode.Rmd

  |                                                                            
  |                                                                      |   0%
  |                                                                            
  |...................................                                   |  50%
  ordinary text without R code


  |                                                                            
  |......................................................................| 100%
label: unnamed-chunk-1 (with options) 
List of 6
 $ echo   : logi TRUE
 $ eval   : logi TRUE
 $ include: logi TRUE
 $ message: logi FALSE
 $ error  : logi TRUE
 $ warning: logi TRUE

Loading required package: limma
Loading required package: edgeR

output file: tempcode.md

[1] TRUE


processing file: tempcode.Rmd

  |                                                                            
  |                                                                      |   0%
  |                                                                            
  |...................................                                   |  50%
  ordinary text without R code


  |                                                                            
  |......................................................................| 100%
label: unnamed-chunk-1 (with options) 
List of 6
 $ echo   : logi TRUE
 $ eval   : logi TRUE
 $ include: logi TRUE
 $ message: logi FALSE
 $ error  : logi TRUE
 $ warning: logi TRUE


output file: tempcode.md

[1] TRUE

compcodeR documentation built on Nov. 10, 2020, 2 a.m.