ModEx allows near real-time interaction with your model only using probabilistic sensitivity analyses (PSA) results from your model.
ModEx is based on peer-reviewed research on the use of regression metamodeling.
In the background ModEx fits a regression metamodeling approach to the data.
From this it allows visual and numerical displays, interactive incremental cost-effectiveness ratio (ICER) display,
result decomposition into various parameters, and very fast EVPPI and EVSI computations.
ModEx uses peer-reviewed methods, and Open Source R packages developed by the
Decision Analysis in R for Technology and Health workgroup (DARTH).
To start exploring ModEx, please upload a PSA dataset in the next tab (Import files)
Here is a summary of the various tabs:
Import files: you can upload a CSV of your PSA result using this tab and check the uploaded data.
Diagnostics: shows the quality of the metamodel fit, and a check on pair-wise parameter correlations
Sensitivity: numerical and graphical display of sensitivity analyses, including summaries, CEAC, Scatterplot, Regression summaries, and graphical displays of tornado, one-way and two-way plots.
ICER: interactive display of the ICER under various parameter settings.
Decomposition: interacive displays the contribution of each parameter to each outcomes
Report: allows producing a report of all the analyses conducted and saving it locally
EVSI: (coming soon!) computes the expected value of partial perfect Information (EVPPI) and expected value of sample information (EVSI) using extremely fast algorithms
Disclaimer: This application is provided AS-IS and comes with absolutely no warranty.
If you use ModEx please cite our papers
Jalal, H., Dowd, B., Sainfort, F., & Kuntz, K. M. (2013). Linear regression metamodeling as a tool to summarize and present simulation model results. Medical Decision Making, 33(7), 880-890.
Jalal, H., & Alarid-Escudero, F. (2018). A Gaussian approximation approach for value of information analysis. Medical Decision Making, 38(2), 174-188.
Setting up your files for import
Here you can import your probabilistic senesitivity analysis (PSA) dataset.
The PSA dataset must follow a specific format.
Your PSA dataset must be uploaded as a single CSV file.
The PSA dataset must follow the following format: (Please download the example file below)
- Rows are PSA model runs, and columns are model parameters, and cost and effectivenss outcomes
- Columns indicating model input parameters must start with 'p_'.
- Columns indicating cost outcomes must start with 'c_'.
- Columns indicating effectiveness outcomes must start with 'e_'.
- There should be an equal number of cost and effectiveness columns.
NHB = Net Health Benefit, NMB = Net Monetary Benefit.
Then, use an Example PSA dataset
You can start exploring ModEx using a PSA dataset from a stylistic Markov model involving cancer therapy.
The model involves 8 input parameters and 3 treatment strategies.
To learn more about this example, please refer to Jalal et al. 2013 MDM paper on the Home tab.
You can also downlaod this dataset here.
Note: Once you upload the data, you cannot change the NB units or WTP.
If you want to modify these settings, please do the changes first and then re-upload your PSA.
Uploaded Data Check
Parameters
Costs
Effects
Setting
Pairwise Correlation Analysis
This tab provides a visual and numerical analysis of the correlations
between different input parameters. Scatterplots are provided for each
pair of variables along with their correlation coefficients.
Collinear Variables
Running diagnostics for the linear metamodel
ModEx fits a model in the form Y = β*X + e, where X are the model input parameters,
Y are the model outputs (costs and QALYs), and β are the regression coefficients.
This tab shows the fits for each outcome and also the residuals to determine the degree of fit,
and also guide if further modificaiton of the parameters are needed to improve the fit.
The Correlation Analysis shows the level of correlation in the pair-wise parameters.
Parameters that are entirely computed from other parameters should be avoided.
The coefficients indicate change in the outcome due to 1 unit change in each parameter.
The intercept indicates the outcome value if all parameters were = 0.
Standardized Regression Results
The coefficients indicate change in the outcome due to 1 standard deviation change in each parameter.
The intercept indicates the average outcome value if all parameters were = their mean values.
ICER
Here you can interact with the Incremental Cost-Effectiveness Ratio (ICER) plot by adjusting the model parameters.
It does this quickly by using the regression metamodel in the background.
Decomposition
Here, you can reveal how much each parameter contributes to each strategy outcome, and how these parameters'
contributions change as you interactively adjusts the model parameters.
Download summary report
This report will contain all the tables and figures generated from your uploaded dataset. Please be patient it may take some time to produce.
The regression metamodeling method was developed by Hawre Jalal in collaboration with
Karen Kuntz, Bryan Dowd, Francois Sainfort and John Nyman and was partly
supported by Hawre Jalal's AHRQ Dissertation Award R36HS020868
The Gaussian approximation method was developed by Hawre Jalal in collaboration with Fernando Alarid-Escudero
and was partly supported by Hawre Jalal's Career Development Award K01DA048985.
If you use ModEx please cite our papers
Jalal, H., Dowd, B., Sainfort, F., & Kuntz, K. M. (2013). Linear regression metamodeling as a tool to summarize and present simulation model results. Medical Decision Making, 33(7), 880-890.
Jalal, H., & Alarid-Escudero, F. (2018). A Gaussian approximation approach for value of information analysis. Medical Decision Making, 38(2), 174-188.
The initial version of ModEx used the Shiny App code from the Sheffield Accelerated Value of Information (SAVI).
Since then, it got a full re-write of the code to reveal health economic model characteristics using metamodelling.
In addition, ModEx allows to interactively allow flexible and extremely fast EVSI computation.
If you have any questions, please contact us at hjalal@uottawa.ca.
