Visual Predictive Check

Last Updated: 21 January 2021

Visual Predictive Checks for Time To Event Data

A visual predictive check (VPC) is used for non-linear mixed effect model evaluation and communication of results for time to event data. In contrast to VPCs used for evaluation of continuous data the principle of a VPC is to compare the time course of a statistic derived from event times such as the survivor function with the distribution percentiles of the derived statistic obtained simulated event times e.g. 95% interval of Kaplan-Meier estimates of the survivor function.

WFN provides a mechanism for creating VPCs using NONMEM and R. A Windows batch command file is used to run NONMEM and R. It must be edited to specify the NM-TRAN control stream name for simulation of event times, a data set for observed event times and plotting options for the resulting output.

Installation

The VPC run folder contains the NM-TRAN simulation control stream for simulating the model and producing simulated event times.

The files nmvpc2r.awk, T2E_OBS.R, T2E_VPC.R and T2E_VPC_Functions.R files must be located in the run folder containing a Windows batch command file (e.g. named ttevpc.bat) to run the VPC simulation and plotting process. These files should be copied from the %WFNHOME%\bin\tte folder. An example batch command file can be found in %WFNHOME%\bin\tte\warfarin_TTE.

WFN must be installed with NONMEM and a Fortran compiler. The R system must be installed with the path to the R.exe executable file in the file search path. This can be verified by typing ‘rgui’ at the command prompt in a WFN window. The R path may be set in wfn.bat near line 60 e.g.

set RPATH=C:\Apps\R\R-4.0.3\bin\x64

The NM-TRAN simulation control stream and the ttevpc.bat file will need to be customized before running a Windows batch command file such as ttevpc.bat.

No R packages need to be installed. VPCs are created using a pair of R scripts. These are called by ttevpc.bat using R in batch mode. The VPCs created by ttevpc.bat are saved as pdf files.

When ttevpc.bat is run a tmp_vpc.R script is available which can be sourced in R by users who want to explore how ttevpc is working or modify the R script to customize the output.

Customizing the NM-TRAN Simulation Control Stream

A VPC is constructed from the observed event times used for estimation of the likelihood of the event times and simulated values predicted from the final estimates of the model used for estimation.

1. The data required to describe the observed events should include ID, TIME, DVID, DV and MDV data items. The DVID item must be set to a value corresponding to that used in the simulation control stream that identifies the event record.

2. The NM-TRAN control stream is typically based on a model used to estimate the hazard in the observed event data set. Use nmctl.bat to put the final estimates from the model for the event hazard into the control stream used to run that model e.g.

nmctl warf_INR_est.ctl

3. Copy the control stream containing the final estimates to another folder e.g.from an est folder used for parameter estimation to a tte folder (the VPC folder). This provides a base to use the same model structure for event simulation. It is suggested you use a different name from the estimation model control stream e.g.

mkdir tte

copy est\warf_INR_est.ctl tte\warf_INR_sim.ctl

4. Change to the VPC folder and edit the control stream

cd tte

edit warf_INR_sim.ctl

5. The following changes should be made to use the observed event data for the tte VPC and convert the estimation control stream into one suitable for event simulation:

a. Copy the observed event data file to the VPC folder.

b. Create code for detecting simulated events. There are 2 simulation specific sections (ICALL.EQ.4) which will need to incorporated into the model used to estimate the hazard. An example is provided here: %WFNHOME%\vpc\tte\template_TTE_sim.ctl file