Clinical Trial Simulation

Last Updated: 3 April 2012

Clinical Trial Simulation

WFN provides a mechanism for simulating clinical trials which allows design properties to be varied with different models and model parameters for both simulation and estimation.

The WFN clinical trial simulation (CTS) sub-system relies on a single Command Prompt batch command file to specify the nature of the CTS experiment by setting environment variables. These variables are then used by cts.bat to generate a data file template, simulate data according to a specific design and simulation model and analyse data using a specific estimation model. When these steps are complete the overall results of the CTS experiment can be summarized in terms of power and bias and imprecision. If necessary a null model can be used for simulation in order to use the randomization test to determine critical objective function values which would reject the null at a specified alpha value.

A key distinction is made between the model used to simulate data (the simulation model) and the model used to estimate parameters based on simulated data (estimation model).

Each estimation model can be run independently of the NONMEM run that generated the simulated data. This overcomes a bug in NONMEM design leading to failure to complete a series of sub-problems generated by a single control stream problem.

A sample batch file for use with cts is shown here.

@echo off
rem ******************************************************
set simhome=%wfnhome%\run\cts\lagtime
set simrun=%simhome%\run
set mkdat=simdat
set designs=40
set dose=100
set times=0#0.1#0.2#0.3#0.4#0.5#0.75#1#2#3#4#5#6#9#12
set nsub=100
set par=24
set ctl=lag
set trumdl=tru
set estmdl=est
set est=I
set seed=20040518
set nrep=100
set ctsgotdata=
set startrep=
set ctsclean=n
rem **** process results ***********
set trueval=ka1
set parm=POPCL POPV POPLAG POPTABS PPVCL PPVLAG PPVTABS RUVSD
set peonly=
set rtcrit=n
set success=A
set alpha=0.05
rem ******************************************************
call %wfnhome%\bin\cts

Table 1 Environment Variables Used to Control CTS

Variable	Description	Example	Explanation
CTSHOME	Home for CTS files which are typically kept in %CTSHOME%\bin.	set CTSHOME=%WFNHOME%	These are generic files for use by all CTS simulation projects and do not need user changes %WFNHOME%\bin is the default location for CTS files distributed with WFN
SIMHOME	Home for project specific cts files	set SIMHOME=\docs\drug\cts	Files summarising CTS results are created in %SIMHOME%. %SIMHOME%\bin is the default location for CTS control files. These files refer to a specific simulation project and have to be edited by the user as required. Files include awk scripts to create simulation data template and simulation model control streams, template control streams and a list of true parameter values.
MKDAT	awk script used to generate data file template [if omitted then CTSFILE is used as the data file template] The following arguments will be automatically available to the MKDAT awk script: DSN individual designs from DESIGNS (see below) DOSE, TIME, NOCC, NSUB, NOBS, NSEQ (see below) The user is responsible for interpreting these arguments in the awk script.	set MKDAT=simdat	By default it is located in %SIMHOME%\bin and has the extension .awk. A fully qualified path can be used to refer to a file not in %SIMHOME%\bin. The example refers to %SIMHOME%\bin\simdat.awk
CTSFILE	Input template data file for MKDAT [optional if MKDAT can generate template file without an input file]	set CTSFILE=theopd	Name of an input data file for MKDAT. By default it is located in %SIMHOME%\bin and has the extension .dat. A fully qualified path can be used to refer to a file not in %SIMHOME%\bin. The example refers to %SIMHOME%\bin\theopd.dat
DESIGNS	List of one or more design names	set DESIGNS=sparse dense	A design refers to a set of properties e.g. number of subjects, sampling times or models and parameters. DESIGNS can be used to control output of both MKDAT and FIXCTL e.g. sparse and dense data sets are simulated.
CTL	Name of one or more template NM-TRAN control streams	set CTL=mymdl	Each template control stream is used by FIXCTL to create simulation and estimation control streams. By default it is located in %SIMHOME%\bin and has the extension %NMDIR% (e.g. .ctl). A fully qualified path can be used to refer to a file not in %SIMHOME%\bin. The example refers to %SIMHOME%\bin\mymdl.ctl
MKCTL	Awk script used to generate simulation and estimation models from CTL This must be derived from the mkctl.awk template. The following arguments will be automatically made available to the MKCTL awk script: DSN individual design from DESIGNS (see below) PAR, EST, SEED, NREP ( see below) DAT design specific data file SMDL individual simulation model from TRUMDL (see below) EMDL individual estimation model from ESTMDL (see below) The user is responsible for interpreting these arguments in the awk script.	set MKCTL=fixctl	Much of the flexibility of CTS comes from use of MKCTL to define different simulation and estimation models based on the template control stream(s) in CTL. By default it is located in %SIMHOME%\bin and has the extension .awk. A fully qualified path can be used to refer to a file not in %SIMHOME%\bin. The example refers to %SIMHOME%\bin\fixctl.awk In the simplest case there is no need to specify the MKCTL variable. If MKCTL is not defined then %CTSHOME%\bin\mkctl.awk will be used by default.
TRUMDL	List of one or more simulation model names	set TRUMDL=s0	TRUMDL is used by FIXCTL to generate simulation models in NM-TRAN control streams
ESTMDL	List of one or more estimation model names	set ESTMDL=s0 sm	ESTMDL is used by FIXCTL to generate estimation models in NM-TRAN control streams
SEED	Simulation seed	set SEED=20030727	Seed passed to NONMEM for simulation
EST	Estimation method	set EST=C	NONMEM estimation method option chosen from Z[ERO] (default), C[OND] or I[NTER].
NREP	Number of simulation replications	set NREP=100	Typical numbers of replications would be 10 to estimate bias, 30 for RMSE, 100 for power and 1000 for 95% predictions or null hypothesis critical values.
TRUEVAL	File containing names and true values of model parameters	set TRUEVAL=mymodel	True values are used to compute estimation bias. The user is responsible for ensuring he true values in this file correspond to those used for simulation (as defined by CTL and FIXCTL) Format is white space delimited list of parameter names (case insensitive) in line one and corresponding numeric values in line 2 By default it is located in %SIMHOME%\bin and has the extension .tru. A fully qualified path can be used to refer to a file not in %SIMHOME%\bin. e.g. True values in mymodel.tru are in 2 lines: POPCL POPV POPEMAX POPEC50 1.5 30 100 10
PARM	List of parameters whose bias and imprecision (RMSE) are to be estimated	set PARM=POPCL POPV	Parameter names must match those used in the estimation control stream generated by FIXCTL. WFN extended control stream parameter names may be used to identify $THETA, $OMEGA and $SIGMA estimates. Otherwise parameters must be identified by THETA1, OMEGA1, SIGMA1, etc. Case insensitive.
	Command used to execute CTS	call %ctshome%\bin\cts	If %WFNHOME% is the same as %CTSHOME% (the usual default) then the command "call cts" is all that is required.

Table 2 Optional Environment Variables Used to Control CTS

Variable	Description	Example	Explanation
SIMRUN	Run directory for project.	set SIMRUN=%SIMHOME%\FOCE	Simulation and estimation sub-directories and files are created in this directory. The default value for %SIMRUN% is %SIMHOME% e.g. Modelling files saved in FOCE sub-directory of %SIMHOME%
NSUB	Number of subjects. This can refer to number per treatment group of total number	set NSUB=500	Total number of subjects in the trial. Used by simdat.awk
DOSE	String of dose sizes separated by #	set DOSE=0#50#100#400	Four dose treatments are defined. Used by simdat.awk.
TIMES	String of observation times separated by #	set TIMES=0#1#2#3#4#5#9#12	Times are used by simdat.awk.
NOCC	Number of occasions	set NOCC=4	Four occasions are defined. Used by simdat.awk.
NSEQ	Number of sequences of treatments	set NSEQ=3	Three sequences of treatments are defined. Used by simdat.awk.
PAR	Used by FIXCTL to modify simulation model parameters	set PAR=0.5	FIXCTL uses this value to set a simulation model parameter. This simple example defines a value of 0.5 for parameter referred to in FIXCTL. More complex values for PAR can be interpreted by the user defined awk script FIXCTL
ISTBL	Used to request that estimation $TABLE output should be saved	set ISTBL=y	If ISTBL not set to 'y' then any $TABLE records generated for estimation models by MKCTL will be ignored.
CTSCLEAN	Determines if simulation data is removed at end of run	set CTSCLEAN=n	If CTSCLEAN is set to 'n' then simulated data is not removed. This must be set on a CTS run prior to using CTSGOTDATA.
CTSGOTDATA	Determines if simulation step is performed	set CTSGOTDATA=y	If CTSGOTDATA is set to 'y' then the simulation step is not performed. This saves time when re-running with different estimation steps that use the same simulated data
STARTREP	Starting replication number for estimation step	set STARTREP=10	The default value of STARTREP is 1. Can be set to a higher value when re-starting the estimation step e.g. if the CTS run was stopped prematurely.
PEONLY	Determines if simulation/estimation steps are performed prior to computation of prediction errors etc.	set PEONLY=y	if PEONLY is set to 'y' then simulation/estimation steps are omitted
ALPHA	Type I error level for rejection of the null hypothesis	set ALPHA=0.05	Used to calculate the power of a design to reject the null hypothesis. When used with a randomization test simulation it is used to estimate CRIT
CRIT	Critical value for decrease in NONMEM objective function required to reject the null hypothesis with a specified Type I error	set CRIT=-3.84	Value of CRIT is typically estimated by performing a randomization test. A randomization test uses TRUMDL to simulate data under the null hypothesis and ESTMDL under the alternate hypothesis. CTS can be used to estimate the value of CRIT required for a subsequent simulation which estimates the power for rejecting the null hypothesis
RTCRIT	If estimation runs are intended to determine the difference in objective function value under the null hypothesis then RTCRIT is used to indicate that the value of CRIT for a specified ALPHA should be computed.	set RTCRIT=y	If RTCRIT is set to 'y' then objective function differences are used to determine CRIT and results are reported in a *_rt.txt file in the SIMHOME folder

CTS Batch Commands

Simulate Template Data Set

The following awk script is used to simulate a template data set for the LAGTIME problem. It uses the DSN and PAR arguments to define the number of lagged doses and the dosing interval.

#simdat
BEGIN {

NDOSE=DSN
II=PAR

if (NSUB=="") NSUB=10
if (DOSE=="") DOSE=100
if (TIME=="") TIME="0#0.1#0.2#0.3#0.4#0.5#0.75#1#2#3#4#5#6#9#12"
if (NDOSE=="") NDOSE=1
if (II=="") II=24

ntimes=split(TIME,TIMES,"#")
printit("#id","time","dosen","dosint","amt","dv","mdv")
for (id=1; id<=NSUB; id++) {
   for (j=1; j<=NDOSE+1; j++) {
      printit(id,TIMES[1]+(j-1)*II,NDOSE,II,DOSE,".",1)
      if (j>(NDOSE-1)) { # collect samples after last 2 doses
         for (k=1; k<=ntimes; k++)
         printit(id,TIMES[k]+(j-1)*II,NDOSE,II,",",dv,0)
      }
   }
}
exit
}

function printit(id,time,dosen,dosint,dose,dv,mdv) {
printf("%s,%s,%s,%s,%s,%s,%s\n",id,time,dosen,dosint,dose,dv,mdv)
}

Generate NM-TRAN Control Streams

mkctl.awk Template

The generic template for a MKCTL awk script is shown below. For the LAGTIME problem no changes are required so the MKCTL variable is not defined and the default awk script %CTSHOME\bin\mkctl.awk is used.

More commonly the template control stream(s) specified in the CTL record will require user specific changes. An example of this can be found for the THEOPD problem.

#mkctl
BEGIN {
### CTS User Specific ####

# put user specific initialization code here

### CTS General -- Modify with Care ####

DSN=toupper(DSN)
PAR=toupper(PAR)
SMDL=toupper(SMDL)
EMDL=toupper(EMDL)
split(EMDL,ARGS,"#"); EMDL=ARGS[1]
isonlysim=(ARGS[2]=="ONLYSIM")
split(SMDL,ARGS,"#"); SMDL=ARGS[1]
isonlyest=(ARGS[2]=="ONLYEST")
gotsimrec=0

EST=toupper(EST)
if (EST=="") {
isfoce=(substr(EMDL,length(EMDL),1)=="C")
isinter=(substr(EMDL,length(EMDL),1)=="I")
} else {
isfoce=(EST=="C")
isinter=(EST=="I")
}
istbl=(toupper(TBL) == "Y")
CTSCTL=TMP
if (SEED=="") SEED=20021101

} # end BEGIN

/\$INPUT/ {
print
#The following code creates a default list of items for the simulation $TABLE record
#i.e. the $INPUT data items are replicated in $TABLE
$1=""; $0=$0
output=$0
do {
   ok=getline
   if (ok) {
      if (match($0,/^\$/) || match($0,/BEGIN/))
         ok=0
      else {
         print
         output=output"\n"$0
      }
   }
} while (ok)
#If other items are required then replace the following expression defining
#the output item list
simoutput=output

### CTS User Specific ####

# put user specific simulation output list here

# e.g.

# simoutput="ID TIME CMT LOCF DV MDVX DVIDX EVIDX"

### CTS General -- Modify with Care ####
}

/\$TAB/ {
if (isonlysim) exit
if (istbl) print
   do {
      ok=getline
      if (ok && (match($0,/^\$/) || match($0,/BEGIN/)))
         ok=0
      else
         if (istbl) print
   } while (ok)
}

isonlyest && match(toupper($0),/;BEGIN SIM/) {
do {
   ok=getline
} while (ok>0 && ! match(toupper($0),/;END SIM/))
next
}

isonlysim && match(toupper($0),/;BEGIN EST/) {
do {
   ok=getline
} while (ok>0 && ! match(toupper($0),/;END EST/))
next
}

/\$DATA/ {
if ( DAT != "" ) {
   do {
      ok=getline<CTSCTL
      if (tolower($1)=="simrun") simrun=$2"\\"
   } while (ok>0)
   $0="$DATA "simrun DAT" IGNORE=@"
}
}

/\$EST/ {
if (isonlysim) next
print "$ESTIM MAXEVAL=9990 SIGDIG=3 NOABORT"
if (isfoce)
   print "METHOD=CONDITIONAL"
else if (isinter)
   print "METHOD=CONDITIONAL INTERACTION"
else if (istbl)
   print "POSTHOC"
next
}
/\$COV/ {
if (isonlysim) next
}
/\$SIM/ {
simrecord=$0
gotsimrec=1
next
}

/METHOD\=COND/ || /POSTHOC/ { next }

{
### CTS User Specific ####

# Put User Specific code here

### CTS General -- Modify with Care ####
}

{ print }

END {
simrec()
}

function simrec() {
if (isonlyest) return
if (NREP=="") NREP=1
if (isonlysim) {
   print "$TABLE ONEHEADER NOAPPEND NOPRINT FILE=cts.fit"
   print simoutput
   onlysim="ONLYSIM"
} else
   onlysim=""
   if (gotsimrec)
   print simrecord
else
   print "$SIM ("SEED" NEW) ("SEED+1" UNIFORM) SUBPROBLEMS="NREP" "onlysim
}

The lag control stream used as a template for the LAGTIME problem is shown here.

It should be noted that the $TABLE record is ignored when CTS creates simulation or estimation control streams.

$PROB lag
$DATA lag.csv
$INPUT ID TIME NDOS DOSI AMT DV MDV
$ESTIM MAX=9990 SIG=3 NOABORT PRINT=0
METHOD=COND INTER
$THETA
(0.1,3,) ; POPCL
(0.1,10,) ; POPV
(0,0.3,) ; POPLAG
(0,0.5,) ; POPTABS
$OMEGA
.09 ; PPVCL
.09 ; PPVV
.09 ; PPVLAG
.09 ; PPVTABS
$SIGMA .01 FIX ; RUVSD

$SUBR ADVAN2 TRAN2
$PK
IF (NEWIND.LE.1) LN2=LOG(2)
CL=POPCL*EXP(PPVCL)
V=POPV*EXP(PPVV)
IF (ID.GT.1) THEN
TLAG=(NDOS-1)*DOSI
ELSE
TLAG=0
ENDIF
ALAG1=TLAG+POPLAG*EXP(PPVLAG)
TABS=POPTABS*EXP(PPVTABS)
KA=LN2/TABS
S2=V

$ERROR
CP=F
Y=CP + RUVSD

$TABLE ID NDOS DOSI ALAG1 TIME CP Y
ONEHEADER NOPRINT FILE=lag.fit

Simulation

This is the simulation control stream created by CTS using mkctl.awk:

$PROB lag
$DATA ..\..\..\40\40.dat IGNORE=#
$INPUT ID TIME NDOS DOSI AMT DV MDV
$THETA
(0.1,3,) ; POPCL
(0.1,10,) ; POPV
(0,0.3,) ; POPLAG
(0,0.5,) ; POPTABS
$OMEGA
.09 ; PPVCL
.09 ; PPVV
.09 ; PPVLAG
.09 ; PPVTABS
$SIGMA .01 FIX ; RUVSD

$SUBR ADVAN2 TRAN2
$PK
IF (NEWIND.LE.1) LN2=LOG(2)
CL=POPCL*EXP(PPVCL)
V=POPV*EXP(PPVV)
IF (ID.GT.1) THEN
TLAG=(NDOS-1)*DOSI
ELSE
TLAG=0
ENDIF
ALAG1=TLAG+POPLAG*EXP(PPVLAG)
TABS=POPTABS*EXP(PPVTABS)
KA=LN2/TABS
S2=V

$ERROR
CP=F
Y=CP + RUVSD

$TABLE NOHEADER NOAPPEND NOPRINT FILE=cts.fit
ID TIME NDOS DOSI AMT DV MDV
$SIM (20040518 NEW) (20040519 UNIFORM) SUBPROBLEMS=100 ONLYSIM

Estimation

This is the estimation control stream created by CTS using mkctl.awk:

$PROB lag
$DATA nmsamp.dat IGNORE=#
$INPUT ID TIME NDOS DOSI AMT DV MDV
$ESTIM MAXEVAL=9990 SIGDIG=3 NOABORT
METHOD=CONDITIONAL INTERACTION
$THETA
(0.1,3,) ; POPCL
(0.1,10,) ; POPV
(0,0.3,) ; POPLAG
(0,0.5,) ; POPTABS
$OMEGA
.09 ; PPVCL
.09 ; PPVV
.09 ; PPVLAG
.09 ; PPVTABS
$SIGMA .01 FIX ; RUVSD

$SUBR ADVAN2 TRAN2
$PK
IF (NEWIND.LE.1) LN2=LOG(2)
CL=POPCL*EXP(PPVCL)
V=POPV*EXP(PPVV)
IF (ID.GT.1) THEN
TLAG=(NDOS-1)*DOSI
ELSE
TLAG=0
ENDIF
ALAG1=TLAG+POPLAG*EXP(PPVLAG)
TABS=POPTABS*EXP(PPVTABS)
KA=LN2/TABS
S2=V

$ERROR
CP=F
Y=CP + RUVSD

True Parameter Values

The true parameter values used to simulate the data are recorded in the TRUE file. These values are used to calculate bias and imprecision of the parameter estimates from estimation runs. The parameter names and values in the ka1.tru file for the LAGTIME problem are shown here.

Note: The true values for the estimates of random effect parameters should be expressed as the SQRT of the corresponding OMEGA or SIGMA value used for simulation. This is because b default WFN reports these values as the SQRT of the NONMEM estimate. If the NMRAW environment variable is set to Y then WFN will report the variance values estimated by NONMEM and the values in the TRU file should also be variances.

POPCL POPV POPLAG POPTABS PPVCL PPVV PPVLAG PPVTABS RUVSD
3 10 0.3 0.5 0.3 0.3 0.3 0.3 0.1