WFN Bootstrap

Last Updated: 28 November 2014

A brief background to the bootstrap with an example of the bootstrap algorithm can be found here. A description of some fancier bootstrap methods is here. An alternative method for finding a parameter confidence interval is to use log likelihood profiling. See nmllp.

WFN can be used to perform a bootstrap by sampling from the original data set using each individual's group of records as the sampling unit. The bootstrap is run using nmbs. This command works exactly like nmgo but has an additional argument (NBOOT) which specifies the number of bootstrap replications e.g.

nmbs theopdbs 100

This uses the theopdbs.ctl control stream and does 100 bootstrap NONMEM runs. To minimize screen output and thus speed up each run the PRINT=0 option for $ESTIMATION and the $WARN WARN=NONE option are used. Otherwise this control stream is identical to the theopd.ctl file used to illustrate the use of nmgo.

$PROB theophylline pharmacodynamics
$DATA theopd.dat IGNORE #
$INPUT ID TIME THEO AGE WT SEX RACE DIAG PEFR=DV
$ESTIM PRINT=0
$WARN WARN=NONE

$THETA (0,150.,) ; POPE0
$THETA (0,200.,) ; POPEMAX
$THETA (.001,10,) ; POPEC50

$OMEGA 0.5 ; ETAE0
$OMEGA 0.5 ; ETAEMAX
$OMEGA 0.5 ; ETAEC50

$SIGMA 100 ; ERRSD

$PRED

   E0=THETA(1)*EXP(ETA(1))
   EMAX=THETA(2)*EXP(ETA(2))
   EC50=THETA(3)*EXP(ETA(3))

Y = E0 + EMAX*THEO/(THEO+EC50) + ERR(1)

The bootstrap results are saved in a bootstrap sub-directory based on the runname with the extension ".bs" followed by NMDIR e.g. theopdbs.bs.nm7 The bootstrap runs are performed in a temporary sub-directory of the bootstrap sub-directory named nmbs with the NMDIR extension. Only the results of the last bootstrap run are saved in the bootstrap sub-directory.The numerical information in the WFN run summary is extracted and formatted in a tab delimited record which is added to a summary file called theopd.bs.smy which is saved in the bootstrap sub-directory. If the $COVARIANCE step runs successfully the standard errors of the parameters are added to that record in the final summary file. This tab delimited summary file can be easily loaded into other applications such as Excel for further analysis.

The first 4 bootstrap result records are shown below. The "." in the nsig field of records 1 and 3 means that NONMEM did not converge with at least 3 significant digits for these bootstrap runs.

#lst	obj	neval	nsig	nsub	nobs	E0	EMAX	EC50	CVE0	CVEMAX	CVEC50	SD
1	5762.177	359	.	153	571	144	124	2.7	0.00388587	0.510882	2.17715	81.4248
2	5437.659	346	3.3	153	546	146	195	11.2	0.0174069	0.4	1.09545	75.1665
3	5844.682	363	.	153	577	127	198	6.85	0.258263	0.45607	0.00110905	82.5833
4	6267.924	336	4.2	153	618	161	131	3.29	0.136748	0.471169	1.86011	82.9458

The bootstrap can be restarted by specifying a range of bootstrap replicate values:

nmbs theopdbs 900 1000

will run the bootstrap for samples 900 to 1000. The random number generator controlling the bootstrap will start at the same place for run 900 as it would have done if the bootstrap had completed runs starting from run 1.

The random number seed is determined by the starting replication number. This means that if you re-run nmbs it will produce the same results if the starting replication number is the same. If you are want to perform several nmbs runs in parallel on the same problem then be sure to use different starting values e.g.

nmbs theopdbs 1 1000 [equivalent to nmbs theopdbs 1000]
nmbs theopdbs 1001 2000
nmbs theopdbs 2001 3000

Note that nmbs has to assume a value for the IGNORE character in the NONMEM data file. By default this is "#". If you use a different character then change the value of IGNORE in nmbs.awk or change your data file and control streams to use "#".

If the control stream includes $TABLE records then the TABLE file output (with a filename extension defined by the NMTBL environment variable) will be saved in the bs sub-directory in the nmbsall%nmtbl% file (e.g. nmbsall.fit).

For debugging problems with nmbs you can set the environment variable nmbsclean and the temporary files used by nmbs will not be deleted e.g.

set nmbsclean=n

nmbs theopdbs 106 106

will let you find the results of bootstrap run 106 in the theopd.bs directory.