Home | Installation | Control Streams | Bootstrap | Randomization Test | Visual Predictive Check | Autocovariate | Files | References

Last Updated: 3 April 2012

A brief background to the randomization test can be found here.

WFN can be used
to perform a randomization test by randomly permuting a covariate in the
original data set. The bootstrap is run using *nmrt*.
This command works exactly like *nmgo* but has
two additional arguments, *covname*, which is the name of a data item in the $INPUT record
of the NM-TRAN control stream, and NBOOT, the number of replications of the
data set.

nmrt sex theopdsex 1000

This command
randomizes the covariate SEX in the *theopd.dat* dataset with the *theopdsex** *model and does 1000 NONMEM runs.

The randomization test can be restarted by specifying a range of bootstrap replicate values:

nmrt sex theopdsex 900 1000

will run the randomization test for samples 900 to 1000. The random number generator controlling the randomization test will start at the same place for run 900 as it would have done if the randomization test had completed 899 runs starting from run 1.

The randomization
runs are performed in a temporary sub-directory named nmbs
with the NMDIR extension. The randomization test results are saved in a
randomization test sub-directory based on the **runname**
with the extension ".rt_"COVNAME e.g. *theopdb.rt_wt*. Only the results of the last
randomization test run are saved in the randomization test sub-directory.The numerical information in the WFN run summary
is extracted and formatted in a tab delimited record which is added to a
summary file called *theopdbs.txt* which is saved in the randomization
test sub-directory. If the $COVARIANCE step runs successfully the standard
errors of the parameters are added to that record in the final summary file.
This tab delimited summary file can be easily loaded into other applications
such as Excel for further analysis.

If *theopdsex* is run in the normal fashion with *nmgo* there is a fall in objective function to
5798.270 compared to the base model *theopd*
value of 5802.086. The change in objective
function is 3.816 units. If the chi-square distribution is used to predict the
probability of falsely accepting the null hypothesis that sex did not affect
Emax with alpha=0.05 and 1 degree of freedom then a change in objective
function of greater than 3.84 would usually be accepted as sufficient evidence
to conclude sex had an effect. The change in objective
function in this example is close to the critical value if we assume the
chi-square distribution but how close it the true P-value to the one based on
this assumption?

The true P-value
associated with rejection of the null hypothesis can be calculated by
performing the following steps in Excel.

1. Open the thepdsex.rt_sex\theopdsex.txt file with Excel

2. Insert 3 columns
between the "#rep" and "Obj" columns.

3. Name these
columns "Delta", "Pvalue","Quantile".

4. Enter the
following formulas into row 2 and copy to rows 3:1001.

Delta: = 5802.086 - E2 ;
5802.086 is the Obj value for the *theopd*
base model. F2 is "Obj"

Pvalue: =chidist(B2,1)
; B2 is "Delta". 1 degree of freedom.

Quantile: =(row()-1)/count($E$2:$E$1001)
; Computes the quantile for each replication

5. Sort of column
"Obj" ascending

6. Find the row with
quantile closest to the desired alpha e.g. 0.05

7. The value of
Delta in this row is the critical value of the difference in objective function
that you should use with this data set to test the hypothesis that sex is a
covariate that explains variability in Emax.

8. The value of Pvalue is the P value predicted by the chi-square
distribution.

Home | Installation | Control Streams | Bootstrap | Randomization Test | Visual Predictive Check | Autocovariate | Files | References