# Datasets with high random correlations

## Introduction

In our paper, we ran simulations where the range for the random
correlation was [-.8, .8]. A reviewer asked us to consider high
correlations (>.8), based on the fact that high correlations are often
seen in the output from `lmer`

. (Note, however, that a high
correlation may reflect numerical estimation problems rather than high
random correlations "in the world.") Here, we verify that our main
results hold for datasets with high correlations (>.8). To simplify
matters, we consider only datasets including a single
within-subject/within-item factor, and with 24 subject and 12 or 24
items. We also considered only the best performing stepwise model
(the backwards "best path" model).

## Method

We created 10000 additional datasets. In the first 5000 datasets, the by-subject random correlation varied from .8 to 1 or -.8 to -1, while the by-item random correlation varied from -1 to 1. For the second 5000 datasets, the by-item random correlation varied from .8 to 1 or -.8 to -1, while the by-subject random correlation varied from -1 to 1. All other parameters had the same ranges as in the paper.

## Results

### Pre-defined random effects structure

Simulation | Model | typeI.12i | typeI.24i | power.12i | power.24i |
---|---|---|---|---|---|

High Subject Correlation | LMEM, Maximal, \(\chi^2_{LR}\) | 0.062 | 0.056 | 0.460 | 0.614 |

High Subject Correlation | LMEM, No Random Correlations \(\chi^2_{LR}\) | 0.064 | 0.055 | 0.460 | 0.612 |

High Item Correlation | LMEM, Maximal, \(\chi^2_{LR}\) | 0.059 | 0.054 | 0.456 | 0.611 |

High Item Correlation | LMEM, No Random Correlations \(\chi^2_{LR}\) | 0.060 | 0.055 | 0.458 | 0.608 |

Note that all results for Type I error are within .006 of the original simulations, and all results for power are within .004 of the original simulations (results from the original simulations can be found in Table 6 of the main paper).

### Data-driven random effects structure

The Type I error / power of the best stepwise model when random correlations are high is not appreciably different from its performance from the original parameter space (see Figure 2 of our paper), and asymptotes toward the performance of the maximal model.