• G*Power installed on your laptop
• Your own project
• Optionally, R with the simr package installed

Slides are available at https://humburg.github.io/Power-Analysis.

• Introduction
• Using G*Power
• Advanced power analysis

• Null Hypothesis Testing
• Study planning
  • Determine number of subjects required
  • Is it possible to recruit enough participants to reliably identify effects of the anticipated size?
  • What effect size can you reasonably expect to detect reliably?
• Provides rough estimates
  • Helps to ensure feasibility
  • Treat as ball park estimate not precise number

• May miss real effects
• If results are significant, risk of overestimating effect size is increased

• May miss real effects
• If results are significant, risk of overestimating effect size is increased
• Increased false discovery rate (but false positive rate doesn’t change)

• Risk of false positive result for individual study not affected by power.
• BUT low powered studies as a whole, produce higher proportion of false positive results.

• Research hypothesis
  • Study design
  • Analysis plan
    • Analysis methods
    • Tests to be carried out
    • Significance threshold
• Variability of outcome measure
• Size of effect

• Power analysis requires information about the data prior to data collection

• How do you know what effect size to use?

  • Literature review
  • Published data
  • Pilot experiment
  • Practical significance

• What about variability?

  • Other studies using the same outcome measure
  • Variability in control group
  • Only need a rough estimate

• G*Power is available from http://www.gpower.hhu.de/.
• Provides GUI to facilitate power calculations for a variety of common tests.
• No build-in help but manual and tutorial are available online.

• One-way ANOVA.
• Factor with 4 levels.
• Equal group sizes.
• Expected standard deviation within each group is 2.
• Expected mean for groups 1 and 2 is 1, 0 for the others.

What is the required sample size if you use \(\alpha = 0.05\) and aim for 80% power?

• 2 \(\times\) 3 ANOVA.
• Variance explained by effect of interest is 10% of error variance.

What is the required sample size if you use \(\alpha = 0.05\) and aim for 80% power for

• the main effect of the two level factor?
• the interaction between the two factors?

• Multiple regression.
• Determine sample size for single effect of interest.
• Variance explained by predictor of interest is 10% of residual variance.

What is the required sample size if you use \(\alpha = 0.05\) (in a two-tailed test) and aim for 80% power?

• Treatment and Control group.
• Measure before and after intervention.
• Variance explained by intervention effect is 10% of within group variance.

What is the required sample size if you use \(\alpha = 0.05\) and aim for 80% power?

• How to calculate power for methods not covered by GPower?
• Find a specialised tool that supports power analysis for the analysis method you intend to use.
• If in doubt, simulate.

• Not easily accomplished with GPower.
• Can assess power to detect deviation from null model by treating RMSEA as effect size.
• Online tools available at http://www.quantpsy.org/rmsea/rmsea.htm and http://timo.gnambs.at/en/scripts/powerforsem

• GPower supports simple repeated measures ANOVA designs.
• More complex models best handled with simulation.
• simR supports simulation from lme4 models.

• Use existing data to fit model of interest.
• Adjust effect and sample size as desired.
• Repeatedly simulate datasets from model and determine proportion of these for which effect of interest is significant.
• Also workes without data, but need to guess all parameters.
• Detailed example available online.