Re: power analysis and sample size

[<jlittell@xxxxxxxxxxxx>]

Power is the ability to detect effects (or differences) that are "real." In 
statistical terms, it is 1-b, where b is the probability of a Type II error 
(the probability of failing to reject the null hypothesis when the null 
hypothesis is false). 

Power is affected by the size of the effect (or difference) one wants to 
detect (larger effect sizes are easier to find), sample size (larger samples 
provide greater power), the "p-value" or alpha level (probability of making a 
Type I error), and the statistical test one is using. Parametric tests are 
more powerful than non-parametric tests. Tests that control for other sources 
of variance (ANCOVA, multiple regression) are more powerful than bivariate 
tests. 

To use G*power (or any other power analysis program) to determine how many 
cases you need (an "a priori" power analysis), 

1) determine what kind of test you will use. (If you want to compare the 
proportion of runaways in parent-visited vs non-visited cases, you could use 
Chi-square. If you think that number of visits may relate to the probability 
of running away, you might use logistic regression. Or maybe you want to look 
at the correlation between number of visits and number of runs.) 

2) Determine what size of effect matters to you (large samples are needed to 
detect small effects and you might miss clinically important differences if 
you seek only large effects, so I'd recommend that you look for medium 
effects). 

3) Decide what level of power you need (Cohen recommended a minimum of .8, 
which means you'd only find "real" effects in 80% of the samples you could 
look at); most of the time we want higher power levels, say .9 or .95. 

4) Decide whether you will use a one-tailed or two-tailed tests (one-tailed 
tests are more powerful).

5) Set the alpha level (probablility of Type I error). This is usually .05, 
but you might settle for .1, since higher alphas increase power.

6) Plug these values into an a priori power analysis (you can contact me off 
line for more specific info on how to do this in G*power) and the program 
will tell you how many cases you need.

You can fool around with these parameters (try different tests, effect sizes, 
etc.) to see what effects that has on sample size requirements. 

If the results suggest that you need more than 75 cases total and you think 
you can't do more than 75, try a "post hoc" power analysis. That is an 
analysis that starts with a fixed sample size and determines what can be 
detected and how. To do this, plug in the number of cases you think you can 
do, your test, your effect size and the test will tell you how much power you 
have. (or you can hold power constant and it will tell you what effect size 
you can detect). 

Mark Lipsey's book Design Sensitivity (Sage) is an excellent resource on 
this, as is his chapter on design sensitivity in Bickman and Rog's (1998) 
Handbook of Applied Social Research Methods (Sage).

Best wishes,
Julia 

bill higgins <bill_higginsus@yahoo.com> said:

> We are planning to do a study using case records.  We
> want to understand the probability of running away as
> it related to parental visits.  We have little
> research knowledge in our office so could someone help
> us resolve the issue of power and sample size.  I have
> the G-power program which was a free download, but we
> are not sure how to use it.  We were thinking of using
> 75 cases which will take a lot of time, but is this
> enough?  How many is enough?  What exactly is power?  
> 
> Bill
> 
> 
> __________________________________
> Do you Yahoo!?
> New Yahoo! Photos - easier uploading and sharing.
> http://photos.yahoo.com/
> 



--