power.prop.test() should do it for you. To get the math to work, I converted your “ignored” data into impressions by summing up my columns.
> power.prop.test(p1=8/200, p2=60/2000, power=0.8, sig.level=0.05) Two-sample comparison of proportions power calculation n = 5300.739 p1 = 0.04 p2 = 0.03 sig.level = 0.05 power = 0.8 alternative = two.sided NOTE: n is number in *each* group
This gives 5301 that for each group, so your sample size should be 10,600. Subtracting 2,200 that have already been run, you have 8,400 "tests."
In this case:
sig.level matches your p value.power is the probability of finding significant results that exist in your example. This is somewhat arbitrary, 80% is a common choice. Please note that choosing 80% means that in 20% of cases you will not find the value when you want. Increasing power means you will need a larger sample size to achieve your desired level of significance.
If you decide how long it takes to achieve value, divide 8400 by the number of impressions per day. This may help determine whether to continue the test.
You can also use this function to determine the required sample size before testing. There's a nice entry describing this on the 37 Signals blog.
This is a built-in R function, so you do not need to add or download any packages. Other than that, I can't say how much this looks like pwr.p2pn.test() .