I've been keeping track of my scroll pass, failure and boom rates ever since I started 1.5 years ago. These are my findings. Chaos scrolls are obviously full of bullshit! 60% pass rate my ass... 10% scrolls used 424 ... passed 53 12.50% ... failed 371 87.50% 30% scrolls used 1,259 ... passed 392 31.14% ... failed 486 38.60% ... boomed 381 30.26% 60% scrolls used 727 ... passed 426 58.60% ... failed 301 41.40% 70% scrolls used 1,361 ... passed 965 70.90% ... failed 244 17.93% ... boomed 152 11.17% Chaos Scrolls used 293 ... passed 151 51.54% ... failed 142 48.46% Average stat bonus -0.02 ... +5 75 9.16% ... +4 74 9.04% ... +3 73 8.91% ... +2 74 9.04% ... +1 71 8.67% ... 0 74 9.04% ... -1 80 9.77% ... -2 75 9.16% ... -3 73 8.91% ... -4 74 9.04% ... -5 76 9.28% Anyone else out there who likes statistics who keeps track of his/her scrolling results? :O
These are all just your own? If you've had 109 chaos scrolls pass, how do you have ~70 samples for each chaos scroll result?
That would depend on the number of lines of stats that the equipment has that is being scrolled. If you chaos an item that only has 1 line of stats that can be changed, e.g. if there was a shoe which had 5 dex, and no def or anything else, then it would only either do one -5, -4, -3, -2 -1, 0, +1,+2, +3, +4 or +5 change. If it was on a shoe with 5 dex, 5 wdef, 5mdef, then there would be 3 stat changes. The total of Marty's stat changes comes to 699, which means that on average each passed chaos scroll was changing around 7 lines of stats.
Yeah somewhere around that. PGCs have 3 lines, and I've chaos'd loads of other stuff (HTP/Zhelm/Dkhanjar/Ring and whatnot). I think the number isn't 100% correct, though, I'll double-check it. Hell, I've even chaos'd the Rudolph's nose and a set of bottoms. Edit: Just checked all my Chaos screenshots (yes, I screenshot every Chaos result I get). It's more Chaos scrolls, since I have a total of 568 screenshots, each before&after, so a total of 284 Chaos scrolls used. I'll go through them all to double-check the fails/passes. I made the Chaos statistics with the screenshots a lot later, so I guess I miscounted or something... Edit: Checked all the passes/fails, it's 144/140. still really low passrate.
Well, most numbers are only 1-2% above or below the expected average, so yeah. Only Chaos haven't balanced out so far O: (And the boom rates are quite low compared to fail rates on 30/70%)
That is not how statistics work. If we assume that Chaos scrolls are supposed to have a sixty per cent success rate, the outcome Marty has observed is statistically significant at fairly low significance levels. In a Z-test of proportions with the null hypothesis as above and the alternate hypothesis of the success rate being lower yields a p-value of just below a tenth of a per cent, which is quite convincing evidence to reject the hypothesis of the success rate being sixty per cent. (This assuming one believes the data isn't biased, of course.)
The LLN would still certainly be expected to apply if he increased his number of attempts massively, but I think I understand what you're saying. Adonkooo would be incorrect if he meant that, assuming Marty used ~300 more, the success rate of those would be around 70% to balance out the previously low success rate (gambler's fallacy); however he would be correct if he was saying that, assuming Marty used ~1500 more scrolls, the success rate of those 1500 could be expected to be very near to 60%. Then combining those results were combined with the earlier 284 attempts would cause the overall success rate to move much closer to 60%. Obviously I can't tell exactly what he meant, but you're correct if he meant it in the first sense, which is a misconception that occurs much too often.
Scrolling is a chance process that has outcomes that we cannot predict but since the success rate is 60%, it will have a regular distribution in many many repetitions (which is what I meant by saying in the "long run"). As you can see, his sample size for chaos scrolls is smaller compared to any other scrolls he used. When you scroll there are obviously two possible outcomes; it either fails or it works. The proportion of successes might vary at first, but as we scroll more and more, the proportion of successes gets close to .6 and it stays there. This fact is guaranteed by the law of large numbers. I don't know how you misinterpreted what I said and started prattling on about significance tests.
Completely unrelated, but I changed the OP to tables so I can update it whenever I have some new results. I've used 9 Chaos Scrolls ever since I posted this thread, 7/9 worked. Too bad I got mediocre results. ): So that puts it at 151/293... only 1/495 people who used 293 chaos scrolls would get this or a lower passrate. O: @David I'd love to prove it, but... B> 1500 chaos, rip all my mesos
We are arguing different things. You are arguing that if the success rate of a Chaos scroll really is sixty per cent, then the expected outcome is that six in ten experiments succeed. As you rightly point out, the law of large numbers tells us that we must approach this mean as we increase the number of trials. Marty—or his data—however, is suggesting that the ostensible success rate of sixty per cent for Chaos scrolls is false. In particular, it is suggesting that it is smaller. If this is the case the law of large numbers certainly still applies—it always does—but the mean we will approach won't be 0.6. Further, almost three hundred data points is not a small sample size, especially when considering something as simple as this with literally no confounding variables. Indeed, as I indicate above, if we assume that the probability of passing a Chaos scroll really is 60%, as you do, then the chance of passing only 144 or fewer out of 284, in any trial of 284 Chaos scrolls, is less 0.0009, which is certainly of statistical interest.
Of course the mean would not approach .6 if the success rate of chaos scrolls was not actually 60%, which would indicate that there is an error in the system. Although the sample size, 284, is not small, it is not big either.
