Some reviewers of my Dice Roller app have said the distribution of rolls isn’t as random as it could (or should) be. This is a record of my attempts to test the randomness of the app. I appreciate (almost) all the reviews I get, even the negative ones.
One complaint is that D20 rolls seem skewed toward the bottom of the range. Another is that rolls on a D20 don’t seem random enough, and that rolling a group of D20s gave a surprising number of duplicate values.
First, let me say that there is no intentional biasing of results. I’m making every effort to make sure rolls are random. On the default settings (in particular, the “throw” roll method), each die gets:
- a random initial rotation,
- a random forward force,
- a random left/right force, and
- a random spin.
After that, the rest of the roll is handled by the physics engine in Unity, the game engine I’m using.
100D20
The first test was to roll 100 D20s in the app twice, then roll a physical D20 100 times, and compare the distributions. A disclaimer: I have no idea whether my physical dice are fair. I have no reason to believe they aren’t.
Before you scroll too far, compare the three bar charts below. The x axis is the result (1–20) and the y axis is how many times it came up in 100 rolls. Two charts are from the app and one is from a physical die. Pick the one you think is the physical die, then scroll down to see if you’re right.
If you picked the second chart as the physical die, you’re right. While shockingly unscientific, this first test suggests the app gives no worse a distribution than rolling a real, physical D20.
Pearson’s chi-squared test
The second test uses Pearson’s chi-squared test, as explained on Stack Exchange. I’m just following along with that post, so if there’s something wrong with the method, please let me know.
I rolled four times the minimum number of dice for each type. The results:
| D4 | D6 | D8 | D10 | D12 | D20 | |
|---|---|---|---|---|---|---|
| Rolls | 80 | 120 | 180 | 200 | 240 | 400 |
| Result | 0.70 | 5.90 | 10.49 | 13.00 | 13.80 | 9.50 |
Checking these against the upper-tail critical values (reproduced from the Stack Exchange post below), every result falls below the 0.90 value, which suggests the dice probably aren’t biased.
| v | 0.90 | 0.95 | 0.975 | 0.99 | 0.999 |
|---|---|---|---|---|---|
| 3 (D4) | 6.251 | 7.815 | 9.348 | 11.345 | 16.266 |
| 5 (D6) | 9.236 | 11.070 | 12.833 | 15.086 | 20.515 |
| 7 (D8) | 12.017 | 14.067 | 16.013 | 18.475 | 24.322 |
| 9 (D10) | 14.684 | 16.919 | 19.023 | 21.666 | 27.877 |
| 11 (D12) | 17.275 | 19.675 | 21.920 | 24.725 | 31.264 |
| 19 (D20) | 27.204 | 30.144 | 32.852 | 36.191 | 43.820 |
Duplicates in groups
One user emailed to say they rolled 5D20 twice: the first roll had three 10s and the second had three 3s. That surprised them (and me), and it just doesn’t seem right.
I’m not sure how to test this properly. Just for fun, I rolled 50 groups of 5D20 in the app and 50 groups with physical dice (again, I don’t know if my physical dice are fair):
| Pairs | Two pairs | Three of a kind | |
|---|---|---|---|
| App | 18 (36%) | 2 (4%) | 1 (2%) |
| Physical | 17 (34%) | 2 (4%) | 0 (0%) |
I’m not sure we can draw conclusions from this, but the app doesn’t seem to give many more duplicates than real dice.
Conclusions
The app seems to be rolling fair dice. The Pearson test suggests the dice probably aren’t biased, and comparing the app with physical dice, I don’t see anything unusual.
We humans see patterns everywhere, even when they’re created by chance. Sometimes crazy things happen because of randomness, and it’s tempting to say there’s an order or a bias to it, but often there just isn’t.
And remember: if we held a global coin-flipping championship, someone would win.