Forum    Members    Search    FAQ

Board index » Erfworld Things » Gems and Mine4Erf




Post new topic  Reply to topic  [ 9 posts ] 
 
Author Message
 Post Posted: Mon May 14, 2018 8:20 pm 
User avatar
This user posted the comment of the month Here for the 10th Anniversary Has collected at least one unit Mined 4 Erf This user got funny with a rodent This user is a Tool! Shiny Red Star Won Mine4erf for the Gobwins For when you need it most
Offline
Joined: Thu Dec 05, 2013 9:33 am
Posts: 1285
The Problem
I've seen it argued, repeatedly, that a single spin of 1,024 shares (1,024-in-1,024 chance) is more likely to produce a gem than 1,024 separate one-share spins (1-in-1,024 chance each, 1,024 times). I've repeatedly pointed out that the law of large numbers, more commonly called the law of averages, states that as the number of identically distributed, randomly generated variables increases, their sample mean (average) approaches their theoretical mean.

In other words, when you roll a standard six-sided die once, you get a number between and including one and six. If you averaged the numbers on the faces of that die, they average to 3.5. If you roll that die enough times and keep averaging the total result, it will over time approach and then become 3.5. That’s the law of large numbers at work.

So when I say that 1,024 one share spins are, on average, as likely to produce a gem as one 1,024 share spin... I'm talking about the law of large numbers.

For my trouble, I've been insulted, accused of incompetence, had someone try to convince me that I believe in the gambler's fallacy (that after you lose a bunch you will soon win). Most insultingly of all I was accused of not being willing to admit when I'm wrong. Me! The guy who obsesses about intellectual honesty. Of all people. Frankly... I'm pissed. And I'm dangerous when I'm pissed.

The Solution
So, rather than argue theory endlessly, I wrote a program. I simulated both a) over one billion (1,024,000,000) one-share rolls and b) one million (1,000,000) 1,024-share rolls.

It took days to fine tune the app so it wouldn't eat every bit of RAM my system has (16GB) in under a minute. The simulations took approximately 12 hours for the 1-in-1,024 x1,024,000,000 simulation, and ~3.5 hours for the 1,024-in-1,024 x 1,000,000 rolls.

Guess what? I was right. <insert two middle fingers here>

But Don't Take My Word For It
Here are the results of my test:

1-in-1024 x1024000000.txt
1024-in-1024 x1000000.txt
And again...

I did the test repeated 1,024,000,000 times for the first and 1,000,000 times for the second based on the fact that both would have an expected outcome of 1 million wins. Apples to apples.

Don't believe the app works as advertised? Try it yourself. There's a little setup involved, but there's a lot of techies in this community too. It should run on Windows and Linux (and possibly BSD and Mac as well). Think I did something funny with the code? The source is on github too for the whole world to see. I've got nothing to hide.

The tests above were done with 1.0.0-rc4, the gold version unless you guys find a bunch of bugs. 1.0.0 is now available and is essentially the same as rc4. It's actually a really simple program, considering, so I don't expect bugs to be an issue. For the more taxing tests (like the 1-in-1,024 x over a billion test) you will need at least 25GB of disk space and at least 2GB of free RAM. Fewer "iterations" mean less disk space required and smaller "chunks" mean less RAM is required. Note that smaller chunks tend to be slower, however.

The Analysis
The 1,024-in-1,024 rolls all produce exactly 1,000,000 wins each (every simulation was a win, as expected). Even after repeating the simulation a number of times, the 1-in-1,024 chance based rolls never deviated more than 953 wins from the predicted average value, and most were within 100. Now, if 953 wins seems like a lot, remember we're talking about 1,000,953 vs 1,000,000 (less than a 1% difference of the observed versus the expected). It's worth pointing out that my +953 win outlier was in the positive... as in more gems than the average. Sure, you could get 953 gems fewer than the average... but you can also get 953 more than the average.

Let's bring this back to something relevant: none of us are going to spin a billion times. At a max of 24 spins a day (for those of using our home gaming machines and presuming you’re only mining ETH and not the altcoins alongside) it would take 114 years to accumulate that many spins. Assuming Erfworld keeps the mining option with spins as it is for another 10 years, we produce an average of 20 shares every single hour, and we continue rolling for gems for those 10 years, we're all looking at about 1668.75 gems on average. Now, considering the deviation above, that means 1668.75 gems plus or minus 1.5853125 gems. That’s a difference of fewer than two gems over ten years of 24/7 spin accumulation. Two freaking gems! In ten years! And again, that's based on the largest outlier I observed (the worst case in my repeated tests).

The Exception
One of the people I debated this with who was not a douchebag pointed out that there is an exception. They pointed out that the law of large numbers can be shown 100% accurate if you allow for an infinite number of attempts on the random roll. The observed average will become the expected average in that case. The fewer rolls, the less likely the observed and expected average are to converge. As we only have so many spins a month, the odds are not 100% identical. They might be, but they don’t have to be, just like my simulation didn’t show the exact predicted number but one that was very close. Apparently, this exception happens when you reach "an end." The end can be stopping Mine4Erf altogether or it can be the end of that month's contest.

I remain dubious of this. Dubious in the same way I am dubious of the Monty Hall Paradox. And I accept the Monty Hall Paradox as correct... changing your answer increases your chance of winning. It's just proof human minds suck at understanding the world from a statistical perspective. That goes for this mess here too. It makes no sense to me. But I've seen the proof. And, though my sanity tries to slip every time I give it serious thought, I refuse to deny the evidence.

But here's the thing. It basically affects only one spin in the whole month. One. Out of ~720 (for those of us mining ETH non-stop). And the effect it has on that one spin? A fraction of a fraction of a percent change in likelihood. Your last spin is still _very_ (almost indistinguishable) close to a [number of shares]-in-1,024 chance of producing a gem.

How this could be considered statistically meaningful is another conversation I need to have. ‘Cause it really doesn't seem meaningful to me, but statistics are weird like that.

Conclusion
In conclusion, the law of large numbers holds up to empirical testing. Suprise!

One must remember that these are averages and probabilities we’re talking about. Yes, it's technically possible to spin one million times at 1,023-in-1,024 and never get a single gem. Yes, it's also possible to spin 1024 million times at 1-in-1,024 and get 1024 million gems. However, it is ridonkulously unlikely (that's ridiculous squared... for those who don't know). More than likely you would get very close to 999,023 gems in the first case, and very close to 1,000,000 gems in the second... on average... because that's how probability and averages work.

And for those who decided to be insulting assholes: How's yer crow? Tasty? No? Good. :lol:

_________________
Everyone's Favorite Naughtymancer,
JadedDragoon

Fear the would-be hero. For against the pangs of conscience, there is no more effective anesthesia than a righteous cause. And no one sees themselves as the villain, especially the villain.


Last edited by JadedDragoon on Thu May 17, 2018 9:35 am, edited 1 time in total.
  • Tipped by 4 people!
  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Thu May 17, 2018 9:33 am 
    User avatar
    This user posted the comment of the month Here for the 10th Anniversary Has collected at least one unit Mined 4 Erf This user got funny with a rodent This user is a Tool! Shiny Red Star Won Mine4erf for the Gobwins For when you need it most
    Offline
    Joined: Thu Dec 05, 2013 9:33 am
    Posts: 1285
    reserved

    _________________
    Everyone's Favorite Naughtymancer,
    JadedDragoon

    Fear the would-be hero. For against the pangs of conscience, there is no more effective anesthesia than a righteous cause. And no one sees themselves as the villain, especially the villain.

  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Sat May 19, 2018 9:27 am 
    This user was a Tool before it was cool Here for the 10th Anniversary Has collected at least one unit Mined 4 Erf This user is a Tool! Won Mine4erf for the Marbits
    Offline
    Joined: Fri Sep 27, 2013 12:46 pm
    Posts: 35
    Nice job on the mathamancy. That matches my experience and what I saw when I was helping a coin farming op fine tune their rigs and pointed it towards the mining pool. I was getting anywhere between 480-640 shares per hour with about a 2-3% stale rate on the NA site. My hit ratio on Gems for those spins sat at around 58%

  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Sun May 20, 2018 10:02 am 
    User avatar
    Here for the 10th Anniversary Has collected at least one unit This user has been published! This user is a Tool! Armored Dwagon Monthly Winner Mined 4 Erf Won Mine4erf for the Gobwins
    Offline
    Joined: Wed Nov 09, 2016 3:05 pm
    Posts: 212
    Location: Philly
    I guess I don't pay too close attention to the mining forums, but I'm surprised this post even had to be made. I thought this was, like, the most basic stats&prob possible. As your link even explains (holy crap Encyclopedia Brittanica still exists?!), this was proven in 1713 and generally accepted as true well before then, likely for thousands of years (dem Greeks).

    I did see someone else's post do the Math and show that there was about a 30% chance not to get any gems doing 1024 spins of a single share each, but the important point is that there was also about a 30% chance you'd get more than one. In the end it will average out.

    ...given the chosen algorithm's approximation of true randomness.

    _________________
    The 25th Magic, in three parts

  • Tipped by 1 person!
  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Sun May 20, 2018 1:02 pm 
    User avatar
    This user posted the comment of the month Here for the 10th Anniversary Has collected at least one unit Mined 4 Erf This user got funny with a rodent This user is a Tool! Shiny Red Star Won Mine4erf for the Gobwins For when you need it most
    Offline
    Joined: Thu Dec 05, 2013 9:33 am
    Posts: 1285
    lordfisch wrote:
    I guess I don't pay too close attention to the mining forums, but I'm surprised this post even had to be made. I thought this was, like, the most basic stats&prob possible. As your link even explains (holy crap Encyclopedia Brittanica still exists?!), this was proven in 1713 and generally accepted as true well before then, likely for thousands of years (dem Greeks).

    I did see someone else's post do the Math and show that there was about a 30% chance not to get any gems doing 1024 spins of a single share each, but the important point is that there was also about a 30% chance you'd get more than one. In the end it will average out.

    ...given the chosen algorithm's approximation of true randomness.


    If it's the post I'm thinking of, they are the ones who later went on (on IRC) to equate the law of large numbers to the gambler's fallacy in a desperate attempt to prove I can't admit when I'm wrong and, ironically, succeeding in proving that they can't admit when they are wrong. ::shrug:: There's a lot of people like that out there.

    As to your comment regarding randomness... good point. I'm confident that the source of pseudorandomness in Random Sim is sufficiently random. I used a cryptographically secure pseudorandom number generator (or CSPRNG). The standard for what counts as a cryptographically secure PRNG is that the pseudorandomness it generates is functionally indistinguishable from true randomness (as much as true randomness even exists). Specifically, I used Node.js' crypto library, which is, in turn, based on OpenSSL. I was also careful to ensure that the libraries and math used would not introduce bias into the results.

    Of course, it still isn't actually random. But it is functionally as random as what Mine4Erf is using for popping gems and cards and the like. Possibly more random if the PRNG functions Erf Team used aren't cryptographically secure (which would also make it possible to predict the results of spins... so I doubt that's the case).

    _________________
    Everyone's Favorite Naughtymancer,
    JadedDragoon

    Fear the would-be hero. For against the pangs of conscience, there is no more effective anesthesia than a righteous cause. And no one sees themselves as the villain, especially the villain.


    Last edited by JadedDragoon on Sun May 20, 2018 7:52 pm, edited 1 time in total.
  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Sun May 20, 2018 7:32 pm 
    User avatar
    E is for Erfworld Supporter Year of the Dwagon Supporter Print Book 2 & Draw Book 3 Supporter This user is a Tool! Pin-up Calendar and New Art Team Supporter Mined 4 Erf Has collected at least one unit Won Mine4erf for the Gobwins Won Mine4erf for the Marbits For when you need it most
    Offline
    Joined: Sun May 17, 2009 7:30 pm
    Posts: 102
    JadedDragoon wrote:
    I was also careful to ensure that the libraries and math used would introduce bias into the results.


    It's been a long day here, but ITYM: ....would NOT introduce bias into the results.

    _________________
    Imageshack ate my old avatar from the "lost" version of Page 144. So, I picked a new avatar for the way I feel about this community.

  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Sun May 20, 2018 7:56 pm 
    User avatar
    This user posted the comment of the month Here for the 10th Anniversary Has collected at least one unit Mined 4 Erf This user got funny with a rodent This user is a Tool! Shiny Red Star Won Mine4erf for the Gobwins For when you need it most
    Offline
    Joined: Thu Dec 05, 2013 9:33 am
    Posts: 1285
    ripple wrote:
    JadedDragoon wrote:
    I was also careful to ensure that the libraries and math used would introduce bias into the results.


    It's been a long day here, but ITYM: ....would NOT introduce bias into the results.


    Lol, yes... "would not." I've made the needed correction. Thank you for pointing it out.

    For those who are curious: some methods of mathematically producing randomized integers (whole/non-decimal numbers) within a certain range can bias the results. That bias makes such randomness unusable for simulating real-world unbiased randomness or unbiased pseudorandomness. A lot of developers are not math wizards and do not realize this. I'm not a math wizard either... but I did my homework.

    _________________
    Everyone's Favorite Naughtymancer,
    JadedDragoon

    Fear the would-be hero. For against the pangs of conscience, there is no more effective anesthesia than a righteous cause. And no one sees themselves as the villain, especially the villain.

  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Sat Jun 02, 2018 9:52 am 
    User avatar
    Print Book 2 & Draw Book 3 Supporter This user is a Tool! Year of the Dwagon Supporter This user was a Tool before it was cool Pin-up Calendar and New Art Team Supporter Has collected at least one unit Erfworld Bicycle® Playing Cards supporter Mined 4 Erf For when you need it most Won Mine4erf for the Marbits
    Offline
    Joined: Fri Nov 18, 2011 10:47 pm
    Posts: 165
    lordfisch wrote:
    I guess I don't pay too close attention to the mining forums, but I'm surprised this post even had to be made. I thought this was, like, the most basic stats&prob possible.


    In my experience, when cognitive biases and statistics fight, the cognitive biases usually win, even if the person knows the statistics.

    I was having a conversation with my mother about cryptocurrencies, having made a bit on XRP. She said I should sell, locking in my profits before I lost everything. I disagreed.

    My argument is that I would never put a few thousand dollars into cryptocurrencies. It would be over-extending myself. Yet having earned a few thousand dollars by investing a few hundred, I had a few thousand dollars leveraged without over extending.

    Nothing about my investment thesis had changed: while there is a good chance of losing everything, everything I know about startups, cryptocurrencies and financial markets says the size of the market, if a success, way out performed the loss. It would be like betting a dollar to earn $10 if a dice comes up a 6. Sure, I'll lose five out of six times I play, but I will come out ahead if I play enough times.

    Hearing this, I can see my mom getting super frustrated. See, she's an accountant, and could easily follow what I was saying.

    Eventually she threw up her hands, saying "I understand the math, I still think you're wrong!"

    Those biases scream in your ear even when you know they're wrong. To be completely honest, I know where she's coming from, because it's also screaming in my head, I just had more practice ignoring it.

  • Tipped by 2 people!
  • Tip this post

    Make Anonymous
  • Top 
       
     Post Posted: Fri Jun 08, 2018 4:33 pm 
    User avatar
    This user posted the comment of the month Here for the 10th Anniversary Has collected at least one unit Mined 4 Erf This user got funny with a rodent This user is a Tool! Shiny Red Star Won Mine4erf for the Gobwins For when you need it most
    Offline
    Joined: Thu Dec 05, 2013 9:33 am
    Posts: 1285
    kiyote wrote:
    lordfisch wrote:
    I guess I don't pay too close attention to the mining forums, but I'm surprised this post even had to be made. I thought this was, like, the most basic stats&prob possible.


    In my experience, when cognitive biases and statistics fight, the cognitive biases usually win, even if the person knows the statistics.

    I was having a conversation with my mother about cryptocurrencies, having made a bit on XRP. She said I should sell, locking in my profits before I lost everything. I disagreed.

    My argument is that I would never put a few thousand dollars into cryptocurrencies. It would be over-extending myself. Yet having earned a few thousand dollars by investing a few hundred, I had a few thousand dollars leveraged without over extending.

    Nothing about my investment thesis had changed: while there is a good chance of losing everything, everything I know about startups, cryptocurrencies and financial markets says the size of the market, if a success, way out performed the loss. It would be like betting a dollar to earn $10 if a dice comes up a 6. Sure, I'll lose five out of six times I play, but I will come out ahead if I play enough times.

    Hearing this, I can see my mom getting super frustrated. See, she's an accountant, and could easily follow what I was saying.

    Eventually she threw up her hands, saying "I understand the math, I still think you're wrong!"

    Those biases scream in your ear even when you know they're wrong. To be completely honest, I know where she's coming from, because it's also screaming in my head, I just had more practice ignoring it.


    Recent research shows that you're right.

    Most of us are familiar with the idea of cognitive dissonance. The idea that holding two opposing veiws to be true at once is unpleasant for the individual doing so. The assumption, going back to the time when the term "cognitive dissonance" was coined, was that when faced with it people would consider the two competing views and, based on their individual merits, keep the better one and discard the other.

    Research shows, however, that when faced with exactly that scenario the more common outcome is the rejection of the new view in favor of the existing one regardless of their merits. Not only that... having had their existing views challenged they will then hold to their existing views even more strongly and be even more unwilling to consider opposing views in the future, again, irrespective of the merits of the views in question. This has been dubbed "the backlash effect."

    My own personal hypothesis on this is that this happens specifically because people become emotionally invested in their views and make those views part of their identity. And this is inherently dangerous because that makes any challenge to one's views a challenge to one's identity. To say that their views are wrong is to say that the type of person they are is wrong... at least subconsciously. Obviously, a person's views can be wrong without that person being a bad person... but emotions are not inherently rational or logical. Ironically, this fear of being a bad person leads to people doing bad things with complete conviction in their righteousness.

    _________________
    Everyone's Favorite Naughtymancer,
    JadedDragoon

    Fear the would-be hero. For against the pangs of conscience, there is no more effective anesthesia than a righteous cause. And no one sees themselves as the villain, especially the villain.

  • Tip this post

    Make Anonymous
  • Top 
       
    Display posts from previous:  Sort by  
     
    Post new topic  Reply to topic  [ 9 posts ] 

    Board index » Erfworld Things » Gems and Mine4Erf


    Who is online

    Users browsing this forum: No registered users and 2 guests

     
     

     
    You cannot post new topics in this forum
    You cannot reply to topics in this forum
    You cannot edit your posts in this forum
    You cannot delete your posts in this forum
    You cannot post attachments in this forum

    Search for:
    Jump to: