JavaScript is required to use Bungie.net

Destiny

Discuss all things Destiny.
Edited by BeegieB: 2/19/2015 8:58:47 PM
16

Don't blame Xur for being a dick. Blame randomness. A statistical analysis in the randomness in uniform distributions.

This is going to be a TL;DR, and a little stats heavy. So, yeah, it's gonna be ignored ;-) I'll cut to the chase with the end results: You can't say Xur is being a dick because he's selling Sunbreaker's for the 5th time. Why? Because you don't account for randomness. Just because some items appear to be sold more than others, does not imply that Xur is more likely to sell certain items than others. Below, I present two distinct statistical tests that both come to the same conclusion: there is not enough supporting evidence to say Xur favours selling sunbreakers over any other Warlock exotics. This same analysis can be applied to other class items and the weapons Xur sells. If you manage to get to the end, I give you a gold star. If you manage to contribute to the discussion, two gold stars. If you just want to be a prick or cry, well, it is bungie.net after all, that's the norm. ---- Being a guy who works in statistics, hell, I'll go as far as to say, I LOVE statistics, I can't help but notice a lot of people make very false claims regarding the randomness in the game. You often hear claims like: "Crota has a higher chance of dropping no land beyond than anything else", "Xur is selling crappy items on purpose", blah blah. The point here is, a lot of people don't understand that pure, uniform randomness can give rise to the appearance of 'certain things happening more frequently than others'. When you consider your small sample size (how many exotics have you gotten in your gameplay? 30? 40? Still a small sample size bruv), and the relatively small pool of exotics in the game, you can see how it can happen. I will illustrate by example. A statistical exploration on the likelihood that Xur will have sold 'sunbreaker' gauntlets 5 out of the 22 weeks Xur has come to the tower. We want to ask the question: is Xur more likely to sell sunbreakers than other Warlock exoitcs? But, since we're scientists, we also need to have a control, the null hypothesis. This null hypothesis is: "Xur has equal chance to sell any exoitc". Now what we want to do, is look at the historical data and try to figure out, does Xur indeed seem to have a higher tendency to sell Sunbreakers, or is he just using a RNG and we're all just getting screwed over by randomness? Here's the setup: Using [url=http://ca.ign.com/wikis/destiny/Xur_Visit_History]IGN's xur item history[/url] as the data set, we're going to explore two things. First, we'll look at the likelihood that Sunbreakers are indeed more likely to be sold by Xur than other exotic warlock gear. Second, we'll assuming that each exotic has equal chance to be sold by Xur, and then look at how likely it is that Xur would sell Sunbreakers 5 / 22 weeks. Before we begin though, we need to set ourselves a significance threshold (we call them p-value's in statistics). What this means in very simple terms is: if the results we look at are above the significance threshold, then we can't make any concrete claims about our hypothesis, in statistical parlance, we fail to reject the null hypothesis. If we are below the significance threshold, then we can say that we have rejected the null hypothesis (and there may be merit to your hypothesis). What's a good significance value? Well, in industry and science, a good ad-hoc choice is 1 in a million (0.000001). But let's be generous here, and say our significance threshold is 1/20... or 5%. So basically, if we can show that Xur selling Sunbreakers is less than 5% likely, we'll reject the null - that items are equally likely to be sold by Xur. If we can't show this, then we can't with high certainty say that this isn't all just random. So, lets concretely define our Null Hypothesis. That all warlock items have equal chance to be sold by Xur. This is pretty easy to figure out. There are currently 9 exotic warlock items in the game. So, if we assume that each has equal chance of dropping, the probability that sunbreakers will drop is 1/9, or 11% What is our hypothesis that we want to study? (The alternate hypothesis). That Xur has a higher tendency to sell sun-breakers over other warlock exotics. [b]1. Are Sunbreakers more likely to be sold by Xur?[/b] For this, I will employ a statistical test called a proportion test. Since this isn't a math class, I won't get into the details of how this test is performed. The idea is this: you look at the statistical distribution of your observed data, and try to see how likely it is that it could be explained by a distribution given by your null hypothesis. If the p-value falls below our significance threshold, we can say that we have rejected the null hypothesis, ie, it is unlikely that what we have observed can be explained away by randomness alone. For this test, I'm using the prop.test tool in [url=http://www.r-project.org/]R[/url], which is a language and tool used frequently for statistical computing. This tool just takes the proportion of successes, in this case, the number of times sunbreakers have been sold (5), and the total number of trials (22). It also takes a value p, which is the probability of the null hypothesis (1/9). Here are the raw results: [quote] > prop.test(5, 22, p=1/9, alternative = 'greater') 1-sample proportions test with continuity correction data: 5 out of 22, null probability 1/9 X-squared = 1.9446, df = 1, p-value = 0.08158 alternative hypothesis: true p is greater than 0.1111111 [/quote] The value of interest here is the p-value of 8.1%. This is higher than our significance level, and hence, we have failed to reject the null hypothesis. Thus, we can't say with much confidence that Xur is more likely to sell Sunbreakers than any other Warlock exotic. [b]2. How likely is it that Xur will have sold Sunbreakers 5 times?[/b] For this test, we're going to take the analogy of coin flipping. We're going to assume that our coin has a 1/9 chance of coming up heads (Xur sells Sunbreakers), and a 8/9 chance of coming up tails (He sells something else). Then we'll flip the coin 22 times, and see how likely it is that we get 5 heads out of 22 flips. For this test, we'll employ Bernoulli Trials. Basically all a Bernoulli Trial is, we flip a coin a bunch of times, and look at the results. The resulting probability distribution is called a Binomial Distribution, it is well defined, and gives us exact probability values for the quantities we are looking for. Again, I'm going to use R for this so I don't have to use a calculator. The specific function used is called dbinom, which takes 3 values, the number of heads (5), the total number of trials (22), and the probability of getting a heads (1/9). The output is the probability that you can expect to see heads 5 times in 22 coin flips with a probability of heads of 1/9. The raw results are below: [quote] > dbinom(5, 22, 1/9) [1] 0.06021593 [/quote] In this case, the probability of getting 5 heads out of 22 coin flips with a probability of heads of 1/9 is 6%. Again, this is higher than our significance threshold, and we cannot say with confidence that Xur has a tendency to favour Sunbreakers over the other Warlock exotics. --- As a word of caution, I took a simplifying assumption to make the analysis easier. I didn't take into account that until ~December, there were only 6 exotics available for Warlock, so the probability of dropping Sunbreakers under the Null Hypothesis then was 1/6. But this will only make my argument stronger, not weaker. I just wanted to keep it as simple as possible. So if anyone wants to quote this analysis, be aware that it's not 100% correct, but it DOES get the point across. --- Phew ... aren't we all glad that's over? What can we take away from this? 1. Randomness is tricky, and it's difficult to make claims about the randomness without a lot of data 2. We CAN use data to tease out the true nature of the randomness 3. Science is -blam!-ing hard! (1 in a million for a significance value is incredibly small, you need to be very certain about your claims if you want to be taken seriously in science) --- If you managed to get here ... cheers. Feel free to comment, ask me questions, or provide any criticism you may have (unconstructive criticism is ignored, I already know I'm a nerd). If you see anything wrong with this analysis, definitely let me know, we all make mistakes and I am always looking to learn to become a better statistician and scientist, I have taken pains to make sure there are no errors in the analysis, but sometimes shit happens.

Posting in language:

 

Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Thanks for bringing a solid contribution to this forum. The randomness of the rewards system in this game is exactly what makes it so great. Does this mean I can stop sacrificing my fellow guardians to the Traveller and teabagging exotic chests too?

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Thanks for taking the time.

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Basically. Not Xur fault. RNG sucks sometimes.... For those of you who skipped the article lol.

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • We know that Bungie employees control what's for sale, or at least can change it. DeeJ let that slip a few weeks ago in a video.

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • My question is what are the chances that on the dark below weekend what is the chance that it would have had all dark below hear like it did. Adding a few exotics in each category and hitting all of the numbers needed seems like bungie can control him for special events but then make him random again

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • I couldn't read all of it but I'll say this; just because it is random doesn't mean it isn't weighted. Here's what I'm saying: xur is controlled by RNG but that doesn't mean everything is set equal to each other. For an example Let's say numbers 1-50 will make xur sell plan c, 51-60 will sell ghorn, and 61-100 will be truth. There is a much lower chance for ghorn than the other exotics even though it is still random.

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

  • Question... Sort of. Heart of praxic fire was sold one time by xur. Assuming a fair coin, or equal rng table, what are the odds in 22 weeks that xur will sell that item only once? We now have nine items to choose from, so we'll just use that as our base. HoPF has a 1/9 chance to drop, which means the odds of it NOT dropping are 8/9. Spread over 22 weeks, this is 8/9 * 8/9 etc. to 22 times. So (8^22)/(9^22) = .0749, Or 7.5% Every week this continues the odds get worse. What we have here is crazy. The odds of getting sunbreakers 5 times is about 6%, and the odds of getting only one HoPF drop is 8%. So my question is, how can both low probability occurrences be actually happening in a fairly weighted RNG? I am not arguing against a the existence of an RNG system, I just believe certain items are weighted to occur more or less often. Perhaps, though we only have nine exotics, the program picks from twenty tokens. Sunbreakers and voidfang vestments have multiple tokens in the jar and HoPF has a single token. This, in my mind, would explain the low odds that we see occurring today.

    Posting in language:

     

    Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

    5 Replies
    • I tried to read it all but was short for time. Did you broach the subject of how computers cannot truly provide RNG because they are executing an algorithm?

      Posting in language:

       

      Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      1 Reply
      • Very well-reasoned. There simply isn't enough data to come to any kind of conclusion about xûr. It's also funny to remind ourselves that it's completely irrelevant, since he's going to sell what he's going to sell, random or not.

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      • TLDR Xur is not random. Watch when you see him sell all dark below gear this weekend.

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      • Wonderful! I'm not nearly into stats as much as this, but I am aware of p-values and chi-squared and all that jazz. I did my own analysis using some web tools and came to similar conclusions. Nice job bringing it to everyone.

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      • Good work man, lets keep bumping.

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      • I'm really surprised this thread hasn't gotten more attention. Have you done analysis on anything besides sunbreakers? I'm no stat guru but would a run test be possible with such a small sample?

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      • Ah, I see.

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

      • Too long for me to stay interested, but yeah, I've taken basic programming classes, and I've always known ways that they could easily program the rng, but that's not to say that they can't still just lie to us and change it in the end lol

        Posting in language:

         

        Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

        2 Replies
        • Lol quite an,education in stats. Dude get your life and ignore these complainers. I'm one of em,sometimes but not often

          Posting in language:

           

          Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

          1 Reply
          You are not allowed to view this content.
          ;
          preload icon
          preload icon
          preload icon