JavaScript is required to use Bungie.net

Destiny

Discuss all things Destiny.
Edited by BeegieB: 2/19/2015 8:58:47 PM
16

Don't blame Xur for being a dick. Blame randomness. A statistical analysis in the randomness in uniform distributions.

This is going to be a TL;DR, and a little stats heavy. So, yeah, it's gonna be ignored ;-) I'll cut to the chase with the end results: You can't say Xur is being a dick because he's selling Sunbreaker's for the 5th time. Why? Because you don't account for randomness. Just because some items appear to be sold more than others, does not imply that Xur is more likely to sell certain items than others. Below, I present two distinct statistical tests that both come to the same conclusion: there is not enough supporting evidence to say Xur favours selling sunbreakers over any other Warlock exotics. This same analysis can be applied to other class items and the weapons Xur sells. If you manage to get to the end, I give you a gold star. If you manage to contribute to the discussion, two gold stars. If you just want to be a prick or cry, well, it is bungie.net after all, that's the norm. ---- Being a guy who works in statistics, hell, I'll go as far as to say, I LOVE statistics, I can't help but notice a lot of people make very false claims regarding the randomness in the game. You often hear claims like: "Crota has a higher chance of dropping no land beyond than anything else", "Xur is selling crappy items on purpose", blah blah. The point here is, a lot of people don't understand that pure, uniform randomness can give rise to the appearance of 'certain things happening more frequently than others'. When you consider your small sample size (how many exotics have you gotten in your gameplay? 30? 40? Still a small sample size bruv), and the relatively small pool of exotics in the game, you can see how it can happen. I will illustrate by example. A statistical exploration on the likelihood that Xur will have sold 'sunbreaker' gauntlets 5 out of the 22 weeks Xur has come to the tower. We want to ask the question: is Xur more likely to sell sunbreakers than other Warlock exoitcs? But, since we're scientists, we also need to have a control, the null hypothesis. This null hypothesis is: "Xur has equal chance to sell any exoitc". Now what we want to do, is look at the historical data and try to figure out, does Xur indeed seem to have a higher tendency to sell Sunbreakers, or is he just using a RNG and we're all just getting screwed over by randomness? Here's the setup: Using [url=http://ca.ign.com/wikis/destiny/Xur_Visit_History]IGN's xur item history[/url] as the data set, we're going to explore two things. First, we'll look at the likelihood that Sunbreakers are indeed more likely to be sold by Xur than other exotic warlock gear. Second, we'll assuming that each exotic has equal chance to be sold by Xur, and then look at how likely it is that Xur would sell Sunbreakers 5 / 22 weeks. Before we begin though, we need to set ourselves a significance threshold (we call them p-value's in statistics). What this means in very simple terms is: if the results we look at are above the significance threshold, then we can't make any concrete claims about our hypothesis, in statistical parlance, we fail to reject the null hypothesis. If we are below the significance threshold, then we can say that we have rejected the null hypothesis (and there may be merit to your hypothesis). What's a good significance value? Well, in industry and science, a good ad-hoc choice is 1 in a million (0.000001). But let's be generous here, and say our significance threshold is 1/20... or 5%. So basically, if we can show that Xur selling Sunbreakers is less than 5% likely, we'll reject the null - that items are equally likely to be sold by Xur. If we can't show this, then we can't with high certainty say that this isn't all just random. So, lets concretely define our Null Hypothesis. That all warlock items have equal chance to be sold by Xur. This is pretty easy to figure out. There are currently 9 exotic warlock items in the game. So, if we assume that each has equal chance of dropping, the probability that sunbreakers will drop is 1/9, or 11% What is our hypothesis that we want to study? (The alternate hypothesis). That Xur has a higher tendency to sell sun-breakers over other warlock exotics. [b]1. Are Sunbreakers more likely to be sold by Xur?[/b] For this, I will employ a statistical test called a proportion test. Since this isn't a math class, I won't get into the details of how this test is performed. The idea is this: you look at the statistical distribution of your observed data, and try to see how likely it is that it could be explained by a distribution given by your null hypothesis. If the p-value falls below our significance threshold, we can say that we have rejected the null hypothesis, ie, it is unlikely that what we have observed can be explained away by randomness alone. For this test, I'm using the prop.test tool in [url=http://www.r-project.org/]R[/url], which is a language and tool used frequently for statistical computing. This tool just takes the proportion of successes, in this case, the number of times sunbreakers have been sold (5), and the total number of trials (22). It also takes a value p, which is the probability of the null hypothesis (1/9). Here are the raw results: [quote] > prop.test(5, 22, p=1/9, alternative = 'greater') 1-sample proportions test with continuity correction data: 5 out of 22, null probability 1/9 X-squared = 1.9446, df = 1, p-value = 0.08158 alternative hypothesis: true p is greater than 0.1111111 [/quote] The value of interest here is the p-value of 8.1%. This is higher than our significance level, and hence, we have failed to reject the null hypothesis. Thus, we can't say with much confidence that Xur is more likely to sell Sunbreakers than any other Warlock exotic. [b]2. How likely is it that Xur will have sold Sunbreakers 5 times?[/b] For this test, we're going to take the analogy of coin flipping. We're going to assume that our coin has a 1/9 chance of coming up heads (Xur sells Sunbreakers), and a 8/9 chance of coming up tails (He sells something else). Then we'll flip the coin 22 times, and see how likely it is that we get 5 heads out of 22 flips. For this test, we'll employ Bernoulli Trials. Basically all a Bernoulli Trial is, we flip a coin a bunch of times, and look at the results. The resulting probability distribution is called a Binomial Distribution, it is well defined, and gives us exact probability values for the quantities we are looking for. Again, I'm going to use R for this so I don't have to use a calculator. The specific function used is called dbinom, which takes 3 values, the number of heads (5), the total number of trials (22), and the probability of getting a heads (1/9). The output is the probability that you can expect to see heads 5 times in 22 coin flips with a probability of heads of 1/9. The raw results are below: [quote] > dbinom(5, 22, 1/9) [1] 0.06021593 [/quote] In this case, the probability of getting 5 heads out of 22 coin flips with a probability of heads of 1/9 is 6%. Again, this is higher than our significance threshold, and we cannot say with confidence that Xur has a tendency to favour Sunbreakers over the other Warlock exotics. --- As a word of caution, I took a simplifying assumption to make the analysis easier. I didn't take into account that until ~December, there were only 6 exotics available for Warlock, so the probability of dropping Sunbreakers under the Null Hypothesis then was 1/6. But this will only make my argument stronger, not weaker. I just wanted to keep it as simple as possible. So if anyone wants to quote this analysis, be aware that it's not 100% correct, but it DOES get the point across. --- Phew ... aren't we all glad that's over? What can we take away from this? 1. Randomness is tricky, and it's difficult to make claims about the randomness without a lot of data 2. We CAN use data to tease out the true nature of the randomness 3. Science is -blam!-ing hard! (1 in a million for a significance value is incredibly small, you need to be very certain about your claims if you want to be taken seriously in science) --- If you managed to get here ... cheers. Feel free to comment, ask me questions, or provide any criticism you may have (unconstructive criticism is ignored, I already know I'm a nerd). If you see anything wrong with this analysis, definitely let me know, we all make mistakes and I am always looking to learn to become a better statistician and scientist, I have taken pains to make sure there are no errors in the analysis, but sometimes shit happens.

Posting in language:

 

Play nice. Take a minute to review our Code of Conduct before submitting your post. Cancel Edit Create Fireteam Post

View Entire Topic
You are not allowed to view this content.
;
preload icon
preload icon
preload icon