Monday, June 3, 2013

Even More Betting Strategies at the DotA 2 Lounge

My previous post, Betting Strategies at the DotA 2 Lounge, is to date the most popular post I have ever made to this blog.  It's popular enough that I decided that I can do better!  There were some unanswered questions in the original post:  What are the most overrated and underrated teams?  Are people who bet rares more prudent with their bets than uncommons or commons?  Would the inclusion of all of the historical data from the DotA 2 Lounge affect my results at all?  I scraped all of the data from the first 400 matches at the DotA 2 Lounge using wget in a shell script, and threw out all matches where the bets were either cancelled or where the data was incomplete.  This left me with data on 334 historical matches from which to draw conclusions, including the odds on commons, uncommons and rares separately.  Basically, this analysis will give far better confidence than my previous one, and allow me to ask questions that I was previously unable to.

Let's start again by examining the crowd favor of the winning teams.  Just as in Figure 1 of my previous post, I expect to see this plot skewed to the right if the betting crowd has any ability at all to pick the winning team.

 Figure 1:  Histogram of crowd favor of the winning team.  Clearly, the crowd usually favors the winning team (everything to the right of x=0.5 on the graph) and chooses correctly about 2/3 of the time.

Indeed, Figure 1 shows that the crowd usually picks correctly and favors the winning team about 2/3 of the time.  Remember, though, that the goal of the gambler is not to pick correctly:  the goal is to profit!  First, as in my previous post, I need a control group to compare everything else to.  I will start by getting a handle on how well I will do over time if I flip a coin and bet randomly.  I will put 1 common, uncommon or rare on the team determined by the coin flip and chart my behavior.

 Figure 2a:  Number of commons won by random coin flip in the first 334 matches at the DotA 2 Lounge.  Average = -2.7 items, Standard deviation = 19.6 items.
 Figure 2b:  Number of uncommons won by random coin flip in the first 334 matches at the DotA 2 Lounge.  Average = -4.0 items,  Standard deviation = 19.4 items.
 Figure 2c:  Number of rares won by random coin flip in the first 334 matches at the DotA 2 Lounge.  Average = -3.4, Standard deviation = 19.6 items.

Contrary to my original study with a smaller data set, Figure 2 gives a more flattering view of the wisdom of the crowd.  Betting randomly appears to be a bad idea, paying out at about 0.99:1 over 334 matches.  This is basically the behavior we'd expect in a system where the entire betting public has good information on what's going on.

Let's find out what happens if I bet 1 common, 1 uncommon, and 1 rare on the crowd favorite based on the payout per item for every match.  If there is no crowd favorite (1:1 odds), then I will abstain from betting.  I will track the behavior of my winnings over the 334 matches that had complete data and that were not canceled.

 Figure 3a:  Behavior of betting 1 common on the crowd favorite for each of the first 334 complete, non-canceled matches.
 Figure 3b:  Behavior of betting 1 uncommon on the crowd favorite for each of the first 334 complete, non-canceled matches.
 Figure 3c:  Behavior of betting 1 rare on the crowd favorite for each of the first 334 complete, non-canceled matches.
Figure 3 shows the behavior of a hypothetical scheme where I bet 1 common, 1 uncommon, and 1 rare on the crowd favorite of each match through the history of the DotA 2 Lounge.  My hypothetical winnings fluctuate around 0 and never come close to the 40-item mark that is necessary in order to reach statistical significance.  This is another mark of a rational crowd:  going with the crowd neither makes you significant gains nor losses over time.

Now let's try the same experiment, except this time I'm always going to bet against the crowd.  Is this a winning strategy?

 Figure 4a:  Behavior of betting 1 common on the underdog for each of the first 334 complete, non-canceled matches.
 Figure 4b:  Behavior of betting 1 uncommon on the underdog for each of the first 334 complete, non-canceled matches.
 Figure 4c:  Behavior of betting 1 rare on the underdog for each of the first 334 complete, non-canceled matches.
Figure 4 shows that betting on the underdog is also not a great long-term strategy.  Like betting with the crowd, it never approaches the 40-item mark we need in order to reach statistical significance.  So, it seems that the crowd is actually pretty decent at picking the proper odds of winning at the DotA 2 Lounge after all.  Reassuringly, I never indicated that betting with the crowd or against it was a statistically significant improvement over random betting in my previous post.  There is a claim, however, that was statistically significant:  that betting for the left column is a losing strategy over time, and betting for the right column is a winning strategy over time.  In my previous study, these actually were at the border of statistical significance.  Given the larger data set, will these assertions be supported?

 Figure 5a:  Behavior of always betting 1 common on the team in the left column.
 Figure 5b:  Behavior of always betting 1 common on the team in the right column.
 Figure 5c:  Behavior of always betting 1 uncommon on the team in the left column.
 Figure 5d:  Behavior of always betting 1 uncommon on the team in the right column.
 Figure 5e:  Behavior of always betting 1 rare on the team in the left column.
 Figure 5f:  Behavior of always betting 1 rare on the team in the right column.

Figure 5 does seem to show a sustained trend towards gamblers betting favorably towards the team in the left column, and indeed there seem to be times when this could be construed as significant.  In particular, the peak around match 180 in 5b, 5d and 5f seems to be fairly significant given the number of matches played thus far.  Fast forward to today, though, and the crowd has since rectified its irrational ways.  Whether you're betting commons, uncommons or rares in the right-column strategy, you are nowhere near statistical significance today.

I plan on following up with another post on the most overrated and underrated teams, but first I want to correct my conclusions from my original post.  Upon gathering more data from the DotA 2 Lounge, my previous conclusions are definitely affected:  betting for the right column may have been a statistically significant good strategy in the past, but it is not anymore.   The strategies of flipping a coin, always betting with the crowd, and always betting for the underdog all remain statistically insignificant.

I definitely made an amateur mistake in overstating the trends I did find.  It is not correct to advocate a strategy (such as flipping a coin or betting for the underdog) that is not statistically significant--I've learned from this mistake, and will not repeat it in the future!

If you'd like to gather your own data from the DotA 2 Lounge and perform your own analysis, I've made the scripts I used available.  Please feel free to use the data you gather for any purpose you like.  If you decide to make your own blog post about the trends you find, let me know and I'll link you from here!

Update:  If you enjoyed this post, you may also like my more recent post: What is the Most Underrated DotA 2 Team?

1. It may also be worth looking at how many people are betting on various games and how that effects outcome.

right now there are 3 games up for betting, The Alliance game has 21 000 people betting. The other two have just 6 000.

I wonder how the volume affects the accuracy of the outcome?

1. That's an interesting question! I'll look into it for my follow-up post.

2. I think the idea is that:

"Is the group more accurate in predicting winners when they are larger?"

Another question I think about is:

"Are underdogs more successful in best of 1 matches?"

My theory is that skill needs time to have an effect. I can beat Roger Federer in Tennis if we only have to play one ball and I get lucky. But the longer the game goes the more his edge (skill) has an effect on the outcome.

I feel that betting on underdogs in best of 1 matches where they can get lucky (enemy team tries new strat, lag issues or server issues affect outcome, etc) is smarter than betting on underdogs in a best of 3 where the more skilled team has a chance to let their edge come into play.

A case in point is last nights game with Empire vs DD. Empire are 20-80. Complete underdogs and rightfully so.
BUT
The game is best of 1.
There is a new patch.
The game is not being played live and one team is in Russia and another Europe. (so server lag plays a big part)
My odds are insane if I bet on Empire. So betting 4 rares to win close to 20 there is probably a fantastic bet.

As it turns out the DD player loses connection and they have to play 4v5. This is exactly the opportunity that I am referring to. In a Best of 1 you can have the skill edge completely mitigated by an outside factor.

So that turned out to be a great bet. Obviously Empire threw the game and I spazzed out but the point is that if the group gets the ratio correct they do so in a vacuum. So I would love your opinion on how the number of games in a match affects outcomes.

tx for the posts enjoying them.

1. Wow!! dats a good thought!! I always sucked at betting .. I usually used to bet on teams with higher percentage!! No wonder i have 0 rares!! can u lemme knw ur steam Id??!! i just ask for suggestions!! lost 2 pages of uncommons cause i bet on Na'vI at Alienware cup!!
http://i.imgur.com/H4CwKsq.jpg

may be a little help :) I wuld like to hear your deep thoughts about dis betting system

3. How about playing for the opposite choice of the crowd only in the indecisive crowd matches and betting for the choice of the crowd when there is significant difference in the choice? I mean we have to select a line for the indecisive e.g. like around %35 x %65 and bet for the %35. When distance of the percentages is higher than this we play for the choice of the crowd. Is calculating this does any good ?

4. In the past, the return on rares for the unfavored team was always higher than uncommon and uncommon always higher than common, but that appears to have changed significantly recently, but I still suspect there's some correlation to team popularity and irrationality.

5. Nice article! I was planning to do similar analysis. This saved me the trouble :)
So have you made any progress in analyzing individual teams? I think there's much to gain by finding out the underrated teams and overrated teams. Thanks!

6. Hi Ben, just saw your article and I wanted to download dota2 match results through R, and wondering if you knew which website to download from or if you have code in R or python that downloads the results? Alternatively if there is a way to download results, a blog post on this would be very kind of you.

Thanks,
Mike

1. Hi Mike--

Right now, I don't have any R or Python to prepare the requisite data, but I do have bash scripts. You want the script "study.sh" from the link at the end of my post, which should run natively on Mac, Linux or Windows under Cygwin.

~Ben

2. Could you give more specific instructions on how to run these scripts? I'm having trouble because I'm not familiar with Cygwin nor programming =/

Thanks

7. any update on this?