Betting Strategies at the DotA 2 Lounge

[Update:  I have revisited these conclusions with more data and better data in a new post!  Long story short:  don't rely on these suggestions!]

Lately a friend and I have been betting items for DotA 2 at the DotA 2 Lounge.  Having something riding on a game adds an element of excitement that isn't there otherwise--you personally, have something on the line and a reason to root for your favorite team.  Further, I've always found parimutuel betting pretty interesting.  Assuming zero house take and a crowd with perfect information, then the only way to reliably win in the long term is to identify irrationality in the crowd and exploit it.  Playing to simply win is an amateur mistake:  playing to maximize profit is the correct approach.  Using a collection of 190 historical data points from past matches at the DotA 2 Lounge scraped from the internet archive, I will attempt to identify some irrationalities in the DotA 2 fan-base.  If any strategy I propose turns out to be successful, then it can be said that the underlying strategy corresponds exactly to an irrationality in the betting public!

Figure 1:  A histogram of the proportion of bets by value in favor of the winning team on the DotA 2 Lounge.    Bars on the left side represent cases where the underdog won (and represent a correspondingly better payout), and bars to the right side represent cases where the crowd favorite won.  The crowd picks the correct winner about 2/3 of the time.

Figure 1 shows a simple histogram:  the distribution of bets in favor of the winning team.  I expect this plot to be skewed towards the right if the crowd has any ability at all to pick a winner.  Indeed, the distribution in Figure 1 is skewed right, but not very heavily.  I want to propose some cheesy betting strategies and see how well they work, but before I can discuss the significance of those strategies I need to know what the standard deviation of random strategies are.  I will run 10,000 simulations of random betting strategies (flip a coin to determine which team to bet an item for) and examine how well I would be expected to do just with dumb luck.

Figure 2:  Histogram of outcomes of 10,000 random betting strategies on my data set of 190 games at the Dota 2 Lounge.  Average = 9.7 items, Standard Deviation = 17.11 items.

It turns out that in my attempt to produce a control group, I have already identified a glaring irrationality of the crowd.  Figure 2 appears to be a complete paradox:  if you flipped a coin before every match and bet based on the outcome of the coin toss, you'd end up, on average, ahead!  This paradox is possible because real gamblers at DotA 2 Lounge do not bet randomly nor does the crowd correctly predict the fair odds of a team winning.  If either were true, then the outcome of random betting would be centered at zero.

Let me propose two alternate betting strategies, then, in direct opposition to each other.  First, I propose, "Always bet in favor of the crowd favorite."  I will always bet one item for the crowd favorite.  If there is no crowd favorite (50-50 tie), then I abstain from betting.  It turns out that if I pursue this strategy, even though I would win in 118 of 187 matches, I would be down 14.6 items.  This is an example of the amateur mistake I mentioned earlier.  Here, I have bet to win, and in fact I did win 118 of 187 times.  I did not, however, bet with profit in mind.  My winnings were small because I had to share a smaller pool of items with a lot more people.  Betting with the crowd seems to lose in the long term at 0.92:1.  This is plausibly within statistical noise at 1.42 standard deviations from the mean.

For my second cheesy strategy I propose, "Always bet against the crowd favorite."  Again, if there is no crowd favorite, I will abstain from betting.  With this strategy, I will always bet one item for the underdog.  In this case, I win only 69 of 187 matches but I am up 33.7 items!  Even though I only rarely win in this scenario, when I do win I tend to win big!  Betting against the crowd seems to win in the long term with a payoff of 1.18:1.  Don't be the farm on this approach, though:  this is plausibly within statistical noise at 1.40 standard deviations from the mean.

So far it appears that, if anything, the crowd tends to be slightly biased towards the favorite to win.  Perhaps part of this is that people are more focused on winning than on the potential payoff and bet for the genuinely better team more frequently than the other.  

Let me try some other strategies next and further probe the depths of the crowd's irrationality.  For my third strategy, I'd like to try, "Always bet for the team in the left column."  If betters are rational and the placement of the teams in columns is random, then my net haul using this strategy should be close to zero.  In fact, it turns out that betting one item per match for 190 matches on the team in the left column would win 90 times--but I would be down 24.6 items!  Betting the left column every time is a losing strategy with a long term payoff of 0.87:1.  Remarkably, this is outside statistical noise at 2 standard deviations from the mean!  This is thus probably not a fluke--people really do tend to bet for the left column preferably!

On the other hand, my fourth strategy will be, "Always bet for the team in the right column."  Again, I would expect the net haul using this strategy to be close to zero.  This strategy blows away all of my others, winning me a whopping 43.66 items in 100 victories!  Because the number of victories is very close to the expected number of 95, this indicates that gamblers are strongly biased to favor the team in the left column versus the team in the right column!  Betting the right column every time is a winning strategy with a long term payoff of 1.23:1.  This is barely inside statistical noise at 1.98 standard deviations from the mean.

In conclusion, the gamblers at the DotA 2 Lounge do seem to have an exploitable irrationality:  people tend to vote too much for the left column.  The best simple strategy is to always bet for the right hand column to win--a strategy that does significantly better than flipping a coin!  Beware, though:  if many people read this article and use these techniques, then these strategies may stop working entirely.  My results are based on historical data, and if the crowd is able to identify its fallibility based on this information and adapt then an entirely new strategy will be necessary to win!

Good luck with your wagers, and most importantly enjoy the game!

[Update:  I have revisited these conclusions with more data and better data in a new post!  Long story short:  don't rely on them!]

A Breast Cancer Survival Modeling Competition

California Whipsnake

This week, a paper on breast cancer survival modeling that I co-authored was published in PLoS Computational Biology.  Using a pool of patient information including clinical covariates, gene expression and copy number variation data for a collection of breast cancers we each attempted to produce the best predictive model of breast cancer survival that we could.  Our results and code were available in real time on a competition-style leaderboard that was automatically updated as we submitted new models to the competition.  In the end, we did achieve predictive models using the genetic data which performed statistically significantly better than the clinical-only models.  This is actually a fairly remarkable feat, since the genetic data was quite noisy.  In fact, while signals abound in the genetic data, actually adding this data to a clinical model tended to confuse the modeling algorithms and hurt our predictive power as often as it helped.

In the end, while our models were statistically significantly better, I don't believe that the difference was of much practical significance.  More worthwhile is knowing which signals actually improved our predictive power--these may be worthy of further investigation to better explain the correlations between the feature and cancer survival.

I'm not going to regurgitate the paper which is freely available via the link above, but I do want to highlight a few points that I believe are important and of general interest:

1. Random Survival Forest is the best out-of-the-box survival model we found.  After testing this on some other data sets, I believe that Random Survival Forest may in fact be your best bet if you are doing survival modeling.
2. A leaderboard or real-time model evaluation system is an enormous motivator in one's research.  The fact that you can quickly tweak a model and have it evaluated and placed next to your others for comparison takes much of the grind out of research.
3. Competitions are not an inexpensive option to hiring personnel or doing your own research.  The computational overhead, tech support, and manpower to procure the data, produce the evaluation system, advertise the challenge, police the participants, and evaluate their submissions requires a substantial amount of manpower and expense.