Image courtesy of Riot Games.
“Information is a source of learning. But unless it is organized, processed, and available to the right people in a format for decision making, it is a burden, not a benefit.” – William Pollard, Physicyst
In traditional sports, data analytics has seen a surge in popularity over the last decade. Most are familiar with the story of the book “Moneyball”, in which Michael Lewis portrays the Oakland Athletics baseball team and how their general manager at the time, Billy Beane, constrained by a limited budget, shifted the focus of assembling a team to a more analytical approach that was based on hard data to evaluate prospects. Since then, more and more teams have shifted resources to analytics department. While some, like the Houston Rockets (NBA) under Daryl Morey, have fully embraced the advanced statistical-based analysis, others have their doubts about its value. The reasons for that doubt are multi-fold. Statistics are sometimes frowned upon by executives and coaches who think along the lines of “these people play around with numbers, but don’t know much about the game, hence they cannot provide any value,” or “they let the numbers distract them from what is important.”
I think there are three main issues that explain the different opinions: 1. Stats people often times are bad at explaining themselves in a manner that is understandable to non-stats people, 2. Some team officials are arrogant and not open to these new — completely different — methods and 3. Statistical analysts too often do a bad job of using the data and are overconfident and lazy in their conclusions, which leads to understandable mistrust.
Still, the development over the years has been positive and given that computer games take place in a digital sphere, there is a lot of data available and it is being used more and more. Websites like Oracle’s Elixir, Games of Legends and our own League of Analytics have brought forward a lot of data and — along with more writers making use of the stats — are pushing the boundaries of data analytics for fans in League of Legends. While a lot of the metrics help viewers to better understand the game, their use for professional teams themselves is not quite as clear. Most rely on their own, more qualitative analysis — mainly VOD reviews — to gain insights. I personally think there is a lot of value in hard data, but the application is difficult and must be done with a healthy regard for context and what I would call common sense.
To gain more insight on the teams’ perspective, I sat down with one of EU’s top coaches, H2k’s Neil ‘pr0lly’ Hammad, after their victory over the Unicorns of Love at the EU LCS finals last weekend. What he had to say is very revealing and echoes some of the points mentioned above.
“Sometimes the numbers are deceiving”
“Unfortunately, it is not as easy as ‘you can look at numbers and it tells you stuff’.”
Asked how he and the team make use of statistics and hard data, Hammad highlights some of the limitations: “It is definitely a mix, just because the game is so riddled with mistakes. Sometimes the numbers are deceiving. They could be doing something because they are making this mistake and then they, like, learn. So if they did something 60 or 70% of the time, that could be a mistake that they just didn’t understand then, so now you’re rolling with a probability that they won’t do, because they learned from their mistake.” He says applying the numbers directly is tricky and that “there is a lot more analysis that needs to go on, to see what they’re thinking and then you can use the numbers once you kind of understand them a bit. […]. Unfortunately, it is not as easy as ‘you can look at numbers and it tells you stuff’.” This statement makes a lot of sense and I agree that the stats in isolation can be more misleading than helpful. It comes down to the application and context. Hammad’s conclusion? While the moto “numbers don’t lie” is generally correct, you should also apply the opposite, “numbers don’t tell the truth”, as well and “you’ll be a lot more successful when dealing with analysis […].” This might seem like a contradiction, I think it captures the very essence of data analysis pretty well. “As long as you apply ‘these numbers might not mean anything’, so you have to see what they show and then kind of dig for the meaning, I think it is really easy to make pretty much everything work.” Yes, the numbers are correct, but that doesn’t mean they cannot be misleading. What you do with the numbers matters mightily and a wrong application can be counterproductive.
“[…] I feel like people, once they found the numbers, they stop doing the work”
Sadly, the wrong application seems to be the norm rather than the exception. According to H2k’s coach, finding a good analyst is difficult and before he found his, the ones he talked to did not convince him. He says they “spoke in absolutes with probabilities” and that if someone is doing something 70% of the time, it does not mean that it is what they are going to do next time. “70% could have been just ‘this happened to be happening because of how the enemy team played’.” While this statement is definitely correct, it falls upon the analyst to make use of the numbers and apply them in combination with other information. Watch games, talk to the coach, put stats in context and then decide whether they can be of use or not. Often times, this is not how the data people approach it. Hammad critiques: “[…] I feel like people, once they found the numbers, they stop doing the work. They are like ‘I found the number so I am done’. It is really hard for me to judge what can be useful and what can’t, because I just see so many people do it really lazy,” but that “theoretically it should be a really effective kind of thing.” He just hasn’t had someone who did a good job of using the data and as a result rightfully fells like the ones who should know are “not using it right”. He sees the possible applications, but isn’t convinced by what people are currently offering.
So how does ‘pr0lly’ use the data, if at all?
“[…] I really have to write it down myself or see a picture to really get a feeling for it.”
Hammad likes to see, rather than being told. “I have a very, like, kinesthetic learning style […], but I really have to write it down myself or see a picture to really get a feeling for it. When I see a team and I hear they warded this like 5000 times, if someone just shows me a picture like ‘this is where they ward’, it is much more efficient for me, so I don’t dig analytics like that, it doesn’t convince me as easy as a picture would, I will just forget it very quickly.” This should serve as a reminder of the importance of visual analysis and presentation.
In H2k’s head analyst Michael “Vereran” Archer, it looks like Hammad has found someone he trusts, but it does not sound as if advanced data analytics is a big part of the job. “[…] (I) sometimes am like ‘’hey, can you find this percentage for me or (show me) a warding heat map?’”
“I think that is also an important thing, you can’t have one person doing that kind of job, because it is really easy to trick yourself.”
Talking to Hammad — himself a math major — you do see that, despite his reservations, he has a knack for an analytical approach (he is a coach after all) and he definitely has the mind of a very analytical person. The way he thinks about the topic, the layers of his critique and the thought process behind it are revealing of someone who has thought about the nuances of data analysis and is very aware of its many pitfalls, but also its potential. Like when he says: “I think that is also an important thing, you can’t have one person doing that kind of job, because it is really easy to trick yourself. You just sit with a number and look at it and you are gonna convince yourself that this is right. So I think it is really important to have a second pair of eyes.” Data is tricky and mistakes and biases are hard to avoid and Hammad seems to be very aware of that. There is a reason why good data analysts — those who know the theory, know the data, but are also good at putting it into context and explaining it to others — are hard to come by.
I do think one way statistics can provide value without doing much harm is by making other analysis more efficient. For teams, there is only so many man-hours you can dedicate to watching VODs of the opposition and it is impossible to find every important aspect. Scanning the data to look for clues can be a great way to improve that process. Hammad agrees and puts forward a good example: “Yeah that makes sense. That could also help […] in standard lanes, if some certain lane is always ahead, it could be they are a good laner or because they put jungle pressure there.” When you see that one laner is always ahead, you cannot conclude that he is a god in lane, but he might be. And if he is not known for his strong laning you might be onto new information. Or not. But it is a hint and after finding that piece of info in the data, go check it out and find out more.
What does the future hold?
“Right now teams really have like a set playstyle and not many teams are extremely flexible.”
Hammad admits that teams are behind on what can be done with analytics and numbers. What does he think the future holds? “I assume in the future it will be applied correctly as in like ‘we know where the enemy jungler wants to go like 70-75% of the time with a large probability’ and then not only knowing that and being accurate with that probability, but then being able to translate that into a coaching environment where the coach now teaches the team how to either counter play […] or come up with a strategy where you just avoid top pressure and then you play that style against this team. […] I feel like that would be the evolution of the game where if the jungler always goes there, you won’t just let it happen. […]. That would be the result of using the analysis correctly.” But he does think that it is a long way until such thinking will take hold and actually be applied, because “it is really hard to get that correct analysis and then right now it is pretty close to impossible to convince a team off of probability that they should play this way. Right now teams really have like a set playstyle and not many teams are extremely flexible.” Currently, even if teams had access to more in-depth data, according to him, they would not base a strategy on that.
While a lot of this conversation confirms my initial critique of “bad application”, the last paragraph really highlights the interplay of data people often times being bad at explaining themselves or not being thorough enough and of people on the other side not being open to this data driven approach. Despite all this, there are huge opportunities in the future and a lot of people need to do a better job. Data analysts need to be more transparent and look beyond the number crunching to make themselves useful and teams need to learn to be more open to the possibilities and not give up because they haven’t found the right people yet. I do think that those involved in eSports teams are more likely to be favorable to this than people in traditional sports initially where, so the development may be more rapid in League of Legends. But right now, Hammad says that teams will not come up with a strategy based on these insights, “because it is not really worth the time because no one is convinced that it’s real.”
Eike ‘Timbolt’ Heimpel is a co-founder of League of Analytics and frequent contributor. You can follow him on twitter.
All images are taken from here and are courtesy of Riot Games.