Usability in Video Games and Heuristic Evaluation of DotA2

Written by: Omais Rasool Khan

Usability in Video Games

In this post I’ll talk about usability in video games, and I will present to you my evaluation results of one of the most popular games, Defense of the Ancients 2 or DotA2.

When we hear the phrase ‘usability of something’, what comes to our minds is to what extent is this ‘something’ usable? This is the linguistic definition of usability. In terms of software, it simply is a measure of effectiveness, efficiency and satisfaction with which the user can perform tasks when using a software. So in order for a software system to be optimal, it must satisfy usability requirements. Since we are talking about video games here, does this definition apply to video games? Let’s look at some examples from video games. [2]

Most of us have played the Mario Series by Nintendo. In this, or any game in general, the player must complete levels to advance. This can be attributed to a task, which by definition of usability, must be done efficiently, effectively and satisfactorily. But all Mario players know this isn’t the case in the game. Nintendo games are known for their difficulty and to complete Mario levels are really hard and player fails dozens of time or even more before completing a level, which makes it a frustrating experience. But we all know how outstanding the Mario series is. On the contrary, the difficulty is the attraction of the game and gamers love it.

I have no idea how to clear this level

Let’s look at one more example. You all must have played, or heard of, the resident evil franchise. Yes, I’m talking about that game which gave us nightmares. If you have played the game, you must have noticed that the game, through low lighting and camera angles, manipulates the view of the player in such a way that an enemy (a zombie in this case) is difficult to see and pops up suddenly. This was purposely done to give us jump scares. From the usability point of view, these ‘Zombies’ are an obstacle in the game which the player has to eliminate in order to complete the task. So instead of player completing the task efficiently and effectively, he faces obstacles that makes it difficult for him to complete the task. So does this usability problem make the game bad? Well the ratings of the game says the opposite. [2]

It’s dark. Watch out for that Zombie!

To conclude what I have said until now, to measure usability in games, efficiency and effectiveness are secondary, and user satisfaction is primary. So if the task is difficult for the player, but provides satisfaction, then it’s not much of an issue. The reason behind this is usability enhances the productivity of the user. But games are not primarily productive, rather they are a distraction for the user. The user deliberately face obstacles to gain satisfaction. [2]

Now that we are clear about usability in video games, let’s look at some of the methods we can use to test usability in video games.

Video game developers mostly use ‘Play Test’ to measure usability of their games. It is a method by which the designers expose the game to players to detect bugs and flaws in the game. This method is expensive as it requires a selected number of users, a lab in which the player can play, and a usability tester who can evaluate player’s responses. The player’s performance is measure on some protocols. As this method can be costly, small game studios find it difficult to implement it. One other method used to test usability is the heuristic evaluation.

Heuristic evaluation is simply evaluating a software based on defined rules accepted by the scientific community. For example we have Jakob Nielsen’s heuristic evaluation for user interface. And the user can evaluate the software on its own by applying these heuristics. But as usability is measured differently in video games, different set of heuristics must be applied. In the literature there are many heuristics available for video games. But the one that I find most satisfactory is Heather Desurvire and Charlotte Wiberg’s "Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games”.

In the rest of this post I will mentioned the results which I obtained by applying the PLAY heuristics on one of the most played game today ‘DotA2’. One important thing to mention, PLAY doesn’t specifically measures usability, but user experience overall. The difference between usability and user experience is: usability measures how effectively, efficiently, satisfactorily a user achieves a specific goal; while user experience is about the overall user’s experience with the product. So usability is specific to a task, user experience is about all aspects of a product.

Heuristic Evaluation of DotA2

To keep this reading concise and to the point, I’ll only mention only those heuristics which were not satisfied (according to me) by the game. If you want full details on PLAY heuristics, you can follow [1] in the references.

So before we start let me give you a brief background of DotA2. DotA2 is a sequel of DotA, and DotA itself is a modification or custom map of the famous game Warcraft 3. DotA2 falls under the genre of Massive Online Battle Arena (MOBA). And main feature that define this genre is competitiveness. A picture below shows some in-game information.

In-game interface of DotA2

As you can see in the picture, the in-game interface shows a lot of information to the player. There is a mini-map on the bottom left corner, at the bottom-center there is player’s hero info (abilities/health/mana), bottom right shows player gold, top-right shows score, and top-center shows info of other player’s heroes. The rest of the screen is the battle area and shows where the player is currently looking in the entire map.

Being a veteran of this game (I’ve been playing the game for about 10 years now), I felt it a personal responsibility to evaluate the game myself. Enough with the introduction, lets jump into the action. If you’re curious about the game, I suggest you to play it. There’s a reason it is one of the most played games in the world.

Here is the format I’ll follow in the rest of this post. First I’ll mention in the heading the name of the heuristic from PLAY which I conclude is not satisfied. Then I’ll give a description of how it is violated. Remember! PLAY talks about user experience in general and only some points may refer to usability.

Problem 1: Challenges are positive game experiences, rather than negative experiences, resulting in wanting to play more, rather than quitting. [1]

The game provides a ranking system also known as Matchmaking Rating (MMR) which ranks players according to their wins and losses and helps determine the skill level of the player. This is also one of the highest motivating factor for DotA 2 players to play the game as this develops competition. But this also becomes the most demotivating factor when the player loses MMR by losing the game, or doesn’t reach his/her expected MMR even after spending months playing the game. Look at me for example. My MMR is currently around 3700, and for a long time I have been trying to reach 4000. Recently I finally reached the 4000 mark, but due to consecutive loses, I dropped down again to where I was before. Imagine 10 years playing the game and making no progress. I feel like a loser. A picture is shown below.

My MMR at 4k

 My MMR reduced down to 3.7k, again!

Therefore according to the heuristic, the game should provide positive experience for the players and encourage them to play more rather than frustration or wanting to quit.

Problem 2: The game gives rewards that immerse the player more deeply in the game by increasing their capabilities, capacity or for example, expanding their ability to customize. [1]

Oh the avatars. We all love to personalize our avatars. In DotA2, the players receive items which they can use to customize their characters. The items are categorized from common to rare, mythical and Arcana items (Common being least interesting and arcana being highest of value). An image of different avatars of same heroes is shown below.

 

ITEM SETS FOR A CHARACTER IN DOTA2: 1. COMMON, 2. RARE, 3. ARCANA

Don’t you just love that arcana set in the last! So the game host certain special events at different times of the year which allows players to accumulate points and get these awesome items in return. But recently, the game has been cancelling these events for god knows what reasons. See the image below.



Frostivus cancelled! How should we get our rewards now?

These events were one of the best ways the players can be rewarded with precious items. So doesn’t this violate the heuristic mentioned above? I think it does.

Problem 3: Player does not need to access the tutorial in order to play and Players should be given context sensitive help while playing so that they are not stuck and need to rely on a manual for help. [1]

Okay let’s be honest. Who likes to read long manuals or go through tutorials before playing a game? Most likely not many of us. On the contrary we like to jump into the game and learn on the way. This is not the case with this game. Although this game offers the player option to directly play the game, it really isn’t much of an option as the player offers no in-game help. Now myself as a veteran of this game, I can assure you that you won’t be able to do anything in the game unless you go through the tutorial first. Here is what the tutorial looks like.

This is one long tutorial

As you can see, you get to play the game with the real player at stage 6 of the tutorial. So to me it clearly not satisfies the heuristic mention above.

Problem 4: Player error is avoided & Player interruption is supported, so that players can easily turn the game on and off and be able to save the games in different states. [1]

“The players can easily turn the game on and off”, if only this were true, my mom wouldn’t have to yell at me. This turning the game on and off at will or saving the game doesn’t seem to apply to DotA2; and it shouldn’t be also. As DotA2 is an online game, pausing the game or leaving in between ruins the experience for other players. Although it doesn’t follow the heuristic, we can’t blame the game for this. For the former point of avoiding player error, see the image below.

Player can accidentally buy or resell wrong items

So from the image you can see that there a lot of items a player can buy from the side. These items are so tightly packed together that there is a high chance the player will buy the wrong item. What happens then is a timer starts to run. If in this timer the player resells his item, he will get full price back. Otherwise there is a 50% penalty on the price. What I’m about to mention now is something which has happened to me personally. I bought the wrong item, now I know the timer is running and I’m in a state of panic that I have to sell the item as quickly as possible. And in this panic, I sold the wrong item from my inventory. So it is safe to say that player error to some extent is not avoided. Which dissatisfies the heuristic above.

To reduce the chances of error happening here, I would suggest to increase item size so that it becomes easy to click on the intended item. The game can also increase the timer to prevent player from panicking. Or shops can be made ‘safe zones’ where player can buy the item and resell at full price as long as he/she is in this safe zone. When he leaves this zone, the price get reduced to 50%.

To Conclude

Congrats to you all you have reached the end of this post. To all the stuff I mentioned in this post, I will honestly say that if the issues mentioned in DotA2 remains, it still won’t affect the ratings of the game. Yes some users must be annoyed by these issues but the game is still a hit and people passionately play the game. I believe the reason behind this is despite the issues, the game is successful in providing an immersive and amazing experience to the player, thereby providing satisfaction. And this makes every other issue secondary.

References

[1] Heather Desurvire1, Charlotte Wiberg. "Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration ". A.A. Ozok and P. Zaphiris (Eds.): Online Communities, LNCS 5621, pp. 557–566, 2009. © Springer-Verlag Berlin Heidelberg 2009

[2] David Buchheit, “The ergonomics of video games - Part 1: késako?” and “The ergonomics of video games - Part 2: How to evaluate it?”. In Ludic Universe, LudoTIC, January 2013