Heuristic Evaluation of UNITY3D

By Oga Ajima.

Unity3D is one of the more widely used game design engines for the design and development of video games. It is available for installation across multiple platforms including Microsoft Windows, Apple Mac OS X and Linux and can be used for cross-platform development of both 2D and 3D games. This makes it a very compelling option for developers looking to make video games as it allows for flexibility and freedom in regards of platforms to deploy on while eliminating elaborate modifications prior to deployment. Despite these positive features, there are multiple alternative game engines available and Unity3D is competing in a category of software applications that is experiencing rapid growth and the various options available are providing stiff competition with respect to features and capabilities. Among these alternatives include game design software such as Epic Game’s Unreal Engine, Amazon’s Lumberyard, Crytek’s Cryengine and YoYo’s GameMaker just to name a few.
A cursory glance at the history of software development reveals that particular applications eventually come to dominate any given category and examples abound in word processors (Microsoft Word), design (Adobe Photoshop) to name a few. This is due to variety of reasons and among these is the usability of the application itself. Game engines are a class of software that are in their relative infancy and prejudices formed now could probably influence user’s choice for a very long time especially considering user inertia with regards to established software. A lot of the game engines competing with Unity3D are at feature parity or getting there rapidly, and with free licensing models becoming more popular, some other form of differentiation becomes ever more important in order to stand out from the crowd.

Usability of a software application is a major differentiation factor and is widely recognized as being crucial to the user experience of an application. Since Nielsen and Molich’s original presentation of the heuristic evaluation guidelines in 1990, heuristic evaluation has become a widely used usability testing method due to its advantages with regards to cost, time and expertise. Nielsen’s revised guidelines are among the most popular of the heuristic guidelines employed but there are also other heuristic guidelines including Schneiderman’s 8 Golden Rules, Gerhardt-Powals’ principles, amongst many.

The users of the Unity3D game engine range from the absolute beginners to experts. This represents a very wide spectrum to cater to hence the need for the user interface (UI) to account for varying degrees of competence. To test the usability of the Unity UI, a heuristic evaluation was undertaken to evaluate the usability of the software. Nielsen’s 10 heuristic guidelines [1] were used together with the severity rating scale [2] (ranging from 0 – 5) to rate the severity of the problems found. The task evaluated where those involved in the creation of a Scene which include the manipulation of objects in the Scene View and the creation of programming scripts to add functionality and logic to the Scene. Creating a Scene is one of the basic task engaged in while bringing a gaming concept to life so this task is representative of what a beginner or expert user may do while using the application. The usability problems found, the severity rating and suggested solutions are presented below. The evaluation was done by the author of this post but it is accepted that more evaluators will find more problems compared with using one evaluator so it is very likely that there are more usability problems in the application than those that have been identified.

Problems

Problem - Unclear GUI icons in “Transform Tools” Toolbar

Figure 1 - Transform Tools Toolbar (located upper right)

The Transform Tools toolbar contains the icons that are used to manipulate the objects in a Scene View with regards to select, position, rotation, and scaling. These manipulations happen in a 3D field that is represented along three Cartesian axes x, y and z. There are multiple options to perform these manipulations including the use of the keyboard shortcut keys (where Q -> Pan, W -> Move, E -> Rotate, R -> Scale) and by exact values in the Inspector Window under the Transform field (where you also have the option of using the mouse to drag the values, though this is not immediately obvious). Despite these other options, the Transform Tool icons are the most obvious and I believe the most convenient as they allow immediate and rapid prototyping as different objects can be selected and the corresponding Transform icon chosen to adjust that particular option. The problem lies in knowing which icon represents which option. Beginners new to Unity are likely to make the correct association of the hand icon with the desired option (pan) which is what a UI designer wants; the other icons however are not quite so obvious and one of the causes may be a beginner user exposure to a different meaning of that icon in a different software or application. The Move icon bears some similarity to the Scale icon but most importantly is used by Google Maps to pan, by Microsoft Word to move an image, by Paint.Net to move “selected” pixels and probably in many other programs.

Figure 2 – From left to right, the four arrow Icon in Google Maps, Microsoft Word, Paint.Net and Unity3D.

This is an overly used icon and depending on the background of the user, there may already be some meaning attached to it. To compound the difficulty in ascertaining the meanings associated with the Transform Tools icons, no tooltip is provided. Tooltips are widely used help mechanism in allowing and enabling the discoverability of graphical user interface (GUI) elements. They appear or pop up when a user hovers the mouse over an icon, or UI element and provide information regarding the action(s) about the element. A convention in application development is to make tooltips and help options verbose as a default on initial installation and as users get comfortable with the application and can recognize the various GUI controls with ease, these can be turned off or the density reduced. Curiously, Unity does use tooltips for some icons but not consistently across the whole UI.

Figure 3 - Tooltip for the Gizmo Display center icon

Severity and Suggested Solution

This is a pretty serious oversight on the part of the UI designers as it leads to a lot of aggravation on the part of the user and requires memorization of the icons. One of the 10 heuristic guidelines is to choose recognition over memorization. This flaw breaks that guideline as it forces memorization upon the user, also the principle of consistency and standards is violated as it forces users to wonder what the various icons mean due to the use of these icons in different applications to represent different actions coupled with the subtle similarity between the move and scale icon. The severity rating is a 3 as this is a major problem that should be fixed.

In the long term, a better option would be to rethink the choice of icons used in the Transform Tool. Designing a completely new icon set that has no “baggage” carried from other applications would probably be a good idea. In the interim, including into the frequent update packages pushed out to users an enabling of the tooltip should be a priority. This should not be particularly difficult, as it seems the mechanism is already enabled for some icons.

Problem - Lack of a reset option for View Gizmo

The View Gizmo is used to switch between a 3D view and 2D in the Scene. It is particularly useful when the developer is creating a scene where the in-game perspective may shift between 2D and 3D views. It gives the developer greater visibility into how things appear in-game and provides greater design control over a particular scene.

Figure 4 - View Gizmo (3D view)

The default view of the View Gizmo is from a 3D perspective and the problem arises when the developer tries to revert back to this default view after switching to a 2D view. There is no obvious way to do this as repeatedly clicking on the gizmo continuously switches between the various axes (x, y, z) within a 2D context.

Figure 5 - View Gizmo (2D view)

Pausing the mouse cursor over the gizmo produces no indication of how to revert back to a 3D view. An extreme step is to save and close the Scene View and then reopen the same scene and the default 3D view will be restored; obviously, this is not a desired step as it interrupts the work flow and can be very annoying. There is actually a pretty simple way to revert back to the default 3D view – “Shift + Click” restores the 3D view – but this can only be discovered on going through the documentation for the application. While documentation is important, for simple tasks such as this, there should be an obvious way to revert back to the default state without resorting to the documentation. The User Control and Freedom heuristic guideline suggests that users “will need a clearly marked emergency exit to leave the unwanted state” and this has not been provided with respect to the View Gizmo.

Severity and Suggested Solution

While the search for how to revert back to the default view can be frustrating, the command to do so is not a difficult one to recall (Shift + Click) and the main problem lies in it not being easily discoverable so the severity rating has been assigned a 2, it is a minor problem that should be relatively easy to fix.

As has been observed, the fix should be easy to fix and this can be accomplished by the use of a tooltip to provide discoverability. Implementing this shouldn’t be too difficult since as noted previously, certain icons have tooltips enabled so there is likely a framework that handles the tooltip component and adding the View Gizmo may just be another object inheriting from the class. An alternative is to popup a tooltip whenever the view is changed with the command displayed, this can be turned off later when the user no longer requires it.

Problem – Context Switch when Editing a Script

Unity3D provides the ability to add functionality through the use of programming script. It supports the use of C#, JavaScript (UnityScript) and Boo. By providing this functionality, complex logic and interaction requirements can be built into a game which makes this a very powerful feature. In order to add a script, the user has to create the script file within unity and after it has been created, the user then selects the script and chooses the edit command and this is where the flexibility and efficiency of use guideline is violated. Unity3D by default makes use of an external editor (MonoDevelop in OSX and Visual Studio in Windows) and in addition to waiting for a couple of seconds while the external program loads, the user is thrown into a completely different environment; this is a jarring switch in the GUI presented and while there are advantages to using a full integrated development environment (IDE), from a beginner perspective, it can be rather overwhelming; also conventional wisdom for beginner programmers tend to recommend the use of a basic text editor when learning a language as this helps in tracing the execution of a program [1].

Severity and Suggested Solution

This is a relatively minor problem as it is applicable mostly to those that are complete beginners to the Unity3D application and to programming in general and it is assigned a severity of one. That said, the experience of waiting for a couple of seconds with no indication of what is happening and then being thrown into a completely different environment could be an unpleasant one and is one that can be easily mitigated by providing a simple text editor as the default script editor within Unity3D itself rather than relying on an external application as the default. While a simple text editor will not have the bells and whistles of a full IDE, it nevertheless gives the user the appearance of control and keeps him within the confines of a familiar environment. As the user gets more experience in both using Unity3D and programming in general, the choice to configure the preferred IDE can then be made.

Problem – Inconsistent GUI Choice (Inspector Window)

Within the Inspector Window, there are multiple fields that have various attributes that provide for options to be selected. Unity3D uses a double arrow icon in addition to other UI conventions to indicate that more options can be selected for a particular attribute.

Figure 6 - More Option dialog in Inspector Window

This is a common usage of this icon and is immediately recognizable for the affordance it provides. The problem however lies in the circular icon that provides a similar functionality.

Figure 7 - A different icon that allow selection of more options

Clicking on this icon (it is not immediately obvious that it is indeed clickable) results into a dialog box that pops up and in most cases in a completely different location.

In this evaluators view, this is a completely unnecessary implementation of a “more option” dialog as there is no advantage by using a different GUI icon and action to achieve a similar functionality especially when the arrow key icons have already been used and in addition, it results into an inconvenient implementation as the cursor has to be moved across the screen in order to make the selection. This violates the consistency and standards guideline which suggests preventing users having to wonder whether actions mean the same thing.

Severity and Suggested Solutions

This is more of a baffling and inconvenient problem and has been assigned a severity rating of 1 as it is mostly a cosmetic problem; it can be addressed when there is say a major overhaul in the UI of the application and simply choosing one of the icons is sufficient as they both require a new dialog box which can be designed to accommodate all the required options. Adding search functionality to the attributes can also be implemented as this then reduces the amount of time searching for an option when the name is known and may be appreciated by more expert users.

Problem – GUI Aesthetic and Color Choice

There seems to be a lot going on in the GUI especially for a scene with a multitude of models and having the Inspector Window open. Aesthetic decisions tend to be subjective but this “busyness” in conjunction with the default color choice of the GUI (different shades of grey), results into a punishingly drab interface. It violates the aesthetic and minimalist design guideline with regards to the aesthetics but as noted, this is largely subjective.

Figure 8 - GUI of Unity3D

Severity and Suggested Solution

This is not a particularly serious problem and may not even be considered a usability problem. It has been assigned a severity of 1 as it is mostly cosmetic. That said, the choice of making the dark theme GUI a premium option seems to be misguided and serves no purpose except maybe to annoy free users in the hope of convincing them to upgrade to a paid version. This is a risky decision, as a user may feel revolted enough by the drab interface and decide to explore other options that provide an interface more to the users taste. The best solution to this would be to make all GUI options available irrespective of the subscription tier and build in customizability of the interface through the use of themes as this will allow the users to express their individuality and can act as a point of differentiation.

Conclusion

Unity3D is a fairly feature-rich and capable application that has been continuously improved since its debut in 2005. Its popularity is a testament to its strength and capability and maintaining its position as the leading game design engine means that it has to address every facet of the application where there might be weaknesses. While most of the problems identified are not severe, Paul Boag makes a good argument in [4] that minor issues can add up and do interfere with the user experience. Addressing these issues can help in preventing users from switching Unity3D to other applications

References

[1]	J. Nielsen, "10 Usability Heuristics for User Interface Design," NN/g Nielsen Norman Group, 1 January 1995. [Online]. Available: https://www.nngroup.com/articles/ten-usability-heuristics/. [Accessed 31 March 2016].
[2]	J. Nielsen, "Severity Ratings for Usability Problems," NN/g Nielsen Norman Group, 1 January 1995. [Online]. Available: https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/. [Accessed 31 March 2016].
[3]	B. MacDonald, "Radar - To IDE or Not to IDE?," O'Reilly, 23 January 2014. [Online]. Available: radar.oreilly.com/2014/01/to-ide-or-not-to-ide.html. [Accessed 31 March 2016].
[4]	P. Boag, "Bad user interface design is death by a thousand cuts," boagworks, 3 February 2015. [Online]. Available: https://boagworld.com/usability/bad-user-interface-design-is-death-by-a-thousand-cuts/. [Accessed 31 March 2016].