Heuristic evaluation of Web-Outlook and Gmail 

 Jagadeesh B.Hooli

1. Introduction

The main objective of any software system to make it a usable product available in the market. In the past we have encountered many circumstances where software products developed with cutting edge technologies, offering high performance but may still annoy the end users. The reason is that those software products are either never undergone any kind of usability evaluation or subjected to poor quality of usability evaluation. Usability evaluation is performed either through heuristic evaluation or usability testing. Heuristic evaluation involves usability evaluation conducted by according to guidelines termed as heuristics by certain set of expert users. In this article, using Jakob Nielsen’s ten popular heuristics listed in the below section, heuristic evaluation has been performed on two popular web-mailboxes: Outlook and Gmail. Gmail is used by all students and Outlook is mostly used by faculty members in University of Ottawa for managing e-mail communications almost every day. The frequent use of these two software products motivated author to perform heuristics evaluation on them. 

Heuristic #	Description
1	Visibility of system status
2	Match between system and the real world
3	User control and freedom
4	Consistency and standards
5	Error prevention
6	Recognition rather than recall
7	Flexibility and efficiency of use
8	Aesthetic and minimalist design
9	Help users recognize, diagnose, and recover from errors
10	Help and documentation

Table 1.1: List of ten heuristics defined by Jakob Nielsen 

2.   Product and Evaluator information

Two web-mailboxes: Outlook 2010 (Developed by Microsoft) and Gmail (Developed by Google) are tested on Windows 10 operating system and Google chrome web-browser (version no: 49.0.2623.87). These two mailboxes are web-applications which allows the users to manage e-mails. Both of them are available as open source and licensed version. In this article, both the mailboxes used are licensed. I have evaluated these mailboxes using heuristics defined in the above table. Currently, I am graduate student in computer science. I have been using these two products extensively from last two years.

3.    Usability issues and recommendations

3.1. No status of process involved in sending an email.

As part of e-mail delivery process user should be notified about the status of the mail sent by him/her. However Outlook fails to do so. It will automatically close the current window and take the user to home-page once the user hits the send button. As a result user will never be assured that whether the e-mail has been sent to the specified person or not. This problem violates the heuristic # 1 which says that user should be always notified about the things happening in the system. Since Outlook fails to acknowledge the user about the current status of email leading to major usability problem. On the other hand, Gmail solves this problem by using two notifications, one to show that the mail is in pending status and another to notify the user about the successful delivery of the email. This is shown in the below Figure 3.1

Figure 3.1: Status messages depicted by Gmail.

Recommendation

Outlook has to implement the status notification to the users like Gmail in order to abide by heuristic # 1. At the same time Gmail can improvise the current feature by displaying fractional status indicator which shows the current status and time. This will be really useful while sending the big messages ( requiring more time to deliver) so that user will be notified about the percentage of message delivered and how much amount of time is still required to deliver remaining message

3.2.  Hassle involved in the reply e-mail.

Gmail doesn’t provide a direct option to edit the subject, signature and also doesn’t even display the e-mail id of the recipient of the reply e-mail. A sample reply window is depicted in the below Figure 3.2(a). This violates the heuristic #6 as the user has to remember the options to choose while replying in order to edit the subject of the e-mail, changing/deleting signature mainly during informal conversation. Also user should recall the e-mail id of the recipient for verification since it only displays the user name of the recipient. This might create a problem especially when the user is replying for multiple conversations having people with similar names. Also Gmail doesn’t allow you to open multiple reply windows there by limiting the user to complete the current reply e-mail before switching to another conversation. This problem violates the heuristic #7 since it minimizes the user flexibility. However, Outlook doesn’t have any of such problems while replying in the mail chain as it provides plain and simple interface.  A sample interface has been depicted in the Figure 3.2(b):

Figure 3.2(a):  Window showing the hassle in the reply e-mail.

Figure 3.2(b): Simple reply interface offered by Outlook

Recommendation

Like Gmail, Outlook should provide a simple user interface displaying the subject, signature and e-mail Id of the recipient while replying for a mail in the conversation. This avoids the unnecessary hassle involved in the process. Also Gmail should provide interface which supports multiple conversation replies at a time.

3.3.  Unknown storage space.

In order to find out storage space consumed by the user in Outlook, he/she has to hover over the topmost folder of the mailbox. Below Figure 3.3(a) illustrates this action. This is non-recognizable for novice or even average users. So user may never delete the less important mails from inbox resulting into consuming lot of available free space. Once system reaches maximum storage size capacity then system will notify user to delete some e-mails. So as a result it becomes difficult for the user to decide the unimportant mails out of heap mails present in the inbox all of sudden and might disturb the user’s other priority tasks. So this violates the heuristics #1 as user was kept blind about the storage status. Also it violates the heuristics #6 too as the user has to remember the option to check the storage space explicitly resulting low visibility and recognition of the option offered by the system. This problem disturbs the organisation of e-mails. Gmail on the other hand displays the used space, which helps user to keep in mind about the consumed storage space. This is illustrated in the Figure 3.3(b)  

Figure 3.3(a): Finding out the storage space in Outlook

Figure 3.3(b): Gmail always displays the consumed storage space

Recommendation

Outlook should Outlook should display consumed storage space either in the bottom left corner so that it becomes visible to the user. This brings awareness among the users to regularly delete the unimportant mails. Gmail can improve current feature by adding total storage capacity information which avoids the user to remember it. 

3.4.  Search toolbar missing watermark.

Gmail doesn’t show the users that they are searching for a mail only inside the inbox or throughout (other folders like sent items) the mailbox. G-mail’s toolbar is shown in the below Figure 3.4(a). Also the visibility of option to customize the search is poor at least for a novice user. Unlike Gmail, Outlook provides watermark which depicts the section where the user is searching for an e-mail. A sample watermark explaining where the system will search for an e-mail once the user hits search button is shown in the below Figure 3.4(b).  So it violates the heuristic #2 as system doesn’t speak the user language to explain the search space of an e-mail.

Figure 3.4(a): Gmail‘s missing watermark in the search toolbar

Figure 3.4(b): Outlook’s Watermark depicting in which section the search is performed

Recommendation

Like Outlook, even Gmail should incorporate the watermark in the search toolbar which will convey the user the nature of the information of search space. Also the icon to customize the search should be highlighted in order to maximize visibility.

3.5.  Design problems of the Inbox interface.

Outlook doesn’t hide the options like move and delete even if user has not selected any e-mail. In other words delete/move button will only work if the user has chosen an e-mail before hitting those buttons. Outlook has ensured that if without selecting any e-mail if the user hits move/delete button than no action will be performed. But it has failed to hide these buttons in such a scenario as their presence is irrelevant. An example of such an interface is shown in the below Figure 3.5. This violates heuristic # 8 which states that interface should always avoid unnecessary information to provide simplistic view to the user. Gmail doesn’t have any such problems. 

Figure 3.5.: Outlook interface depicting the meaningless presence of delete/move buttons

Recommendation

Outlook should grey out the delete/move buttons if the user has not chosen (tick marked) any mail. This will avoid the confusion of their consequences and it also minimizes the irrelevant options presented to the user at the same time.

3.6.  Inconsistent contacts toolbar.

In the contacts screen, Gmail provides different toolbar as shown in the below Figure 6.3.1(a).  This is not consistent with the one present in the inbox screen which is also shown in the Figure 6.3.1(b). The evaluator understands the some of the options like spam, label, and archive becomes irrelevant in the contacts screen, but addition of new contact option, removal of delete, move makes it hard for the user to understand the changes incorporated for contacts screen

Figure 3.6(a).  : Toolbar present in the contacts screen of Gmail.

Figure 3.6(b).  : Toolbar present in the inbox screen of Gmail.

Outlook on the other hand uses metaphors in the contact screen instead of normal words as icon for delete and move options and also option to filter the data in the contacts screen is not provided. Below Figure 3.6(c) shows the contact toolbar. This may actually confuse the user between toolbar present in the inbox screen which is also shown in the Figure 3.6(d)

Figure 3.6(c): Toolbar present in the contacts screen of Outlook.

Figure 3.6(d): Toolbar present in the inbox screen of Outlook

Both these problems of Outlook and Gmail violates heuristic #4 which states that consistency should be maintained thought the system so that it avoids confusion of similar things among the users. This problem may create chaos for the user who starts using the contact screen when they are used to inbox screen

Recommendation

Gmail should provide delete/move options in the contact screen like inbox screen. Adding new contact option in the toolbar can be removed as there is already a separate dedicated button is present outside the toolbar. On the other hand in Outlook, metaphors should be replaced by normal words in the contacts like inbox screen. Also Filter button should be provided in the contacts screen so that this toolbar will be consistent with the one present in the inbox screen.

3.7. Creating new contact.

While adding new contact, both Outlook and Gmail don’t verify format of fields like e-mail, phone number, and name thereby allowing the user to store to junk information in these fields. This may harm the usability of contacts since it may contain meaningless information. So this problem violates the heuristic #9, which states that the interface should be capable of helping users to recognize and diagnose errors. Also while creating a new contact Gmail doesn’t provide contact button which may confuse the user who is looking to cancel the new contact creation process. This can be depicted in the below Figure 3.7. This violates the heuristic #3, which demands that the system should provide control to perform the emergency exit.    

Figure 3.7: Gmail interface to create a Contact missing cancel button

Recommendation

Both Outlook and Gmail should verify the format of fields like phone no, e-mail id, name in the create contact so that user will enter correct values into these fields instead of storing meaningless information. Gmail should also provide cancel button in the create contact screen so that user can cancel while in the process if he/she desired to do so.

3.8. Bad placement of sign-out button.

In Gmail to log out from the mailbox user has to click the profile link (usually a profile picture) then the sign-out button will be visible. This is shown in the Figure 3.8. However, for novice users it is very difficult to figure out the sign-out option even though it is the most important option everybody should be aware of. This violates the heuristic #6, which states that users should be able to recognize items rather than remembering them. This problem demands the user to recall the steps to log out from the mailbox rather than users being able to recognize by themselves. Also this adds unnecessary burden on the user’s memory. Outlook doesn’t have this problem as it provides clearly visible sign-out button at the top right corner which is the ideal position for log-out position for any application.

Figure 3.8: sign-out process of Gmail

Recommendation

Gmail should move the existing sign-out button to top right corner and make it always available without need of any mouse-clicks to log-out from the mailbox. This improves the visibility of sign-out option and also reduces extra effort users have to invest in order to logout from the mailbox.

4.    Conclusion

Outlook and Gmail being the popular web-mailbox choices have attracted users over the years through performance and flexibility. However, they still fall short when it comes to provide ideal user experience. In this article, some of the usability issues with high severity has been discussed along with suggested fixes. By addressing these issues Outlook and Gmail can provide better quality of user experience to their current and future users.