Saturday, March 26, 2016

Duolingo iOS App: Heuristic Evaluation

By/ Obai Mandorah 

  •   Introduction
     Apple announced Duolingo as the app of the year in 2013 [1]. Duolingo is language learning app with high rating review in app store (4.5 out of 5). The mobile app provides online learning for many languages in fun, engaging, and motivating forms. The app uses many combinations of pictures, pronunciation coach, typing, listening, tests, etc. I followed Jakob Nielsen’s 10 Usability Heuristics to evaluate this app along with Severity Ratings for Usability Problems [2], [3], as the following 0 to 4 rating scale can be used to rate the severity of usability problems:
0 = I don't agree that this is a usability problem at all
1 = Cosmetic problem only: need not be fixed unless extra time is available on project
2 = Minor usability problem: fixing this should be given low priority
3 = Major usability problem: important to fix, so should be given high priority
4 = Usability catastrophe: imperative to fix this before product can be released
I used the app version 4.5.9 on iPhone 6 running iOS 9.2.1.

  •   Beginning setup:

Problem 1: Redundant notification
The first thing encounter the user is a pop up notification asking for a permission to use the notification center in the phone. Once you click OK, another pop up notification asking you whether allowing or not the app to send you notifications.

Severity rating: 1
Heuristic violated: Error prevention
     The redundancy of the notification is confusing. It should have been the second notification only. There is no need for the first one. This is a minor issue since it appears only once (when opening the app for the first time), it is not a persistence problem where it pops up each time you open the app.

Problem 2: Different goal’s measurements

Severity rating: 1
Heuristic violated: Consistency and standards  
     When using the app for the first time, you must select one of the 16 languages (the list is updateable). Then pick a goal (one of four levels). After that, select whether you are a beginner or not. The first two selections can be changed later on. when selecting the goal at the setup page (Pic.1) it differentiates the difficulty level by time (how many minutes per day), but when you try to change that goal later (Pic.2) it differentiates the difficulty by XP points per day. I’m assuming the XP refers to experience points. Since users’ learnability differs from one to another and some of them can take more or less time than the picked goal, the XP values seems to be more achievable goal than sitting a goal by time.

  •   Skills selection menu:
            Each course in Duolingo (showed in the circles below) grouped of lessons. The gold bars show the completed lessons, while the grey ones show the uncompleted ones or the lessons that have been taken long time ago. Between each group of skills there are tests of the uncompleted skills if the user passed these tests, the next group of skills well be unlocked.

Problem 3: Delete a language from the courses

Severity ranking: 2
Heuristic violated: User control and freedom
      If the user added the wrong language, or has a change of mind and decided to remove a language from the course list, this option is not available via this app. To delete a language from the list, the user must open the desktop webpage and dig into the menu in order to remove it from the courses list. It is an annoying problem that when you add a language by mistake then decided to remove it, you have to login into your account (in the full website version) and dig in the setting to correct a simple mistake. This option should be available in the app.

Problem 4: Inconsistent page naming

Severity rating: 2
Heuristic violated: Consistency and standards
     When a user wants to go to the settings, he must go through the profile page first then go to settings page. This conflicts with common sense. If a user need to change some of the app settings, there should be a clear button in the main page, not hidden under another page. Furthermore, from the setting page you can go to profile settings. Due to its persistency, this issue is there until the user remember the path of the settings he wants to configure during time.

  •   Lessons selection menu:
Offers multiple-level of lessons depends on previous ones. Each lesson has a verity of exercises.

Problem 5: Lesson description 

Severity rating: 0
Heuristic violated: Recognition rather than recall
     In this page, the user moves around the lessons by swiping. In order to compare the keywords for each lesson, the user has to briefly memorize what’s each page has before swiping to the next page. This is not a big problem since the user going to take the lessons by order. Also, swiping is a convenient way to switch between lessons in the same skill, rather than changing the pages.

  •   Lesson screen:
This is the core of the app, where the user’s most interactions with the app in order to learn a new language.
Problem 6: Misleading pictures

Severity rating: 3
Heuristic violated: Match between the system and the real world
     From the pictures above, the icons can be misleading. For example, on the left picture, the sentence mentions a black shirt, but there is no black shirt in the picture. On the right picture, the sentence mentions an apple, but there is no apple!  For beginners, this could lead to a lot of mistakes. Due to it’s persistence and frequency, it is a serious problem. However, the user can report the issue from the same screen to address the problem in order to fix it in the future.

Problem 7: Unsaved progress

Severity rating: 2
Heuristic violated: User control and freedom
     When a user starts a lesson and decides to close it before finishing it, there is no saving option and all the progress during the session will be lost when quitting. The same situation face the user at the test. Therefore, the only way to save any progress is to finish the lesson or the test.
The frequency of this situation gives it a high impact on user’s score, and learnability. Since this is a mobile app, users tend to use it on the go, and usually there is no specific location or time during the day to practice. The only choice is to leave the app running in hope to pick it up at the point it left, or close it and lose all the progress in the session.

Problem 8: Daily goal vs lesson XP
Severity rating: 2
Heuristic violated: Recognition rather than recall
     After finishing the lesson, a greeting page will show whether the user met the daily goal or not. However, it only shows the earned XP from the last lesson and it doesn’t show the accumulated XP from the same day nor the daily goal. The persistence of this problem is what gives it this rating, but when the user used to the system and remember the goal, this becomes very minor issue.

  •   Profile screen:
This screen has many features; adding the user profile picture, changing the learning language, editing the daily goal, showing progress diagram, and sharing progress.

Problem 9: Errors with Facebook add

Severity rating: 3
Heuristic violated: Error prevention
     When clicking on the add button at the weekly Leaderboard, then chose to Find friends on Facebook, if the user did not enter the Facebook credentials, the Facebook page will close and a pop up message appears to notify the user that the connection failed. After closing that pop up message, the Facebook page appears again, after closing that page an error message partially pop up at upper left corner on the screen. This situation is fairly rare, but still annoying when happening with multiple errors appearing consequentially.

Problem 10: Hidden daily goal
Severity rating: 1
Heuristic violated: Recognition rather than recall
     Edit goal is in the profile screen, however, it doesn’t show the current goal. The user has either to remember the goal or click on the button to see what the current daily goal is. The daily goal is represented in the progress graph as a pointed green line, but there is no clear indication that is the goal. Therefore, most of the users will click on the edit goal button to see what is the set goal, but not very frequently.

  •   Settings screen:
Provide the user the ability to edit the personal information. In addition of controlling the app accessibility such as, connection to Facebook and/or google plus, and enabling/editing the notification reminder.

Problem 11: contents order
Severity rating: 1
Heuristic violated: Aesthetic and minimalist design
     The overall design doesn’t violate this heuristic. However, the order of the contents is inconvenient. For example, Send Feedback and Sign Out buttons are in the middle of the page, while notifications controls are in the bottom of the page. Usually, the Sign out and the Feedback buttons are at the bottom of the page. It is a minor problem, but when a user wants to change the time of the daily reminder, being located at the end of the settings page is not an easy find.

  •   Shop screen:
            It is where the user can redeem the Lingots coins with some add-ons which complement the learning procedure.

Problem 12: Unclear bonus metrics

Severity rating: 2
Heuristic violated: Recognition rather than recall
     The metric used to gain the Lingots is not clear. Furthermore, when buying a bonus skill, it doesn’t show where is the location of that skill. The user must close the shop screen and look in the main screen for the bought item. . If a user buys multiple items from the shop, it is not easy to remember all the redeemed stuff. Therefore, the impact can be big since the new item may not be used duo to the lack of recognition.


1 comment: