From Writing Aids to Creative Collaboration: An UX Evaluation of ChatGPT Across User Groups

By Beril Borali

How does the educational background shape user experience with conversational AI?

Our study, inspired by the study of Skjuve and others, set out to explore how users from varied educational backgrounds engage with ChatGPT (version 3.5), focusing on their behavior and satisfaction levels. The participant pool consisted of 9 university students, representing a range of disciplines including Computer Science and Engineering (CSE) and other disciplines (non-CSE) included Social Sciences, Health, and Professional Studies.

Two User Groups, One Tool

In this study, participants were grouped to engage in different variations of writing and editing activities using ChatGPT, as well as a collaborative writing task. Our study used a method that blends structured surveys with live, interactive sessions, providing a comprehensive view of how people from various academic backgrounds use and respond to ChatGPT's capabilities in a series of writing-focused exercises.

Study Results 

Both CSE and other disciplines participants displayed an average of intermediate proficiency in using ChatGPT.

ChatGPT as a Writing and Editing Assistant:

• Satisfaction levels increased after refining initial drafts, reaching similar levels for both groups.

• CSE participants showed a notable increase in satisfaction with final essay versions.

• Non-CSE participants rated final essays higher in quality and trustworthiness.

• Interaction ease with ChatGPT rated similarly across tasks and groups.

• Non-CSE users more satisfied with ChatGPT responses; error frequency varied between groups.

Collaborative Writing with ChatGPT

• High engagement across all groups, slightly more in non-CSE disciplines.

• Co-writing dynamics: CSE participants found ChatGPT more complementary.

• Creativity ratings higher among non-CSE participants.

• Story development satisfaction varied slightly, with overall experience rated higher by non-CSE groups.

• Future interest in collaborative writing with ChatGPT was mixed, generally higher in non-CSE groups.

Addressing ChatGPT's Content Policy Challenges in Creative and Professional Settings

In our recent exploration of ChatGPT's usability, we uncovered a notable challenge that significantly affects both creative and professional users. This issue centers around the AI's content policy, which can unintendedly hinder the creative process and disrupt professional tasks.

Creative Limitations for Writers

A example of this issue emerged during a co-writing session (presented in the photo below), where a participant's narrative involving mature themes triggered ChatGPT's content filters. 

This resulted in the AI generating error messages and halting the creative flow, leading to user frustration and disengagement. 

Creative Limitations for Writers

The censorship or flagging of ChatGPT could potentially be a problem for fiction writers. As it is apparent, a large number of adult books will deal with mature subject matter, such as profane language, violence or sex. However, since ChatGPT will flag or censor this type of content, this heavily limits its creative use to mostly children books. If ChatGPT wants to be seen as a tool for writers targeting at adult audiences, there will be a need for a model that is able to deal with classic literary tropes such as a detective solving a murder or a cowboy shooting down enemies.

Professional Implications in Legal Contexts

However, the censorship issue doesn’t just limit itself to creativity, it can also be problematic in professional fields, such as law for example. Taking the example of the field of law related to domestic abuse, a statistic from Canadian Centre for Justice and Community Safety Statistics said that there were 358, 244 victims of police-reported violence in the country last 2019. Assuming that a large number of these domestic abuse incidents in this statistic resulted in a court case, this means that lawyers working on any of these potentially hundreds of thousands domestic abuse court cases that year are unable to use ChatGPT to help them since they will most likely include triggering material which could result in flagging or censorship.

Tailoring ChatGPT for Diverse Content Sensitivities 

It is important to note that there are some use cases for a ChatGPT that doesn't deal with mature content, such as a help desk ChatGPT or a elementary school tutor chatbot. However, as it was previously explained, there are many instances where a ChatGPT that can deal with potentially triggering material is necessary. Hence, OpenAI should be able to provide various types of ChatGPT with different sensitivities to mature themes depending on the use case and audience.

Impact of gamification on user experience

By Thomas O. Ajayi

Introduction

The Entertainment Software Association(ESA) reported in 2016 that 65% of US households are home to at least one person who plays more than 3 hours of video games every week.
The success of digital games in the entertainment industry has made product owners and designers of non-game digital products turn to learning from digital games in order to understand what makes them engage the players and success in order to apply the same knowledge to improve not only improve the user experience of their products but also to make them enjoyable.

Interestingly, academicians and user experience design practitioners alike have researched what makes games successful and how to implement them to motivate and engage users of digital products that are non-games to interact more with the products and improve overall user experience. This concept of applying game-like elements  to non-game situations, with the goal of motivating and engaging users is now popularly referred to as “Gamification”.Popular game-like elements used in gamification are leaderboard, avatar, points and badges.

Surprisingly, my recent research into understanding the impact of gamification on user experience showed both the positive and negative influence of gamification on users as opposed to the assumption that gamification is only full of positive rewards. Below are the highlights of the positives and negatives effects of gamification on user experience and how to apply gamification to support more positive user experience in digital products  based on the insights from academic researches of Hammedi et al. , Koivisto & Hamari and Vanroy & Zaman and user experience design practitioners

The positive effects of gamification on user experience

1. It enhances users' motivation to interact with products:

   
   

Vanroy & Zaman researched the effects of rewards with  game elements of badges, points and leaderboard on university students in Belgium based on their participation in a Google+ community created for a course. Analysis of the students' response to online surveys and focus groups confirmed that the rewards they get based on their participation either as individual or group on the Google+ community motivated them to interact with the Online community more.

2. It boosts users' engagement and focus:

   
   

Hammedi et al  found out that gamified health care helps patients focus all their attention on the activity that they are performing. A practical example was the case of when patients used the Vr4Child app (a game-like application designed to relax children to ensure that they will remain calm during medical procedures), their engagement helped them to not focus on the nurses’ and therapists’ actions. Thereby helping the medical practitioners to perform their procedures without the interruption of the kids(patients).

2. It helps transform boring/tedious activities into more exciting and entertaining ones: 
   

   
   

Through the implementation of Wii Fit Plus- an innovative technology that allows patients to improve their balance through challenges such as a snowboarding simulation or walking on a tight, patients with various medical problems, such as the amputation of a limb, a stroke were report by Hammedi et al. reported to continually improve their physical exercise and to work harder to reach their limits. Without this gamified solution, such patients could find exercise which supports their healing process boring.

The negative effects of gamification on user experience

1. It can encourage users to cheat to get rewards: This was seen in the study by Vanroy & Zaman as students were reported to repeatedly perform the behaviour for which they were already rewarded in order to receive more badges.

2. The enjoyment and usefulness of gamification can decline with time Thereby leading to abandonment of the products where gamification is implemented: This negative effects is shown in the study of Koivisto & Hamari on the users perceived benefits of fitocracy where the younger user demography found the online fitness coach marketplace useful when they start tracking their daily fitness achievement using fitocracy application but after some times, they do not find the rewards they get for tracking their fitness motivating again. 

3. Gamified platforms can become addition for users: Lilith & Tamm reported some digital products especially social media e.g Facebook, youtube or Twitter implemented  gamification using Pull-to-refresh feature to see new posts continuously and the users might be attached to the social media because of the gamification experience. Thereby spending their productive time on the applications.

How digital product owners and user  designers can implement gamification to enhance positive user experience

The negative effects of gamification on user experience highlighted above has shown that integration of game elements in digital elements is not always motivating, engaging or sustainable. The question is now how do product owners and or user experience designers implement gamification to support positive effects while the negative effects are reduced or removed. Scholars and design practitioners proposed that using the Yu-kai Chou behavioural framework named “Octalysis” shown in the Figure 1 below could help in designing gamification for good.

Figure 1:  Yu-kai Chou behavioural framework. Source

The proposition is that one can easily understand what drives users' motivation using this framework and that subsequently will guide the game elements used in the gamification process.

Below are the highlights of what the 8 drivers of human motivation in the behavioural  framework stands for.
Meaning: means that users believe that the meaning of what they do is more important than the thing itself, giving a sense of self-mission.
Accomplishment: means the progress and new skills gained by users when they complete things.

Empowerment: providing creative channels for users to take initiative.
Ownership: means users’ ability to own and control things. 

Social influence: users’ interconnection with other users.
Scarcity: means that some things can be owned by only some users. 

Unpredictability: means mobilising users’ curiosity by showing limited things.
Avoidance: means showing the disadvantages of bad events, and users will choose to avoid losses. More details about this can be found here.

In addition to using the above framework as a guide while designing gamification to tailor the game elements based on the motivation factor(s) of users, designers it could also be helpful for designers to monitor products for side effects( i.e choosing which behaviours to encourage, figuring out how to measure those behaviors, and ensuring that the metric being tracked reliably correlates to the behavior that is wanted) and Set engagement limits. For example a mobile app can be designed to  automatically disable  notifications if a user is driving or remind the user to take screen to avoid distractions that can lead to accidents.

Conclusion
Considering the benefits of the gamification in digital products highlighted in this article and the quest for businesses and user experience designers to compete in the digital era to gain users attention and get them glued to their products,  one can say that  the strategy of gamification has come to stay.
A cue can be taken by product owners and designers alike from the successful implementations of gamification or the proposed fireworks from academics and industry
However, it is important to note that no one size fits all framework or guidelines to implement gamification successfully. Instead more research into who the users are, what the users want to achieve with the use of the product and the game elements that match the users types are needed in order to design digital products where gamification enhances positive user experience.

User Experience Evaluation of Indian Passport Seva Website using Nielsen's Usability Heuristics

By Amit Morade

Introduction:

The Passport Seva Website is an important platform for passport applications across India. Almost every citizen of India would at least once visit the website. Having such a huge user base, improving the user experience (UX) of the Passport Seva website is crucial for enhancing efficiency and user satisfaction. I have used Nielsen's 10 Heuristics to gain valuable insights into the usability issues of Passport Seva website. For each issue identified, I have tried to provide a solution that can effectively address these problems, provide severity rating to detect problems that need more attention and that needed to be fixed immediately, and significantly enhance the website's usability.  

The problems are categorized into different severity levels as low, medium, and high.

The green boxes in figures indicate proposed solutions and red boxes or red arrows highlight the problems.

About Passport Seva Website:

Passport Seva enables simple, efficient, and transparent processes for delivery of passport and related services in India. This project creates a countrywide networked environment for Government staff and integrates with the State Police for physical verification of applicant's credentials and with India Post for delivery of passports.

What is Heuristic Evaluation:

It is a method used for identifying problems in user interfaces, and to measure the usability of user interface designs. The evaluation is performed based on a certain set of guidelines that are aimed to make systems easy to use. I have used Nielsen’s 10 usability heuristics, a set of guidelines considering human behavior, psychology, error handling capability to assess usability of Indian Passport Seva Website.  

Fig. 1: Nielsen 10 Usability Heuristics

 

Passport Seva website usability issues:

Problem – Password policy is not directly visible to the user and user cannot view entered password.

Fig. 2: Password policy not visible, entered password cannot be viewed.

Issue: -

The website uses some rules to setup the password. These rules are not displayed directly on the page, but the user must click a link that opens a dialog box displaying the password policy. This reduces users’ efficiency, and the user must remember the password policy and enter the password accordingly. The user is also unable to view the entered password. Thus, while keeping the password policy in mind the user also has to remember the new password resulting in increased cognitive load on users memory.

Severity: High

Heuristic Violated: Recognition rather than recall and  Consistency and Industry Standards

Proposed Solution: - 

Fig. 3: Solution with password view button and password policy visible

A more feasible and efficient solution would be to provide the password policy information next to the password field so that the password requirements remain in front of the user as the user enters the password. Providing a clickable button to toggle the visibility of the password will help to check if the user has entered the right password giving more confidence on his actions.

Problem – Browser back button does not work to navigate between pages.

Fig. 4: Browser back button does not work to go to previous page

Issue:

The website does not use browser default back button to navigate to any previously visited pages. All modern websites make use of browser’s default back button for navigation. Also, no message is if browser back button should not be used. For instance, to change the district after selecting it, the user had to navigate to the “Applicant Home” page and then redo all the process. This may irritate the user. Also, upon selecting the “district” field value the form automatically gets submitted and the user is directed to the next page for further information. This takes away the control from the user. If the user mistakenly chooses the wrong district then he won’t be able to update it and since there is no back button to revisit the page and change the form, he will have to restart the entire process which could be very annoying for most people.

Severity: Medium

Heuristic Violated: User control and freedom and Visibility of System Status.

Proposed Solution: -

Use browser back button same as all the modern websites do for navigation, else provide a physical back button (specific to the site) to navigate to previous pages. If the website expects the user to enter the “district” details only once, then they should have provided a message for user that “You cannot modify the below information”. A button should be provided so that user can navigate to the next page after he has entered the required information.

Problem - Repetitive Information resulting in cluttering and reduced efficiency.

Fig. 5: Applicant home page cluttering with small font size and repetitive information

Issue 1:

On the home screen, the user can see the page is divided into 2 parts, “Services” and “Applicant Home”. All the information listed in the right section i.e., under “Applicant Home” is also displayed in the left section under “Services”. This way of putting up the information is confusing because when clicking on a link, there is an implicit assumption that the link in right section will be the correct link and not the one displayed in the other section. This issue could be a serious problem, especially for new users resulting in frustration, and difficulties in navigation.

Severity: Medium to High

Issue 2:

The font size is also very small, and it is difficult to read the information correctly because this view appears to be cluttered making it challenging for users to navigate and absorb information quickly. It might cause strain on the user’s eyes and might reduce user’s efficiency to find and locate information.

Severity: Medium

Heuristic Violated: Aesthetic and minimalist design and Recognition rather than recall   

Proposed Solution: -

All this information could be displayed under one heading with the most visited or the most important links being positioned at the beginning of the list. This way users might feel easy to find what they need without having to decide between sections. By displaying all the information(links) under one heading, there will be more room on the page and so this information could be displayed in a better way with increased font size and spacing between individual links.

Problem: No hint/help provided to understand the meaning of shortforms used.

 

Fig. 6: No help documentation provided for shortforms used

Issue:

The website has used some shortforms such as “PSK/POPSK/PSLK” without providing any help to understand the meaning of these terms which may cause confusion for the user. Use of such unfamiliar text would make it difficult for user to take further actions and without any provided help the user will have to figure out their meaning by themself, reducing efficiency and increasing time to complete the task. Also, the “Click here” links are not easy to identify.

Heuristic Violated: Help and Documentation and Match Between System and Real World.

Severity: Medium

Proposed Solution: -

Fig. 7: Solution with help documentation, next button and updated click here link

A message could be displayed somewhere below the form like “Click here to know more about PSK/POPSK/PSLK”. This can provide immediate help on unfamiliar terms, making the user feel confident and in control of their next actions. Users will have more control over their interactions with the website if clear messages are given to them.

Problem: Finding payment amount and payment status information is difficult.

Fig. 8: Payment amount missing, success message hard to find

Issue:

Finding the payment success information is difficult. The font used is very small and the light green color which is used for success message is undervalued by the red text that is dominating the screen. Also, the information about the payment amount is missing.

Severity: Medium

Heuristic Violated: Visibility of System Status and Aesthetic and Minimalist Design

Proposed Solution: -

Fig. 9: Solution with payment amount and payment status icon

A more intuitive way of putting the success message could improve readability. A recommendation would be the use of a green tick icon with a “payment successful” message and a field showing the amount paid that gives the payment status enhancing visibility, providing more clarity and confidence to the user about the successful completion of the payment task. The green tick icon will grab user’s attention allowing user to quickly identify success status without the need to navigate or search for information. Displaying payment amount will provide user with the needed critical information preventing any confusion about transaction status.

Conclusion: -

The Indian Passport Seva website has some minor drawbacks in the user interface and design with respect to the Nielsen’s heuristics mentioned above. I have tried to provide some solutions for each of the problems discussed along with their severity. With implementation of recommended solutions, it would result in increased user efficiency and enhanced user experience with the website. 

Acknowledgements: - 

To make the grammatical corrections, enhance the overall quality of the content and address potential language-related concerns Grammarly was used.  

References: -

[1] https://medium.com/@sef.gustingurah.yama/visual-design-the-face-of-your-apps-67a03427824f

[2] https://www.nngroup.com/articles/ten-usability-heuristics/

[3] https://www.passportindia.gov.in/AppOnlineProject/welcomeLink#

[4] https://app.grammarly.com/

Usability Evaluation of Microsoft Teams: Focusing on Text Chats and Video Meetings

 By Behrad Moeini

Microsoft Teams: Crucial for Remote Collaboration

In today’s era of remote work and virtual classrooms, tools like Microsoft Teams have become indispensable in bridging the gap between colleagues and classmates worldwide. Despite its widespread adoption, there's always scope for enhancement in such platforms. In this blog post, I have assessed Microsoft Teams using Nielsen's heuristics, a set of established usability principles, to identify areas where it can evolve to more effectively meet our changing needs. This evaluation spans from the intuitive ease of text chats to the seamless flow of video meetings, with each facet offering potential for refinement. My exploration is not merely a critique but a pathway to improvement, guided by the principle that our digital tools should not only meet but anticipate and adapt to our evolving demands. This approach offers a structured and systematic analysis, ensuring that the critique is grounded in proven usability standards.

Text Chat: Finding and Fixing Issues

Issue

Fig 1 –  Inefficiency in Message Retrieval on Microsoft Teams

My analysis of Microsoft Teams' text chat functionality uncovered a significant discrepancy in the search feature. As demonstrated in Figure 1, a search for '512' reached no results, suggesting an absence of relevant data. However, a following search including an additional digit, '5122', surprisingly retrieved messages containing '512'. This inconsistency in search results can lead to missed critical information and unnecessary time expenditure, pointing to a need for a more robust search mechanism within the platform. Given the obscured visibility of system status in search functionality, I rate this issue a 3 out of 5 for its moderate impact on user efficiency and confidence.

Suggestion

Fig 2 - Refined Search Capabilities in Microsoft Teams (Visualized with DALL.E)

Acknowledging that Figure 2 may not precisely mirror the current aesthetic of Microsoft Teams, it serves as a conceptual illustration to underscore my proposed enhancement. This visual representation depicts an advanced search algorithm, potentially incorporating elements of natural language processing. The key focus here is on a search functionality that transcends mere exact word matches. Instead, it intelligently identifies and suggests content that is contextually related to the user's query. By integrating such a sophisticated search mechanism, we aim to significantly reduce the time and effort users spend sifting through chat histories. This advancement is not just about finding exact phrases; it's about intuitively connecting users to the information they need, thereby refining the overall user experience and efficiency on the Microsoft Teams platform.

Video Meeting: Interface Challenges

Issue

Fig 3 – Navigational Difficulties with Video Meeting Controls in Microsoft Teams

In our exploration of Microsoft Teams' video meeting interface, as showcased in Figure 3, we encounter a layout that can be confusing, especially during critical meeting moments. The current design places various controls - like 'Dial pad,' 'Hold,' 'Transfer,' 'Chat,' and others - together in a single toolbar. This setup often makes it difficult for users to quickly locate and access specific functions, such as the 'Mute' button. In fast-paced or large meetings, this can lead to unnecessary delays and even disruptions. The confusing layout of video meeting controls, which impacts user control and freedom, merits a rating of 2 out of 5 for its potential to disrupt user experience during important meetings.

Suggestion

Fig 4 – Proposed Notification Interface in Microsoft Teams

The suggestion for improvement, as we can imagine in Figure 4, involves redesigning the video meeting controls to be more intuitive and user-friendly. The focus here is on rearranging the existing controls so that key functions like the 'Mic,' 'Camera,' and 'Leave' button are easy to find and use. We could group similar controls together and even introduce a customizable toolbar, allowing users to organize the functions they use most frequently in a way that best suits their needs. Testing different layouts with actual users and applying their feedback would be crucial in achieving an interface that not only looks good but also feels right to use. The goal is to make the video meeting experience on Microsoft Teams smoother and more efficient, reducing the time users spend searching for controls and increasing their focus on the meeting itself.

Notification Overload: Balancing Alerts and Attention

Issue

Fig 5 – Excessive notifications disrupting a user in Microsoft Teams.

A significant challenge identified in Microsoft Teams is the management of notifications, as illustrated in Figure 5. The issue arises from an influx of repetitive alerts from a single contact or thread, which can clutter the user's workspace. This excess of notifications not only hinders productivity but also risks important messages being overlooked or ignored, as the user may become desensitized to the constant stream of alerts. The problem with notification overload in Microsoft Teams, primarily violating the heuristic of aesthetic and minimalist design, receives a rating of 2 out of 5, reflecting its impact on user focus and screen clutter.

Suggestion

Fig 6 – Customizable notification interface in Microsoft Teams  (Visualized with DALL.E)

To address this, as conceptualized in Figure 6, a redesign of the notification system is suggested. The focus of this redesign is on simplicity and user control. One approach is to group similar notifications, thereby reducing the number of individual alerts. The system could also offer features like a 'snooze' option, enabling users to pause notifications for a set period. Additionally, customization settings could allow users to prioritize notifications based on their relevance or urgency. Such enhancements aim to not only declutter the notification area but also empower users to tailor their notification settings to their specific workflow needs. Implementing these changes can significantly improve focus and efficiency by ensuring that users receive only the most pertinent alerts, allowing them to maintain their concentration on the task at hand.

Incorporating Authentic User Feedback for Practical Enhancements

To ensure that the proposed changes truly resonate with the needs of Microsoft Teams users, I integrated direct feedback from actual users into my evaluation process. This approach was complemented by a structured heuristic analysis, aligning with Nielsen's Usability Heuristics. By combining real-world insights with established usability principles, I was able to identify a range of usability challenges – from clear-cut functional issues to more subtle user experience obstacles. This blend of authentic user input and expert analysis forms the cornerstone of my recommendations, ensuring that the suggested improvements are not only theoretically sound but also practically relevant to everyday users.

Conclusion

Microsoft Teams, as a vital tool in our remote work and education era, has several areas primed for improvement to enhance its overall utility. By addressing the specific concerns highlighted in text chat functionality, video meeting interfaces, and notification management, we can significantly improve the platform's usability. These enhancements, rooted in genuine user feedback and expert analysis, are geared towards creating a more intuitive, efficient, and user-friendly experience. Such continuous evolution is essential in keeping pace with the dynamic needs of modern communication and collaboration, ensuring that Microsoft Teams remains a robust and reliable tool for users worldwide.

Acknowledgements

I used Grammarly to check and correct grammar in this blog post, ensuring the writing is clear and accurate. This helped improve the overall quality of the content.

References

[1] Nielsen Norman Group - 10 Usability Heuristics for User Interface Design: https://www.nngroup.com/articles/ten-usability-heuristics/

[2] Nielsen Norman Group - How to Rate the Severity of Usability Problems: https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/

[3] Microsoft Teams admin documentation: https://learn.microsoft.com/en-us/microsoftteams/

[4] Grammarly - English Grammar Check: https://www.grammarly.com/grammar-check