Improving virtual pedagogy outcomes by simulating real-classroom elements online

My Role

Team Lead
UX Designer

The Team

Arpit Ranka
Sandy Jiang
Yao Zhou

My Key Contributions

Project Management
User Research
Research Consolidation
Ideation
Prototyping
User Testing
Process Documentation

Tools Used

Mural
Figma
Canva
Zoom
Google Suite
Keynote

Duration

Semester long - Fall 2020

Process

Desktop View ONLY. Please revisit on your computer/laptop.

Research

User Interviews
Participatory Design
Camera Studies

Consolidation

Empathy Maps
Affinity Diagrams
Personas

Ideation

Design Charette
Sketches
User Scenarios
Storyboards

Prototyping

Hierarchical Task Analysis
Heuristic Evaluation
Lo-Fi Prototypes

Testing

Usability Testing
Iterative Prototype
Journey Map

Introduction

Programming has become an essential skill for the technology worker of the 21st century. We recognize that programming skills are in high demand, luring even non-CS majors to learn it and upskill their resume.
We were a team of 3 NYU grads that employed human-centered design techniques to research, reveal and synthesize : the pain points , feelings, goals of such people as they go about trying to do so. We used these learnings to help guide our design decisions to improve upon or recreate elements that catalyze the process.

User Research

Working remotely hinders many effective user research methods like contextual inquiry, shadowing etc. But we tried to make the best of available alternatives that can be performed virtually over video call, screen sharing and file sharing applications. We conducted our user research with fellow NYU colleagues that had previously learnt or were then learning basic programming :

1. User Interviews

Eight eligible NYU grads were interviewed virtually via Zoom. We asked a dozen non-leading questions around :

  • motivation to learn code
  • challenges faced
  • resources used
  • factors affecting learning outcomes

Asking participants to rate something on the scale of 1-10 gives ambiguous and futile results. Instead, facilitating the process of storytelling using 5 Whys yields great qualitative data.

2. Participatory Design

Ideal interface sketch from a participant.

Three participants were asked to sketch out or narrate their ideal virtual learning process or environment in order to help us understand users’ needs and mental models.

We wanted to see how our participants interact with the online learning resources, and it may give us clues about what the actual constraints towards effective learning are.

3. Camera Studies

E-learning setup of a participant.

We asked 3 participants to take casual photos of their digital or physical learning environment when they are working on a learning task, the task may/may not be related to programming, for a week prior to the interview.

In doing so, we wished to uncover what modes of learning they are using, and make informed guesses about the factors that may affect learning results.

This method did not yield much useful insights for us. Looking back , I think more specific prompts along with triggers might have helped like sending a daily check-in text. This would remind participants to respond and share pictures instantly.

Consolidating Insights

We had collected a rich collection of user data from the research methods we employed, especially user interviews. The user research led us to create two personas that we refer to , through our entire process: Tim and Betty.

Our Affinity Maps gave us these 3 main pain points of our users :

Says

Does

Thinks

Feels

1. Lack of communication with instructor

Insight #1 : Beginners would love to have direct communication with their instructor for guidance and feedback.

2. Dilemma with getting started

Insight #2 : Beginners are paralyzed by and clueless about the plethora of learning options available online.

3. Lack of collaboration/ co-learning

Insight #3 : Students often miss the presence of their classmates online and would welcome collaboration.

Problem Statement

Our problem statement evolved over the course of our design process. We decided to choose a recurring problem across our personas and moulded our problem statement around it.

How might we induce an online collaborative community for beginners to learn programming together?

Ideation

In our Design Charrette, we turned the pain points of users into goals and ideated solutions in isolation. Then we shared our sketch , explained our solution and welcome feedback on it. We collaborated to make our ideas more effective and clear , add / prune elements if needed. 

1. Presence of a TA chatbot for guidance

Tim and Betty would both appreciate personal attention or presence of a TA (Teaching assistant) when they are stuck on debugging a piece of code and the TA can see their screen and specifically point out their errors like a friend would.

2. Help choosing a learning path

Newbies to programming like Betty are uncertain of what all they should learn in order to fulfill a job role could easily select a learning path and it would systematically show them the way. This will enable them to make informed , confident decisions .

3. Simulating a classroom-community feel

Tim makes extra effort to be around a learning community like going to coding meetups.
A virtual community (fb chat like feature) would keep him motivated, accountable and allow for collaboration and co-learning.

Prototyping and Testing

We mocked up wireframes on our ideas and stitched them together on Figma. Our lo-fi prototype was a bare-bones simulation that was used for User testing with 5 participants. Out of these candidates, three were representative of our persona ‘Tim’ and the other two of ‘Betty’.

We conducted our usability sessions (~20 mins long) over zoom with 5 participants in this order:

  • First Impressions : Users were asked to express their first impressions of the interface.
  • Exploratory : Users were encouraged to explore the application while thinking out loud.
  • Directed tasks : Users were asked to perform certain tasks like ‘reach out to the instructor’.

The usability sessions were evaluated on the basis of:

  • Discoverability : Are the users able to discover and understand a new and unexpected feature.
  • Effectiveness : Are the users able to effectively navigate across screen with minimal errors ?
  • Efficiency : How long does it take for them to do the task?

Some screengrabs with findings from the process are :

Only include that which needs to be tested in your lo-fi prototype. Trying to beautify or adding extra elements only confuses the user.

Iterative Prototype

Based on the learnings from our usability study , we fixed the issues we heard across 2 or more participants. These included refining our design or adding and subtracting features from our application.

Learning Paths allow newbies to make informed decisions.
User is able to collaborate with classmates and also directly message instructor for doubts.

Journey Map

Here is a future-state journey map of Betty interacting with our iterated prototype and getting her pain points addressed :

Final Screens

Here are the final high fidelity screens that would be shared with engineers along with a guiding document explaining the design and any other special considerations.

Let's Connect!

COPYRIGHT © 2021 ARPIT RANKA . All Rights Reserved.