top of page

Augmented Reality Interface for NASA Lunar Missions

Problem: How can we give astronauts autonomy to complete their tasks in space without the help of Mission Control?  

Solution: Provide AR tools to help assist and automate the process of major tasks in Space.

Time Frame: 1.5 years

Team: 16-20 Designers & Engineers 

My Role: Project Co-Lead, Design Lead

Client: NASA

Finished Product

AR Interfaces

To assist astronauts during their extravehicular activities

Including navigation on the lunar surface, vitals display, geological sampling tools, search and rescue



Our team's HoloLens Interfaces getting tested by NASA Designer Skye

NASA Spacesuit User Interface Technologies for Students (SUITS) Challenge selected our team to develop Augmented Reality (AR) software on the HoloLens.

With the HoloLens, we can create 3 types of visual elements...


Registered in the environment in a set location

Sticky interface registered in the environment following the user's view 

Registered to the hud following the user's view


Team & Milestones

Team: 16-20 members

(responsibility changing throughout the challenge)

2 Team Leads: Bowen(Me) & Selena

Design Lead: Bowen(Me)

   4 Design Sub-Leads:

   Geological Sampling




2 Software-Leads:





We talked to: 5 AR, voice, navigation designers, 2 NASA Designers, 2 Brown Planetary Geo-science Professors, and 2 Astronauts!


James H. Newman

Former NASA astronaut who flew on four Space Shuttle missions.


Steve Swanson

Retired NASA astronaut who flew on three missions to the ISS.


James W Head

Worked on previous Apollo Programs and trained astronauts. Geological Science Professor at Brown.

What are some challenges astronauts face on moon?

Difficult to Navigate


Dangerous terrain with tripping hazards


Environment with no distinctive features for landmark recognition


No GPS, relied on route memorization and photos

" There are no high mountains or volcano - something distinct that  you could recognize - a landmark would help tremendously." 

Nothing is given to us right now.  We practice memorizing our paths and rely on some pictures. There was one time where astronauts got lost and couldn't find a destination so they had to abandon the tasks there."

Limited Movements


Whole-body movements are required to accomplish simple tasks


Small hand gestures require a large amount of effort


Both hands are required for most tasks

"Glove bulky, no tactile feedback. you are basically in a balloon. Any time you move you are squeezing the balloon."

Limited time 

Each minute in space wasted is a large sum of money lost. Mistakes are costly and there are a lot of tasks to accomplish.

What are their main objectives on the moon?



Long-Range Point A to Point B


Short Range Terrain mapping & Obstacle Avoidance

"We bounce around a lot to navigate. The lighting conditions can be extreme and a lot of hazards are hiding in the shadows. We must always pay attention to the ground in front of us."

Geological Sampling


Taking Samples


"We have specific goals when we arrive at a site, we communicate with the geologists at Mission Control and take exact samples. We take notes, measurements, and photos of the site. "

Staying Alive: Vitals & Rescue


Vitals Display


Emergency Navigation & Communication

"If something goes wrong, how quickly can you go back? You always need to know what's wrong, how much oxygen you have left, and what pace you need to travel at under an emergency."

Design Process

Design Process

Objective-based User Profiles

separated our team into design sub-teams based on the main mission objectives and conducted 3 scenario-specific user research. 


Extravehicular Activities (EVA)

Geological Sampling

Central Profile


Anne McClain

Anne is going on the Artemis Mission where she would travel to multiple unexplored locations on the moon for scientific discovery as well as traveling to established sites to update equipment left by previous missions. Show would have a partner traveling with her and another astronaut in the spacecraft.

Anne's Artemis mission mostly consist of...


Traveling on foot for around 15-20min intervals


Taking samples and documentation from unexplored sites


Maintenance and setting up new equipments

Ideation Summery

Based on the challenges & objectives, the teams went through lighting rounds of ideations.

Untitled (1).jpg

💡 Learning to eliminate

We learned to rigorously eliminate ideas and concepts. There were a lot of amazing ideas that would improve the AR experience which we had cut out because we couldn't do everything.  

We have to ask ourselves, is this function/feature necessary to achieve our goals? If not, let's put these aside into our future features bank.

Flow Diagrams

Simplified This...


To This!


Untitled (1).jpg

Vitals, Geo-sampling, Emergency Communication


Design Iterations

7+ Rounds of Iterations

Nav - Bigger Map – 11.png
Navigation Instructions-1.png
SUITS FIGMA (Copy)_Page_2.jpg
Screen Shot 2022-06-23 at 4.40.10 PM.png

Iteration Learnings

💡Lessen & attach elements in the environment

Problem: During testing, we observed that users are often confused when there is more than 2 sticky interfaces floating and following them in the environment. It became hard for them to understand where things are especially when they are doing other physical tasks at the same time.

Solution: Minimize the amount of interface registered in the enviorment that follows the user's view to one at all times. Attach elements to each other into a unity to avoid spatial confusion.

Navigation Instructions-3.png

💡Set separate use case for each plane of interaction

Problem: Users' attention can only focus on 1 plane of interaction at a time. when they are looking at elements registered in the far environment, they miss elements on the floating interface right in front of them, and vice versa.

Solution: Create different use cases for each plane of interaction so users' attention doesn't have to be separated during a flow. We limited all decision-making processes where users have to be focused on the screen to the sticky interfaces floating right in front of them, all guiding tools such as compass and time to the hud, and all elements that are used while users are actively doing something else to the environment, such as the guiding pathway in navigation mode. 

Navigation Instructions.png

Iterations: instead of a sticky arrow, directions are given with a pathway in the environment.

Decisions such as setting a destination and checking vitals are limited to the sticky interfaces close to the user.

SUITS FIGMA (Copy)_Page_4.jpg
SUITS FIGMA (Copy)_Page_3.jpg

💡Automate processes & combine features

Problem: From astronauts, we learned that they have limited movements and small hand gestures require extra effort. To save astronauts' physical energy and attention, we have to minimize the amount of input needed in a flow. 

Solution: On top of designing voice commands, we automated each flow as much as possible. letting one action directly leads to the next, and interfaces open automatically. We combined multiple features into one, and ask ourselves what is the least amount of elements needed to achieve a goal?

Automate Processes

Combining Features

Mini-map's function of illustrating direction could be eliminated by adding a direction indicator on the compass. It's close range function of showing the route could be performed by having a navigational pathway rather than arrows in the environment.

Nav - Feature set View – 2.jpg


Nav - Suit heading matches Nav HDG; FWD arrow (less intrusive) – 1.png
Navigation Instructions-1.png
Navigation System Expanded View.png

Red direction indicator on compass

SUITS FIGMA (Copy)_Page_3.jpg

Pathway in environmental showing approximate closeness to destination with color

💡Clear the field of view,
Test in the HoloLens early!

Problem: From astronauts, we learned that they have limited movements and small hand gestures require extra effort. To save astronauts' physical energy and attention, we have to minimize the amount of input needed in a flow. 

Solution: On top of designing voice commands, we automated each flow as much as possible. letting one action directly leads to the next, and interfaces open automatically. We combined multiple features into one, and ask ourselves what is the least amount of elements needed to achieve a goal?

On Hover of location-2.png
Selena Home Page.png
Navigation System Expanded View.png
When Nav System is Active.png
On Hover of location-5.png
Rescuer 1 first screen-5.png
Navigation System Expanded View.png
Final Designs

Final Designs

SUITS FIGMA (Copy)_Page_4.jpg
On pressed of location.png

Finished Products & NASA on-site Testing

Feedbacks from NASA



Speeds up the Current Meeting System

" If you have access to all of this, it will walk you through the process. You would know this is something you have to talk about, and you can set it up right here. "

Opens a conversation up

" I like that you are forcing people to write out stuff about the meeting beforehand, so the conversation is open before we got there. The hardest part is starting the conversation. This makes it easier. "

Reduce Barriers

" This could give people access to help 24/7 without having to go through a supervisor, which is embarrassing if you want to talk about mental issues, and takes forever."

"This would be a less stressful and easier way for someone to reach out and talk about a touchy topic."

Moving Forward...


Test the effectiveness for this format of conversations, which involves in-person or video conferencing with three participants, having a structured timeline, etc.


Test for the ideal number of people there can be in a conversation intended for learning and understanding


If the tests indicate difficulty in conversations, test another concept which involves having a third experienced party to facilitate the conversation


Start researching and designing the platform that civilians and civilians in professional fields use to connect with the officers.

Thank You!

Thank you! I am very excited to show you this work and I would love any criticism and welcome any feedback from you!

bottom of page