As part of my Masters Thesis I worked with DB Cargo, that is responsible for all the rail freight transport activities of the German railway company Deutsche Bahn both inside Germany and on a global level to build an Artificial Intelligence based UX for 3D Damage Detection of wagons through camera bridge.
DB Cargo Overview
DB Cargo is a global transport and logistics firm handling rail freight activities for Deutsche Bahn within Germany and worldwide. Their core focus is on transforming production to enhance rail logistics. Prompt service delivery is paramount, but challenges arise with real-time identification of damaged locomotives, impacting deliverables. Currently, the freight rail sector relies on outdated methods like camera bridges for technical detection of locomotive damages.
Understanding the Problem
There are currently 09 camera bridges located at different locations throughout Germany to identify damages in the wagons. Around 70 camera bridges will be implemented by 2030 to improve wagon handling and damage management.
- Employees currently use old-fashioned techniques to detect damages
- The tools they use are not user-friendly and require expert knowledge
- Multiple tools are used in their workflow to identify and report damages to the maintenance team
- Employees are challenged to find a more efficient way to identify damages and increase productivity
- A team of AI engineers is working to analyse and provide insights on damaged wagon data and struggling to develop an intuitive user interface to showcase the data.
Setting the Objectives
- To understand the current user journey of the damage detection process.
- To identify the pain points and challenges faced by the user.
- To understand the use cases for existing interactions in the application.
- Create user persona based on the user behaviour.
- To understand what kind of data Artificial Intelligence can identify out of the images.
- Create an enhanced user interface using the AI generated data.
- Find a possible solution to ease the entire damage detection process.
- Test the solutions with the end users.
- Derivation of recommendations for action for the realization of the solution.
Design Approach
To attain an improved user experience with innovative solutions, I implemented a design thinking framework developed by Stanford that prioritizes a user-centered approach (UCD). This framework comprises five phases, namely Empathize, Define, Ideate, Prototype, and Test.
Learning about the user (Empathize)
To gain insights about the challenges faced by the users, workshops were organized, and observational studies were done.
Conducted workshop with the employees responsible for damage detection to understand the process, their current approach and identify the challenges faced by them.
Observed the employees, how they interacted with the existing elements. Their emotions, behaviours, time consumption, etc.
Camera bridges have been installed at several shunting yards for inspection of wagons. When a wagon passes through the camera bridge, high definition images are captured by the line scan cameras.
Interaction with Artificial Intelligence Team
Interacting with the AI team, insights were gained into the data model being created to detect damages from images. A sample result of an auto-detected damage from an image of a wagon was provided by the AI team, enabling observation of the type of data available and potential further utilization.
Workflow of Artificial Intelligence
Image generated by Artificial Intelligence
Understanding the users (Define)
Deriving Solutions (Ideation)
According to a study conducted by Wei Xu & Dov Furie on Designing for Unified Experience in Jan 2016, they suggest some benefits of having an unified experience design.
Having a unified end-to-end workflow in a single module would be beneficial. A new user journey mapping was created to address this.
Design Feedback
The re-designed screens were presented to the users along with the stakeholders to have a first impression and gather some feedback on the design solutions.
Iterating User Feedbacks
While thinking about other possible solutions, went down the memory lane in thoughts to remember how I used to modify the cars in racing games. Did internet research about the user interfaces.
Why this?
- It gives the user a 360° view of the car
- Immersive gamified experience
- Great user interaction design
According to a study conducted by Mohan Mohanty & Team in June 2021 on “Construction of 3D Objects from 2D Images”, found out that creating 3D model out of images doesn’t give the expected output. However, if we already have a base model, images can be used as textures to obtain a high clarity.
3D models use textures on top of a base model. Images received from line scanner cameras can be used as textures.
.gif)
Improved User Interface
Following the brainstorming session with user journey mapping, concept creations and collaborations with the AI team, I went ahead with designing the user interface for the users. Defined the user interactions in the UI such as interacting with the 3D model, accepting or rejecting a damage, adding a new diagnosis, adjusting the viewport controls from editing panel, etc.
Prototype
Figma prototype tool was used to construct the prototype flows.
Usability Testing
- Wizard of Oz replicates with almost similar situations as compared to when it is finally developed.
- Used Webflow to represent my design to the users.
- Integrated it with Hotjar to record the sessions and analyze the heatmaps.
Design System Components
Outcome
Recognition
- Awarded 2nd Prize at the German UPA UX Challenge 2023, for innovative UX solutions.
Project Highlights
- A team of 3D designers will be designing base models of all types of wagons.
- AI Team will be creating textures from the images received from the camera bridges for corresponding wagon ID.
- Development Team will fetch APIs and develop the new user interface.