Rock breaking via remote operation

“How might the workflow of a rock breaking operator change when they transition from onsite to remote operation & what system will they utilize?”

 

Project info

MFA: Professional Product
10 Weeks

 

Collaboration Partner

Boliden AB
RISE Interactive SE

 

Team members

Yuanrui Li
Justyna Fryczak


Interface shows current status of all rock breakers and status of teammates.

Operator operate with a primary partner daily, and the partner is the first point of contact when help is requested.


PROCESS

 

Contextual inquire in the Arctic Circle

We began with a visit to the Aitik mine in Gällivare to observe and interview the rock breaking operators of Boliden AB. We began with an overview of the mine operation and how rock breaking fits into the overall process, and why exactly is needed. This was followed by observing rock breaking operators performing their task, ending with interviews.



Research Analysis

Based on the research we collected we found the three areas we wanted to focus on for this project: Improved vision / visualization, control usability, and team work environment.

Artboard 1.png

Visualization: Moving from onsite to remote operation, how does this affect the ways the operators visualize their tasks?

Usability: Current machines are not intuitive, and changing operating sites require readjustment which reduces efficiency.

Collaboration: Operators works in teams of two and develops strong bond that complements each other.

We mapped out what we learned to better understand the process and how it applies to our focus areas.

Artboard 1.png

Prototypes

In the case of control usability, we build various mockups to test our various proposed solutions. The mockups were used to test intuitiveness, comfort / ergonomics, and complexity with a number of subjects.


Feature #1: Visualization

By using an array of low cost and easily replaceable cameras, we can allows the operator to virtually pan the view around the pit, easily to look at the operation from a various different angle when they need a better look.

Current system : Cameras placed  arbitrarily

Current system: Cameras placed arbitrarily

Goal : Feeling of looking through a  window

Goal: Feeling of looking through a window

Solution : Camera matrix array. View from  various angles

Solution: Camera matrix array. View from various angles

Simulated camera matrix view

Simulated camera matrix view

 

Feature #2: Simplification & Usability

1. Aiming assistant

An important part of breaking rocks is where to put the hammer tip on the rock, and be perpendicular to the rock before breaking. We provide visual indicators to show 1) when we are perpendicular to the rock, 2) when the hammer tip is approaching the rock and make contact, and 3) when we begin to hammer the rock.
This allows the operator to have very clear visual indication without straining to see or trying to decipher the sound feedback from microphone.

2. Ghost mode

Mechanical rockbreaker moves slower than human operators. This mechanical latency means that the operators have to constantly wait for the machine to "catch up" before the know they are positioned correctly. 

Our proposal is to create a digital silhouette of the rockbreaker that moves as fast as the operators do to provide instant location feedback. This "ghost" can help operators position correctly without annoying pause and adjustments.

controller.gif

3. Controller

Our system is semi-autonomous, and the rockbreaker automatically positions itself incorporating boom-tip control. Consolidate controls from two hands to one hand helps to simplify the control and make it more intuitive.


Feature #3: Team-based UI

Despite working remotely, rock breaker operators continue to operate in teams. Each operator is paired with a teammate, their primary partner, each day. When they need help, their primary partner is the go-to person for the day. This maintains a tight knit team that is currently working well for the current in-person, onsite setup.


Final

For the final concept, we arrived at multi-screen solution below: Infrared view to detect the presence of other humans or animals for safety. The central view consisted of the integration of the visual assistant modes we came up with, and a multi-camera display to monitor the surrounding areas for safety, again for intrusion. The desktop display is for team communication and assist, and finally a single multi-axis joystick for the actual operation of the breaker arm.

IMG_2934.JPG