DEME Group

Remote Crane Control

The Vlaamse Waterweg and DEME Environmental Contractors joined forces to tackle the increasing problem of river pollution. They contacted us for a smartglasses solution to remotely control a crane that is installed in the middle of the Scheldt.

The issue

DEME Group has installed a plastic collector on the river Scheldt to collect floating waste from the water. The mobile installation consists of a smart detection system, a work boat that can navigate autonomously and a charging point. The waste is collected in the floating collection pontoon, where a crane transfers the waste into a container.

Since there’s no direct access to this floating platform and the crane, a remote solution was in order. As it’s essential for the operator to see his physical crane controller at all times, VR headsets were off the table quickly.

What we did

We developed a custom HoloLens application that allows DEME to control the crane without being physically nearby. With the help of a stereoscopic camera, the app visualizes the actions of the crane and its environment in a true 3D livestream, projected on a holographic hemisphere. This allows the operator to perceive depth and perspectives as if he would be on the pontoon, while still being able to see his real-life surroundings.

End-to-end latency

Since there’s no room for errors, the end-to-end latency (the time between making a single video frame, streaming it, and showing on the HoloLens) of the stream was highly important and a determining factor to give the project a ‘No‘ or ‘Go‘. By using a mix of the industry standard streaming protocols and a carefully chosen combination of the videocodec, container and transport parameters we were able to achieve a latency of around 150 to 200 milliseconds! Generally, anything below 400 milliseconds is considered “realtime”.

However, what makes the project truly robust is a secondary “controller” mechanism keeping track of this latency. Should the stream slow down and get off due to network issues for example, a warning (or in elaborated stages even an alarm) appears on the screen that instructs the operator to pause his process until the latency is back at normal values.

Interested in knowing more about the technical setup? We told all about it at Microsoft Ignite 2020.

HoloLens 2 making it all possible

The HoloLens 2 allows for pinsharp stereoscopic projection of the 3D camera image. This is very important as the operator requires an accurate sense of depth in order to control the crane.

Besides that, it’s essential for the operator to see his physical crane controller at all times. As the HoloLens is a Mixed Reality device, the operator has a perfect view of the camera image, while still seeing his physical remote control.

Also, the HoloLens is a fully standalone, untethered headset. It can be used anywhere the operator is, in every office or any location in the field, as long as a good network connectivity is provided. No need for an additional laptop or desktop computer. Having this functionality running on a HoloLens proves to be a very convenient and portable solution.

Project highlights

Months of hard work

Boat trips

0 +

Coffees brewed

0 k

Budget

Your project up next?

Fill out the form for a first conversation.
We will contact you within the next business day to set up a call.

HoloLens Value Canvas

Do you wish to brainstorm internally about the possibilities of the HoloLens? We’ve got you covered! Download our easy step-by-step guide for implementing the HoloLens in your organization.

Contact

hello@mrwatts.io

Th. De Beckerstraat 29
3200 Aarschot
Belgium