Slide 1

Case study

DEME | Remote Crane Control

DEME Group

DEME uses smartglasses to collect floating waste from river

All over the world, river pollution has become a serious problem. In Belgium, De Vlaamse Waterweg and DEME Environmental Contractors decided to join forces and combat pollution in the Scheldt, a river that runs through Flanders with Antwerp as its biggest port. DEME is a Belgian company but also a global leader in the highly specialized fields of dredging and land reclamation, solutions for the offshore energy market, environmental and infra marine works.

To tackle the pollution challenge, DEME and De Vlaamse Waterweg installed a remotely controlled crane in the middle of the river. They contacted Mr. Watts to help them find a solution with smartglasses to operate the crane from the shore or even from an office.

A pontoon, a crane and an operator

DEME Group has installed a plastic collector on the river Scheldt to collect floating waste from the water. The mobile installation consists of a smart detection system, a work boat that can navigate autonomously, and a charging point. The waste is collected in the floating collection pontoon, where a crane transfers the waste into a container.

As there is no direct access to this floating platform and the crane, a remote solution was needed. The crane is equipped with VR technology, but since it is essential for the operator to see his physical crane controller at all times, VR headsets were not an option. Mr. Watts therefore helped DEME to discover how the HoloLens can provide the perfect experience.

Controlling the crane with HoloLens 2

The operator needs an accurate sense of depth to control the crane and must be able to see his physical crane controller at all times. Being a Mixed Reality device, the HoloLens 2 allows for pin -sharp stereoscopic projection of the 3D camera image while still giving the operator a view of his physical remote control and real-life surroundings.

Also, the HoloLens is a fully standalone, untethered headset. It can be used from anywhere, in every office or any location in the field, as long as a good network connectivity is provided. No need for an additional laptop or desktop computer. Having this functionality running on a HoloLens proves to be a very convenient and portable solution.

Mr. Watts developed a custom HoloLens application that allows DEME to control the crane without being physically nearby. With the help of the stereoscopic camera, the app visualizes the actions of the crane and its environment in a true 3D livestream, projected on a holographic hemisphere.

"We always wanted to be able to control the crane from anywhere in Belgium. The HoloLens is a very easy solution. You simply put the device on your head and you see what you would normally see if you were inside the crane’s cabin"

Jan Van de Weghe – Project Manager at DEME Environmental Contractors.

End-to-end latency

Since there is no room for errors, the end-to-end latency (the time between making a single video frame, streaming it, and showing on the HoloLens) of the stream was highly important and a determining factor to give the project a ‘No‘ or ‘Go‘. By using a mix of the industry standard streaming protocols and a carefully chosen combination of the videocodec, container and transport parameters, we were able to achieve a latency of around 150 to 200 milliseconds! Generally, anything below 400 milliseconds is considered “real-time”.

However, what makes the project truly robust is the secondary “controller” mechanism keeping track of this latency. Should the stream slow down and be interrupted due to network issues, for example, a warning (or in elaborated stages even an alarm) appears on the screen to instruct the operator to pause his process until the latency returns to normal values.

“We started by controlling the crane from the pontoon, then we moved the operator to the shoreline, and now we want to upgrade the solution so we can control the crane from anywhere, even from our home office”, says Jan Van de Weghe.

Interested in knowing more about the technical setup? We told all about it at Microsoft Ignite 2020.

Project highlights

0
Months of hard work
0
Boat trips
0 +
Coffees brewed
0 k
Budget

Your project up next?

Fill out the form for a first conversation.
We will contact you within the next business day to set up a call.

    HoloLens Value Canvas

    Do you wish to brainstorm internally about the possibilities of the HoloLens? We’ve got you covered! Download our easy step-by-step guide for implementing the HoloLens in your organization.