Slide 1

Case study

IKEA | Altitude Picking Support

Ikea

Altitude Picking Support

Ikea, the world’s leading home furnishing retailer, has many distribution centers all over the world, where forklift picking is one of the main daily tasks. Forklift picking on higher altitude is difficult as the operator doesn’t have a good view on the exact position of the fork.

Lifting order picking to the next level

During an ideation workshop, many ideas came to the surface quickly. However, we decided to focus on one idea first. Namely to lift order picking to the next level; improving ergonomics, efficiency and creating a safer environment. 

Reach truck drivers collect over 200 items per day. Knowing that racks go up to 9 meters high, many of those items are stored at a high altitude. Ikea saw three opportunities to improve their current way of working.

1

Improve ergonomics

As racks go up to 9 meters high, order pickers are looking upwards a lot of the time. What if this wasn’t necessary? 

2

Safer environment

As it might be difficult to clearly see pallets at high altitudes, sometimes racks get damaged and stability can’t be assured. When this happens, the entire aisle must be unloaded to do a safety check. What if this could be prevented? 

3

Boost efficiency

Ikea assumes that when orders are picked on ground level, employees can work easier and more efficiently. Therefore, they want to investigate whether an innovative solution allowing order pickers to see all racks as if they were on ground level, can boost confidence and efficiency.

Implementing the solution

Because of our experience with real-time stereoscopic 3D video combined with the HoloLens in the past (solution for DEME), we knew such a solution could suit Ikea as well. By attaching a 3D camera to the fork of the reach truck, and streaming a 3D video to the HoloLens, we would be able to bring the eyes of the reach truck driver at the same height of the fork – no matter the altitude.

The idea is simple, but the implementation requires some further explanation. To stream the video from the 3D camera to the HoloLens, we needed two things: (1) a steady network connection and (2) a micro-computer (broadcasting the video). In fact, we created a mini network fitted on a single reach truck – a closed system – of a ruggedized ethernet cable (running through the mast of the machine) connecting the HoloLens via a dock to the micro-computer, which is in turn connected to the 3D camera.

To make the solution as clean as possible, we decided to retrofit the reach truck (meaning make some small adjustments to the truck) in order to include the cables into the pulley system of the reach truck. Therefore, we are working closely together with Toyota, the manufacturer/supplier of the reach trucks at Ikea.

Ultra-low latency

To work confident and without any mistakes, a low latency is in order. This is the time between the camera that records the 2 videos (left eye and right eye), and the moment it is visualized in full HD in the HoloLens. We were able to get a latency between 100 and 130 milliseconds – which is extremely fast! To give you an idea:

  • Perception of real-time: 400 milliseconds
  • Ultra-low latency: 200 milliseconds
  • Ikea solution latency: 100 – 130 milliseconds

Phase 2

In phase 2 of this project, we will exploit the stereoscopic vision of the 3D camera and the HoloLens. By means of AI and Computer Vision, we will support the reach truck drivers with – well… let’s call it “parking sensors”. When picking furniture or pallets out of the rack, we will provide the reach truck drivers with extra visual assistance such as a green overlay when the alignment is correct, and real-time distance measuring from the fork to the rack.

HoloLens 2 making it all possible

The HoloLens 2 allows for pin sharp stereoscopic projection of the 3D camera image. This is very important as the operator requires an accurate sense of depth in order to control the reach truck.

Besides that, it is essential for the reach truck driver to still see his real environment as he’s driving around the warehouse or simply moving forward or backward just a little bit. As the HoloLens is a Mixed Reality device, adding virtual information on top of the real world, the device can be safely used in a warehouse environment.

Project journey

  • Inspire: Ideation workshop​

    We organized an Ideation Workshop with different stakeholders from Ikea. We started by explaining what smartglasses are and why we focus on the HoloLens. By doing so, the HoloLens' capabilities were discussed as well.

    The most relevant ideas were used to fill out our HoloLens Value Canvas. That way, we immediately got a good sense of the feasibility of the project, and the added value it would bring.

    Curious what such a workshop would look like for you? Find out more.

  • Think: processing input

    We organized intake interviews with reach truck drivers to learn what is important for them. The entire solution was built based on their input.

    For phase 2, we will define where the computer vision processing will take place. On the HoloLens device itself, on an external computing unit mounted on the forklift (example: Jetson Nano), or in the cloud.

  • Build: STEREOSCOPIC VIDEO

    Based on the input from the reach truck drivers, we built the following functionalities: (1) low latency video streaming, (2) a latency alarm, both auditive and visual, (3) a settings screen that allows for personalized options such as making the screen smaller or bigger, closer or further, adjusting the brightness or contrast, ... . All settings are remembered by the device on an account-level. By scanning the iris of the reach truck driver, the HoloLens knows which settings to enable.

Project highlights

0
Months lead time
0
Orders picked
0
Big obstacles countered
0 k
Budget

Reach out to Niels
for more information

Fill out the form for a first conversation.
We will contact you within the next business day to set up a call.

    HoloLens Value Canvas

    Do you wish to brainstorm internally about the possibilities of the HoloLens? We’ve got you covered! Download our easy step-by-step guide for implementing the HoloLens in your organization.