Remote assistance and immersive reality, Dynamics 365 Remote Assist and its combination with Hololens 2

As happened last century with the invention of the telephone, personal computers, the internet and smartphones, we are living through an exponential period, with the appearance of new technologies and services that bring us closer and closer to people and allow us to interact thousands of kilometres away as if we were in person.

Industry, Logistics, Insurance, Renewables... Companies of any sector, nature and size need to promote collaborative and increasingly remote work of their teams and of these with their customers and suppliers. The answer to this challenge of bringing people together to achieve real collaboration, bridging distances, can only be found in technology.

All major technology manufacturers are betting on this type of solutions and proof of this is that Gartner itself, in its annual report Hype Cycle for Emerging Technologies for 2021, already included them among the solutions with the highest growth expectations.

Microsoft is ahead in some of these areas. Most of the solutions offered by Microsoft Dynamics 365 are already well known: Business Central, Finance and Operations, Sales, Marketing, Field Service, Customer Service, Commerce... However, there are others that are not yet so widespread and not many companies still provide these services. Among these, perhaps the one that may arouse more astonishment and have more potential for growth is Dynamics 365 Remote Assist and more specifically, the combination of this with Hololens 2.

Dynamics 365 Remote Assist, new scenario of remote collaboration and technical assistance
Dynamics 365 Remote Assist is the service provided by Microsoft for remote incident resolution. With this service, it is intended to increase collaboration and efficiency in the resolution of incidents with technology that allows technicians to keep their hands free to perform the work or resolution of these incidents without having to be in the place of the same.

This enables companies to provide an optimal service to their external or internal customers by examining any incident without the need for technicians to travel, consulting with experts connected to remote assistance in real time. All of this makes it possible to drastically reduce travel costs while at the same time making a qualitative leap in the quality of the service provided.

Microsoft HoloLens 2
Hololens 2 is the Augmented Reality glasses designed by Microsoft for business use, mainly focused on remote assistance, remote training or incident resolution.

This latest version of the device offers a more comfortable and immersive experience for the user, making the benefits of augmented reality available to any organisation. Augmented reality is already incredibly realistic with unprecedented image resolution. The devices, which began to be sold in Spain at the end of 2020, also incorporate accessories that make the glasses a totally ergonomic product with which we will be able to visualise holograms, 3D elements, writing and written indications in a sustained image on the background...

As a work tool that in many cases can be used for several hours a day, it is a prerequisite that the device is fully ergonomic and lightweight. To achieve this, Microsoft uses a material similar to carbon fibre and has redesigned the product to make it more functional and lighter. The glasses incorporate an adapter to adjust the pressure of the support and the glass of the glasses can be raised to clear the field of vision without having to be removed from the head.

Features of Microsoft HoloLens 2
This latest version also incorporates some features that undoubtedly position it as the best and almost unique solution on the market for these functionalities:

• Spatial Mapping: Detailed representation of the real surface in the Hololens environment.
• Natural Interaction: Interact with the holograms through completely natural gestures.
• Spatial Sound: Perceive sound from different points without isolating yourself from your surroundings.
• Voice recognition: Interact with augmented reality glasses using voice commands.
• Computer vision: Set markers to recognise equipment and locations.
• Hologram sharing: Share hologram displays on other devices no matter where they are.
• Biometric authentication: Simplifies login, increases security and personalises the experience automatically. With this iris tracking, the glasses are also able to identify where the user is looking and respond accordingly by adapting the image.
• Spatial anchors: reference points to which to associate all kinds of information of interest.

So we see how HoloLens 2 is the essential complement to Dynamics 365 Remote Assist to create remote support scenarios and direct real-time communication between the expert and field staff members. Thanks also to the native integration of all 365 applications, we will have the ability to connect multiple users at the same time to the same call or incident, connected for example by Microsoft Teams through video calls video conferencing that put the expert in front of the incident giving you the ability to indicate, through graphical annotations, how to solve the fault.

Advantages of Dynamics 365 Remote Assist + Hololens 2
Dynamics 365 Remote Assist and Hololens 2 open up a new scenario of collaboration in companies, especially in industry, in which we will be able to:

• Solve problems in real time: Technicians can share their real-time view with experts in remote locations to get the help they need, reducing costs and travel time.
• Walk around the location without being there: Remote inspectors can use Remote Assist or Microsoft Teams to assess product quality without having to travel to the site to collaborate with a Remote Assist user.
• Visually communicate complex, detailed instructions: Instead of communicating instructions via an audio call, on-site workers and remote collaborators can use drawings and arrows to refer to specific parts of a machine or asset. These annotations are anchored in the Remote Assist user space.
• Escalate expert knowledge: enable experts to share their knowledge effectively from anywhere in the world.
• Display important information: technicians and inspectors can retrieve work order information from Dynamics 365 Field Service and call the assigned resource for technical support. In addition, technicians and inspectors no longer need to carry printed manuals to perform repair or inspection tasks. If the technician uses Remote Assist in HoloLens, remote collaborators can insert reference images, schematics and other useful information into the technician's physical space so they can refer to the schematic while working without losing sight of what they are doing in HoloLens. If the technician uses Remote Assist on a mobile device, remote collaborators can share reference images, schematics and other useful information that the technician can view on their mobile device.
• Combine video, screenshots and annotations for smoother workflows on the devices you already use: Remote Assist is available on mobile devices that do not support AR.
• Drive Remote Assist in HoloLens with voice: Remote Assist in HoloLens provides a hands-free experience by supporting voice commands in these languages for all functions, even in a noisy, industrial environment.
• Capture assets and automate processes using Microsoft Power Platform: Store mixed reality photos and videos on CDS, then integrate the data with business processes through Power Automate.
• Deliver best-in-class service with a click-to-join Remote Assist call: Support suppliers and customers with a one-click, mixed reality video call they can join with a single click.

Success stories
The best way to understand the potential of the solution is through practical examples, so we would like to reflect here an example of a workflow that Microsoft indicates for technicians and remote collaborators:

1. A technician arrives on site to repair a machine, but encounters a complex machine with which he is unfamiliar. The technician dons a HoloLens 2 device, logs in with iris scanning and launches Dynamics 365 Remote Assist (or launches Dynamics 365 Remote Assist from his mobile device).
2. Because the technician's company has enabled Field Service integration, the technician selects a reservation and sees the recommended remote collaborator who has the most experience with that machine.
3. A remote partner joins the mixed reality call on the Microsoft Teams desktop and sees what the technician sees. They begin recording the call so the technician can use it for future reference.
4. The remote collaborator provides verbal instructions. When he wants to draw attention to a specific part of the machine, he draws an arrow on the PC screen. The arrow immediately appears in the HoloLens user's space and remains anchored to that part of the machine, even when the technician walks. If you want to point to the cable that the technician needs to reconnect, draw along the cable. The line immediately appears in the HoloLens user space. The collaborator adds drawings and arrows on the PC screen. These drawings and arrows appear in the technician's space and remain anchored to the cable, even when the technician walks.
5. The remote collaborator then shares a diagram from OneDrive, which will appear in the technician's space. The co-worker can also open a training video on their desktop and share their screen with the technician.
6. The technician successfully repairs the machine, captures a snapshot of the repaired machine and saves it to OneDrive. He thanks the remote collaborator for his help and ends the call. The snapshots and files shared during the call are posted to the Field Service work order for the technical manager to review, and the recording is saved online. The technician shares the recording with colleagues so they can learn how the technician and remote partner repaired the machine.

And for proof a button! We invite you to see some success stories of real implementation that Microsoft itself shares and that apply to sectors as diverse as healthcare, automotive or the manufacture of building materials.

• Mercedes-Benz USA

Mercedes Benz aims to provide the best customer experience, not only in the design of high-end vehicles but also in the provision of services.

To achieve that goal, the company has chosen the combination of Dynamics 365 Remote Assistance and Hololens 2 to improve the efficiency of the technical service provided at dealerships and official workshops, reduce the time to resolve incidents and breakdowns, and reduce the cost and environmental impact of service-related travel.

• Imperial College Healthcare NHS Trust

The London hospital group wanted to use mixed reality technologies when the COVID-19 pandemic broke out to prevent contagion among its medical staff.

At the beginning of the pandemic, each positive case required four doctors to go into isolated wards to treat patients. In the first two weeks, they quickly saw that no one was prepared to deal with the situation, they did not have the necessary PPE and they saw how quickly a large part of the medical team became infected.

To avoid this, they decided to attend each patient with only one doctor and the rest of the team would assist them remotely through the Hololens 2 and the Dynamics 365 Remote Assist function.

The results were immediate: patient service remained the same and infections fell drastically among the medical staff.

• Saint-Gobain

With more than 350 years, the world leader in the manufacture of building materials, especially glass, relied on the possibilities offered by Microsoft to save time and costs in the resolution of incidents occurring in the 67 countries where the company has production plants.

Some of the processes implemented in glass production require a certain expertise that very few company employees possess. These employees, to date, spent almost all of their work traveling around the globe from plant to plant resolving issues, something that was not sustainable in terms of cost and environment. Making clearer the needs of the workers and how to solve them, the testimony of these experts was that most of these incidents could be solved simply by "having a pair of eyes on the ground".

In this way, the company implemented Microsoft's remote support system for incident resolution, achieving huge savings in travel and salaries as well as an incredible reduction in incident resolution times.

These examples can serve as inspiration for many industries and companies of all kinds that want to improve remote collaborative work, reduce travel and time in the resolution of incidents and promote learning for their employees closer to the reality of work in real production conditions. Although it may seem so, it is not science fiction, it has already been with us for some time and it is Microsoft technology.

If you are interested in learning more about Dynamics 365 or all the news that Microsoft launches, contact us to have all the information you need and the best support in your projects.