Authors: Lars Cimber, Studio Lead, Jayway Copenhagen, Johan Månsson (AR/VR expert) and Claus Simonsen (senior iOS engineer).
Apple released iOS 12 for developers in June. As usually, a range of new features and optimisations were announced. Now, having had a chance to take a deeper look, and experiment, what we are most interested in this post is not so much the end-user features of the operating system. Rather this post focuses on what the new developer possibilities means for the end-user solutions we can create with AR Kit 2 which comes with significant improvements in iOS 12. Since June we have discussed implications with current and prospective customers and gained important ideas and insights as to what will deliver value. Two key areas are getting awakening interest. One is the use of augmented reality (AR) to aid customer decision making and e-commerce. The other is educational or training with 3D models. Let’s take a closer look on how value can be offered to end-users.
ARKit 2 – what it brings
After the first version of ARKit released last year, a few super interesting apps appeared. IKEA Place allowed users to put furniture on flat surfaces, and and UJet’s AR app showed how customers can do configuration for their electric scooter. Apple has in the latest version of ARKit really lifted the toolchain to efficiently work with 3D models in mixed reality settings and for users to share these experiences. Now, with the new file format USDZ created with Pixar, and the Adobe Project Aero, the creation of 3D objects and importing them into an AR app for visualization is easier than ever before. The file format also enables a form of quick-view of AR files, much as we know it from using Mac operating systems, and this can be done via e.g. Messenger and mail. Next, with 3D object detection, an iOS 12 device can now detect 3D objects in the real world and add a virtual overlay on top of them to augment the experience. Also, there is now improved face tracking with tracking of over 50 muscle movements. Finally, persistence and shared AR experiences were introduced, meaning virtual objects can be placed in an augmented space and “exist” there for other users to view on their devices, or you can leave them there to admire or work on later.
AR for e-commerce
Let’s illustrate the value of this with a couple of scenarios. Imagine the case of interior design where you want to explore what is the best option for sun-shielding. Options are plenty; curtains, blinds, plissé curtains… well Pinterest can bring on plenty of more suggestions. But how will any of these look in my home? That Pinterest cannot provide (yet). Luckily with ARKit 2 and the right 3D models, trying out objects can be done virtually in your home. We can now create an app with 3D models of various available options along with the permutations of their size, color, and much more. And with the measurement capabilities of of ARKit the app could be built to obtain sizes for best fit for a window based on matching with the properties of the 3D models. A great way to get a much more realistic feel of the product in its use space! Adding to this, the new ability to scan and persist the environment will enable us to share the AR scene created. For instance, I could share the AR scene with you, and you could then view and work further on the sun-shading options for the window. Great for discussing and agreeing interior options! When happy, the selected sun-shading could of course be made so it would be ordered straight from the app.
Above is an experience and relevance webshops today cannot match – and a reason why so many interior design decisions needs to be vetted in a physical store before the order is made.We believe AR can dramatically reduce time to purchase, and create superior conversion rates, and that relevant cases spans all types of furniture and interior artifacts, windows, doors, art, consumer electronics and appliances, and other home and office artifacts.
AR for instructions and training
Training using AR has also been gaining traction. With ARKit 2 and 3D object detection, we are on the verge of creating next-generation interactive and on-site training and education. Imagine how many maintenance operations take place in a large facility management operation or in a production environment every day on different components. Not all operations may be well known and either paper manuals or electronic documentation must be accessed for correct procedure. Imagine how 3D object recognition could change this. For instance, let’s say you have a 3D model of a separator unit and have an app built to recognize this through use of ARKit 2. The 3D model and reality would match and the unit would be identified. So, no more search for part number, or search for ensuring correct part for the unit in question, or looking up in pages and pages of a pdf manual on your mobile. Rather, the model and its parts can be identified directly, and if coupled to IoT sensors for the running unit, maybe even tell you directly which part needing maintenance. Not to forget, that correct procedure is followed by interactive instructions, and that the engineer brings the correct spare parts and tools are in correct and efficient order.
Check out our video on AR and VR to learn more about the magic of augmented and virtual reality.
Contact us in Jayway, or come visit our labs to brainstorm and learn how the new AR possibilities of iOS 12 can bring value to your customers and business!
At Jayway we love creative technologies, and AR is surely one of them.
