Hiya!
We're working on upgrading a complex, interactive HoloLens (vanilla) project to HoloLens 2, and as part of the upgrade, our client has requested image tracking support be added to the application.
For background information, we're very experienced both with HoloLens development and with implementing Vuforia-based image tracking into projects. We have successfully created several multi-platform projects which use Vuforia-based image tracking, including one in HoloLens that uses a version of the 'Vuforia HoloLens Sample' project available on the Unity Asset Store in early 2018.
As an organisation, we specialise in inventing novel ways of conveying a feeling to an audience, so we are well-accustomed to working with experimental technologies, and we feel that this is a challenge that can be solved.
Mixed Reality Toolkit (MRTK) provides a set of components and features to accelerate cross-platform MR app development in Unity. microsoft/MixedRealityToolkit-Unity. Download Option 1: NEW Unity Package Manager MRTK is now available using the Unity Package Manager. For more information, please see Mixed Reality Toolkit and Unity Package Manager. Mixed Reality Toolkit (MRTK) provides a set of components and features to accelerate cross-platform MR app development in Unity. microsoft/MixedRealityToolkit-Unity Download Scroll down to Assets and download to a known location (typically, this is the Downloads folder on Windows 10): Microsoft.MixedReality.Toolkit.Unity.Foundation.2.3.0.
Our first approach was to attempt to port the project into a framework created using the 'Vuforia HoloLens 1+2 Sample' project available on the Unity Asset store, however we found that this sample project is not capable of supporting our client's project, for the following reasons:
Explore building blocks for various types of interactions and UI controls that support HoloLens 2's articulated hand input. Find more on Mixed Reality Toolki. Option 1: NEW Unity Package Manager MRTK is now available using the Unity Package Manager. For more information, please see Mixed Reality Toolkit and Unity Package Manager.
- It is not based on any version of the HoloLens MRTK v2.x plugin provided by Microsoft, therefore does not include many of the key features we want to leverage from the MRTK v2 for HoloLens 2 development, essentially tying us to an already deceased development pathway.
- It is based on a rudimentary version of the HoloToolKit which does not contain several key features available in the HoloToolKit as of early 2018, such as various interaction, spatialisation & networking functionality, each of which is required by our project.
- It is sufficiently distinct from later 2018+ versions of the HoloToolKit (and even more distinct from the current MRTK v2.3) that no clear upgrade path exists to implement image target tracking into our existing HoloLens project using the sample project, or into a HoloLens 2 variant of the project made using MRTK v2.x.
- It does not allow us to use our own, fully featured & heavily customised version of the HoloToolKit, nor does it allow us to use our preferred version of the MRTK v2.x.
Our experience of the Vuforia HoloLens sample seems to echo the general consensus of the reviews for that asset - that it is not fit for purpose when working with HoloLens 2.
As an alternative approach, we spent quite some time trying to Frankenstein some version of Vuforia with versions of the MRTK as well as with our customised version of the HoloToolKit, but we have been unable to get any of these proposed frameworks into a workable condition.
The most common issue we encountered during these experiments was that Vuforia simply would not work. Instead it would silently fail, despite using identical code that worked as expected on other platforms such as iOS and Android, using the same project. In two of these attempts we were able to get Vuforia working on the HoloLens using our customised version of the HoloToolKit, however the HMD lost all ability to orient itself within its virtual space as soon as our test image target was detected, making the framework entirely unusable.
As yet we have been unable to get Vuforia working at all on the HoloLens 2 using the MRTK versions 2.2 & 2.3, using a more conventional approach of importing Vuforia via Unity's Package Manager or from an Asset Store package, and then adding VuforiaBehaviour + DefaultInitializationErrorHandler components to the Main camera, and placing an image target in the active scene.
What we need:
Ideally, we need to be able to use an encapsulated version of Vuforia - as one would with a Vuforia-enabled project on any other platform, so that we can use whichever version of the MRTK or HoloToolKit best suits the needs of the project, and upgrade that module as needed, completely independently from the Vuforia module. Every other plugin we're using allow this form of encapsulation, and we see no reason why this should not also be true for a Vuforia plugin as it merely requires a few additional components placed onto the Main Camera, and an image target.
Mtk Download Apk
So we have two questions:
1) For PTC/Vuforia: Is there an existing timeline for an official release of a functional Vuforia plugin for Unity-based HoloLens 2 development which can be used in conjunction with independently-sourced versions of the MRTK v2.x released by Microsoft? If so, what kind of release date should we expect for this plugin?
Github Mixed Reality Toolkit
2) For other developers using the MRTK v2.x and HoloLens 2: Has anyone else managed to get Vuforia working with any version of the MRTK v2? If so, any insight you can provide would be invaluable!