Hi !
HoloToolkit has 2 main assets to perform a Spatial Mapping scan in an environment with the Hololens.
- Spatial Mapping
- Spatial Understanding
Spatial Mapping is the asset that we use by default and is the one which allows us to have basic control over the scan process performed by the Hololens. For example, once we add this element to an Unity3D project and we enable the drawing of the Mesh, we can see in real time the scan of the environment where while we are using it.
Spatial Understanding allows us to go one step further and perform more complex tasks. For example:
- Define the start and end time for the scan process. This is really useful for example, when we need to perform an initial scan and then we know that the environment will not change. If we disable the scan in real time, we give more resources to Hololens to be used on other tasks
- Perform simple queries on different elements in our point of view. For example, to search for flat surfaces in a wall or in the floor to position holograms.
- Perform more complex queries, to search for items such as a chair, a guitar, etc. I have not used this feature yet, but I have it in in my to do list for when I need it.
Important: Whenever we use Spatial Understanding, we must have added the Spatial Mapping prefab, if we don’t do this we find see some very funny errors.
Personally I think that the 2 best ways to learn about how Spatial Understanding are as follows
- Read Jeff Evertt article, Case study – Expanding the spatial mapping capabilities of HoloLens, he comments on how they created this asset for specific scenarios that where nor contemplated in the original HoloToolkit Spatial Mapping.
- Review the sample Scene located in “HoloToolkit-Unity/Assets/HoloToolkit-Examples/SpatialUnderstanding/SpatialUnderstanding-FeatureOverview/Scenes/“.
It uses Spatial Understanding to perform an initial scan of the atmosphere and then present some options of simple queries.
If you also want it to use in a local project, the next tutorial may help you. The end result is an App that shows in real time a HUD with the status of the scan and at the end disable the visualization of the Mesh.
Easy steps.
- Create a 3D project in Unity3D
- Import HoloToolkit package
- Configure project to
- Support HoloLens projects (UWP, VR, etc)
- enable Spatial Mapping feature
- Clean Scene elements
- Add
- Hololens Camera
- Cursor With Feedback
- Input Manager
- Spatial Mapping
- Spatial Understanding
- FPS Display
- Add Empty element
- Rename to CodeManagers
- Add new C# Script named “GameStartScanner.cs”
The final project may seems similar to this one
You can download the complete source code from GitHub (link).
The script for the new class is below, and it is best to check a couple of interesting notes about the same
- MinAreaForComplete, MinHorizAreaForComplete, MinWallAreaForComplete properties defined the minimum area to be scanned to give the process how valid
- In the Start() function starts the Scan process with operation SpatialUnderstanding.Instance.RequestBeginScanning (); and we also subscribe to us to state changes
- The DoesScanMeetMinBarForCompletion property is that is responsible for validating if the scan process is finished
- In the Update 2 major operations are performed
- The status of the scan process is displayed in a FPS Text
- If the minimal elements, it has been found we finished the scan process, and disable visual scan process
Sample Code
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using System; | |
| using HoloToolkit.Unity; | |
| using HoloToolkit.Unity.SpatialMapping; | |
| using UnityEngine; | |
| using UnityEngine.UI; | |
| public class GameStartScanner : MonoBehaviour | |
| { | |
| public float MinAreaForComplete = 30.0f; | |
| public float MinHorizAreaForComplete = 20.0f; | |
| public float MinWallAreaForComplete = 5.0f; | |
| private bool _scanComplete = false; | |
| public TextMesh DebugDisplay; | |
| void Start() | |
| { | |
| SpatialUnderstanding.Instance.ScanStateChanged += Instance_ScanStateChanged; | |
| SpatialUnderstanding.Instance.RequestBeginScanning(); | |
| } | |
| private void Instance_ScanStateChanged() | |
| { | |
| if ((SpatialUnderstanding.Instance.ScanState == SpatialUnderstanding.ScanStates.Done) | |
| && SpatialUnderstanding.Instance.AllowSpatialUnderstanding) | |
| { | |
| _scanComplete = true; | |
| } | |
| } | |
| void Update() | |
| { | |
| if (DebugDisplay != null) | |
| DebugDisplay.text = PrimaryText; | |
| if (_scanComplete || !DoesScanMeetMinBarForCompletion) return; | |
| SpatialUnderstanding.Instance.RequestFinishScan(); | |
| _scanComplete = true; | |
| // hide mesh | |
| var customMesh = SpatialUnderstanding.Instance.GetComponent<SpatialUnderstandingCustomMesh>(); | |
| customMesh.DrawProcessedMesh = false; | |
| SpatialMappingManager.Instance.DrawVisualMeshes = false; | |
| } | |
| public string PrimaryText | |
| { | |
| get | |
| { | |
| // Scan state | |
| if (SpatialUnderstanding.Instance.AllowSpatialUnderstanding) | |
| { | |
| switch (SpatialUnderstanding.Instance.ScanState) | |
| { | |
| case SpatialUnderstanding.ScanStates.Scanning: | |
| // Get the scan stats | |
| IntPtr statsPtr = SpatialUnderstanding.Instance.UnderstandingDLL.GetStaticPlayspaceStatsPtr(); | |
| if (SpatialUnderstandingDll.Imports.QueryPlayspaceStats(statsPtr) == 0) | |
| { | |
| return "playspace stats query failed"; | |
| } | |
| // The stats tell us if we could potentially finish | |
| if (DoesScanMeetMinBarForCompletion) | |
| { | |
| return "When ready, air tap to finalize your playspace"; | |
| } | |
| return @"Bruno it's time to walk ! | |
| Move around and scan in your playspace"; | |
| case SpatialUnderstanding.ScanStates.Finishing: | |
| return "Finalizing scan (please wait)"; | |
| case SpatialUnderstanding.ScanStates.Done: | |
| return "Scan complete – Now go back to work!"; | |
| default: | |
| return @"I'm working, | |
| ScanState = " + SpatialUnderstanding.Instance.ScanState.ToString(); | |
| } | |
| } | |
| return ""; | |
| } | |
| } | |
| public bool DoesScanMeetMinBarForCompletion | |
| { | |
| get | |
| { | |
| // Only allow this when we are actually scanning | |
| if ((SpatialUnderstanding.Instance.ScanState != SpatialUnderstanding.ScanStates.Scanning) || | |
| (!SpatialUnderstanding.Instance.AllowSpatialUnderstanding)) | |
| { | |
| return false; | |
| } | |
| // Query the current playspace stats | |
| var statsPtr = SpatialUnderstanding.Instance.UnderstandingDLL.GetStaticPlayspaceStatsPtr(); | |
| if (SpatialUnderstandingDll.Imports.QueryPlayspaceStats(statsPtr) == 0) | |
| { | |
| return false; | |
| } | |
| var stats = SpatialUnderstanding.Instance.UnderstandingDLL.GetStaticPlayspaceStats(); | |
| // Check our preset requirements | |
| if ((stats.TotalSurfaceArea > MinAreaForComplete) || | |
| (stats.HorizSurfaceArea > MinHorizAreaForComplete) || | |
| (stats.WallSurfaceArea > MinWallAreaForComplete)) | |
| { | |
| return true; | |
| } | |
| return false; | |
| } | |
| } | |
| } |
Greetings @ Toronto
El Bruno
References
- GitHub, HoloToolkit
- GitHub, HoloToolkit Unity
- Windows Dev Center, Case study – Expanding the spatial mapping capabilities of HoloLens
- El Bruno, How to Import the HoloToolkit Unity
- El Bruno, How to place a Hologram using AirTap and HoloToolkit
- El Bruno, Creating a menu with options with HoloToolkit
- El Bruno, Using voice commands to display a menu with HoloToolkit
- El Bruno, How to create a 3D text always visible using HoloToolkit
- El Bruno, How to create a HUD (3D text always visible without HoloToolkit)
- El Bruno, How to detect hands using HoloToolkit
- El Bruno, Windows 10, Xbox One Controller, Bluetooth and some lessons learned
- El Bruno, How to use Fire Buttons actions with an XBoxOne Controller
- El Bruno, HoloToolkit compiled packages for Unity3D in GitHub
- El Bruno, How to detect AirTap and Click actions using HoloToolkit
- El Bruno, Detect user hand interactions using #HoloToolkit
- El Bruno, Moving and rotating Holograms using an XBoxOne Controller
El Bruno 
Can you detect custom shape via Holo Toolkit. Can you please help me on it
LikeLike
Hi Vipin
Vuforia could help you to do this > https://www.google.ca/search?client=opera&q=vuforia+and+hololens&sourceid=opera&ie=UTF-8&oe=UTF-8
Regards
-Bruno
LikeLike
Hi Bruno, thanks for posting this, it’s been really useful. Quick question about the DoesScanMeetMinBarForCompletion property implementation though. Does the playspace not need to be greater than all of the minimums (lines 104 & 105) rather than any of them? I.e. should the operator not be a logical AND rather than an OR? – Cheers, Jon
LikeLike
Hi Jon
Yes, I just realize that the validation code can be improved there.
Sorry, I’m a lazy coder, I totally relies on compiler optimization for this.
Feel free to improve this, and please share the results !
Regards
/Bruno
LikeLike