Hello!
I had this post in draft mode for while, and just today @Rfog asked me on Twitter about the speed (FPS) on the Hololens. So I’ve tweaked some post to comment as Hololens work Apps.
Hololens Device
Let’s start from the beginning, Hololens has everything we can found for standard Windows 10 device:
- 579 grams
- Uses a micro USB port for charginf
- 2 to 3 battery hours
- 32-bit Intel chip
- 2GB RAM (with an additional 1GB of RAM for the HPU)
- 64GB Flash storage
- 2MP frontal camera
- Video recording at 30FPS – 720p
- Bluetooth 4.0 and Wifi
On the HPU, the best is to refer what was shared in The Register
HPU is a TSMC-fabricated 28nm coprocessor that has 24 Tensilica DSP cores. It has around 65 million logic gates, 8MB of SRAM, and an additional layer of 1GB of low-power DDR3 RAM. That RAM is separate to the 1GB that’s available for the Intel Atom Cherry Trail processor, and the HPU itself can handle around a trillion calculations per second.
Tom Warren, from The Verge, had exclusive access to a “piece by piece” of Hololens. If anyone is interested in more details, the following 2 minutes are essential.
By the way, after seeing this, there is no doubt that Hololens is a piece of art!
Hololens Holographic Features
And now let’s talk about some of the holographic capabilities of the device. One of the most frequent complaints is the small size of the FOV. FOV represents the Field of View, which is the point of view of the user. Here for each eye, we have a 720p resolution, or what is the same 1268 × 720.
The main difference when compared to other devices to Hololens complement dedicated to Virtual reality, is that this 2nd group of devices require work to a few 90FPS or more to make “realistic” experience. With less FPS are symptoms of dizziness, disconnection, etc.
The Hololens scenario is different. As the user sees reality through the lens he is alwasy connected to the reality; holograms are projected in the lens, so that’s why the device avoid these feelings of dizziness or lost of connection. And this is why it is possible to lower the FPS for Apps. Microsoft recommendation is that Apps in Hololens work at 60 fps. And here we must also understand a little how the Hololens to understand this concept.
HoloLens continuously calculates the position and orientation of the head of the user in relation to its surroundings. When an App begins to prepare the next frame to be projected, Hololens predicts where the user’s head will be in the future at the exact moment that the Frame will be displayed on the screens. Based on this prediction, the system calculates the view and projection for that Frame. Here is where the HPU comes into play, since it is responsible for all this work.
Holograms Interaction Distance
Another important detail is the distance which the holograms are projected / displayed. In the Design Guide and Principles for Mixed Reality (see references) we found a nice and very detailed explanation:
Interaction with holograms presents your best experience between 1.25 m and 5 m.
2 meters is the optimal distance, and the experience will degrade if we come up less than 1m. Less distance we will not see holograms, or see them “cut”. The design recommendations aim to use techniques of fading out or clipping in these scenarios.
Hololens Audio
Audio time ! This is a very powerful topic in Windows Holographic and not mentioned very often..
In HoloLens, there is an audio engine which completes the experience of mixed reality through the simulation of sound 3D by environmental simulations, distance and direction. This is known as SPATIAL SOUND
When we use Spatial Sound in an App, this locates us developers sounds in a 3 dimensions space all around the user that uses the Hololens. Sounds will then appear as if it came from real physical objects or holograms of mixed reality in a user environment. I personally think that the Spatial Sound helps to create a much more credible experience and immersive.
This is an interesting topic to read and learn. Analysing how much sound reaches our ears, our brain determines the distance and the address of the object emitting the sound. HRTF or Head Related Transfer Functions, allows you to simulate this interaction that characterizes as an ear receives sound from a point in space. Spatial Sound uses custom HRTFs to extend the mixed world holographic experience and simulate the sounds coming from different directions and distances.
Greetings @ Toronto
El Bruno
References
- Microsoft Hololens Developers
- The Verge
- The Register, Microsoft’s HoloLens secret sauce: A 28nm customized 24-core DSP engine built by TSMC
