#Hololens – Developer Resources for #Hololens2

Hi !

Hololens 2 are on the way to some selected group of developers, so it’s time to start to collect and group some dev resources for the not so near future.

Let’s start with a couple of tweets

There is also a new Mixed Reality Toolkit, available as usual in GitHub https://microsoft.github.io/MixedRealityToolkit-Unity/README.html

What is the Mixed Reality Toolkit

MRTK is a Microsoft driven open source project.

MRTK-Unity provides a set of foundational components and features to accelerate MR app development in Unity. The latest Release of MRTK (V2) supports HoloLens/HoloLens 2, Windows Mixed Reality, and OpenVR platforms.

  • Provides the basic building blocks for unity development on HoloLens, Windows Mixed Reality, and OpenVR.
  • Showcases UX best practices with UI controls that match Windows Mixed Reality and HoloLens Shell.
  • Enables rapid prototyping via in-editor simulation that allows you to see changes immediately.
  • Is extensible. Provides devs ability to swap out core components and extend the framework.
  • Supports a wide range of platforms, including
    • Microsoft HoloLens
    • Microsoft HoloLens 2
    • Microsoft Immersive headsets (IHMD)
    • Windows Mixed Reality headsets
    • OpenVR headsets (HTC Vive / Oculus Rift)

Input Simulation Service

The Input Simulation Service emulates the behaviour of devices and platforms that may not be available in the Unity editor. Examples include:

  • HoloLens or VR device head tracking
  • HoloLens hand gestures
  • HoloLens 2 articulated hand tracking

As usual, a video is the best way to describe this

Unreal Engine 4.22 released

New: HoloLens Remote Streaming Support

Unreal Engine 4 now supports Holographic Remoting through the Windows Mixed Reality plugin! This allows Unreal applications to run on a Windows desktop PC and stream the rendered result wirelessly to HoloLens over a Wi-Fi connection in real time.

Happy coding!
Greetings @ Toronto

Resources

Advertisements

#Hololens – After the huge announcement of #Hololens2, here are some interesting resources

 

 

 

Hi !

The official launch of the Hololens 2 was a couple of days ago during MWC 2019. Even if I miss my days supporting Avanade booth in the MWC, we are incredibly connected today, so I feel like I was there during the launch. So I may recap some resources to be used in the near future in an amazing surprise.

Main Microsoft Resources

Of course, let’s start with the official tweet announcement

The official video

The official post in Microsoft Blog: Microsoft at MWC Barcelona: Introducing Microsoft HoloLens 2

Some tech specs, https://www.microsoft.com/en-us/hololens/hardware

And from the original tweet some amazing Hololens 2 facts

  • #HoloLens2’s Field of View has increased by more than 2X.
  • With direct manipulation, you can now touch your holograms.
  • You can now log into your #HoloLens2 with Iris Recognition.
  • The new universal fit system works for everyone.
  • #HoloLens2’s front enclosure is made entirely out of carbon fiber.
  • #HoloLens2 will launch with a suite of solutions. Today we’re announcing Microsoft Dynamics 365 Guides.
  • Introducing #Azure Remote Rendering. A cross platform service that supports ARKit, ARCore, and #HoloLens. The birth of the internet of holograms. Leverage the power of Azure to directly stream high polygon content with no decimation to #HoloLens2.
  • We’re committed to openness.
    • We believe in an open app store model. Third party app stores are welcome within our ecosystem!
    • We believe in an open web browsing model. Firefox will be joining #HoloLens2.
    • We’ll continue to steer open standards such as Kronos & OpenXR.

And the full MWC session

Client Stories

 

Tech Coverage

  • The Verge

Happy coding!

Greetings @ Burlington

El Bruno

#WindowsML – Create Native AI apps for #Hololens #Windows10

Hi!

A couple of weeks ago I wrote about a new feature in Windows 10 to be able to use ML models natively in W10 Apps. For me, that would mean a breakthrough in the entire Windows 10 Device ecosystem. In example, as we already know, the new version of Hololens incorporates a chip specially dedicated to AI tasks (DNN specifically),
so I assumed that in Hololens V2 we could start using Windows ML using the device capabilities.

What I did not expect is that in the Preview version of Windows 10 RS4 for Hololens we already had access to that feature. I have not had time to update my Hololens to RS4, however, a couple of community experts have shared more than interesting experiences.

For example, let’s see this video by Rene Schulte, where he shows an App that performs a real time image classification using the SqueezeNet ONNX model

Mike Taulty also offers a series of posts where he does an excellent research job in Windows ML and Hololens. He started on how to create a model using Azure Custom Vision, export it to CoreML and then create an ONNX model.

Important: currently ONNX is the type of models supported by Windows ML. It is possible to export CoreML models, TensorFlow, CNTK and other frameworks to ONNX.

From here, Mike comments on how to work with UWP and C # with the exported model. In subsequent posts, Mike also explains how, thanks to Hololens RS4 Preview, these capabilities can also be used in applications for Hololens (see references)

As well, If you dare to upgrade your Hololens to RS4 Preview, it is a very interesting moment to start thinking about scenarios where you can combine WinML and Hololens!

Happy Coding!

Greetings @ Toronto

El Bruno

References

#WindowsML – Ya se puede crear apps con AI nativo en #Hololens #Windows10

 

Buenas!

Hace un par de semanas comente que la nueva capacidad en Windows 10 de poder utilizar modelos de ML de forma nativa en W10 Apps significaría un gran avance en todo el ecosistema de Devices de Windows 10. Como ya sabemos que la nueva version de Hololens incorporar un chip especialmente dedicado a tareas de AI (DNN específicamente), pues era de suponer que en Hololens V2 podríamos comenzar a utilizar Windows ML.

Lo que no esperaba es que en la version Preview de Windows 10 RS4 para Hololens ya tuviésemos acceso a esa capacidad. No he tenido tiempo de dedicarle a mis Hololens para actualizarlas a RS4, sin embargo, grandes referencias han compartido experiencias más que interesantes.

Por ejemplo, veamos este video de Rene Schulte, donde muestra una App que en tiempo real realiza una clasificación de imágenes utilizando el modelo ONNX SqueezeNet

Mike Taulty también ofrece una serie de posts donde realiza un trabajo excelente al respecto. En primer lugar, comenta como crear un modelo utilizando Azure Custom Vision, exportar el mismo a CoreML y luego crear un modelo ONNX.

Importante: actualmente ONNX es el tipo de modelos soportados por Windows ML. Es posible exportar modelos de CoreML, TensorFlow, CNTK y otros frameworks a ONNX.

A partir de aquí, Mike comenta como trabajar con UWP y C# con el modelo exportado. En siguientes posts, Mike también explica cómo, gracias a Hololens RS4 Preview, también se pueden utilizar estas capacidades en aplicaciones para Hololens (ver referencias)

Pues bien, si te animas a poner una version Preview en Hololens, es un momento muy interesante para comenzar a pensar en escenarios donde combinar WinML y Hololens!

Happy Coding!

Saludos @ Toronto

El Bruno

References

#Hololens – 29 new countries are added to the Hololens market, total of 39 and is still targeting developers

e4358372c737c7ac307eaebb86108d19-1024x576

Hello!

Today is a quick post, I’m still trying to understand how the Qubits work and this is consuming me a lot of time.

At the Future Decoded event in London, Microsoft announced that Microsoft Hololens will be available in 29 new markets.

The new countries that are added to the list are Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Iceland, Italy, Latvia, Liechtenstein, Lithuania, Luxembourg, Malta, the Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Switzerland and Turkey.

They are added to the markets where they could already be purchased; United States, Canada, Australia, France, Germany, Ireland, New Zealand, the United Kingdom and Japan. Officially the sale of Hololens was formalized also in China, however, it seems that they still can not be obtained in the Chinese market.

In general this is good news. If we consider some similar movements in the past,
we can think that when Hololens V2 is released to the market, it will be in these 39 countries where Hololens V1 is already commercialized.

 

Happy Coding!

Greetings @ Burlington

El Bruno

References

#Hololens – Se agregan 29 nuevos países al mercado de Hololens, en total 39 y se sigue apuntando a developers

e4358372c737c7ac307eaebb86108d19-1024x576

Hola!

Post corto de miércoles que intentar comprender como funcionan los Qubits me está consumiendo mucho tiempo.

En el evento de Future Decoded en Londres, Microsoft ha anunciado que Microsoft Hololens estará disponible en 29 nuevos mercados.

Los nuevos países que se agregan a la lista son Austria, Bélgica, Bulgaria, Croacia, Chipre, República Checa, Dinamarca, Estonia, Finlandia, Grecia, Hungría, Islandia, Italia, Letonia, Liechtenstein, Lituania, Luxemburgo, Malta, Países Bajos, Noruega, Polonia, Portugal, Rumania, Eslovaquia, Eslovenia, España, Suecia, Suiza y Turquía.

Se suman a los mercados donde ya se podían comprar las mismas; Estados Unidos, Canada, Australia, Francia, Alemania, Irlanda, Nueva Zelanda, el Reino Unido y Japón. Oficialmente se había formalizado la venta de Hololens también en China, sin embargo, parece que las mismas todavía no se pueden conseguir en el mercado chino.

En general esto es una buena noticia. Si tenemos en cuenta algunos movimientos similares en el pasado, podemos pensar que cuando Hololens V2 se lance al mercado, será en estos 39 países donde ya se comercializa Hololens V1.

Happy Coding!

Saludos @ Burlington

El Bruno

References

#MixedReality – #Hololens new, #Steam partnership, a new Halo, new Devices, maybe a #Minecraft VR and more!

I2

Hi !

Today I’ll share some updates in the Mixed Reality and Hololens world. Before the big announcement on IFA, there is a good idea related to Hololens:

LimpidArmor presented a hybrid between a helmet and Hololens. This device is integrated with cameras outside a tank to give the Hololens user a 360-degree view of both optical and thermal systems without exposing them to additional risk.

I really like the photo, and if you want to see more pictures and some field tests, their Facebook page have some other materials.

i3

And, after this quick intro, it’s time to share some of the big Microsoft announcements during the IFA keynote on Berlin, Germany.

Microsoft Mixed Reality to get support for SteamVR, Minecraft and Halo starting this holiday season

Wow! In the references section you can read some articles with more details. I’ll try to highlight the more important topics:

  • Microsoft announced a collaboration with 343 Industries to add Mixed Reality Experiences to Halo. I’m a long time Halo fan, so this is amazing for me!
  • As part of the presentation, Microsoft also shared a long term view on which types of devices will be required to use MR capabilities. This is important because usually we think on VR devices and Expensive Ones, and Microsoft took a different approach here. There will be 2 types of devices:
    • Windows Mixed Reality PCs: will consist of desktops and laptops with integrated graphics.  When plugged into these devices, our immersive headsets will run at 60 frames per second.
    • Windows Mixed Reality Ultra PCs: will consist of desktops and laptops with discrete graphics. When plugged into these devices, our immersive headsets will run at 90 frames per second.
  • And, IMHO the best one. Microsoft is partnership with Steam, I may mean SteamVR. You probably know Steam, and they been investing and working in their VR branch for a while. Virtual Reality is not new for them. Using SteamVR you can already play and use several titles for Oculus Right or HTC Vive. I’m sure the path for Mixed Reality will receive some great ideas and maybe more from the Steam group. Also the chance to have some titles in the Steam Store for the new Mixed Reality devices is a very interesting approach.

So, these partnerships, new projects and devices may open the chance to enjoy Minecraft VR in the new Windows 10 Mixed Reality devices. Also maybe in 4K. If you don’t know Minecraft VR, please take a look at this video

Greetings @ Burlington

El Bruno

References

 

#MixedReality – Algo de #Hololens, partnerships con #Steam, un nuevo Halo, nuevos Devices y tal vez un #Minecraft VR!

I2

Hola!

Hoy toca un repaso por un par de novedades relacionadas con el mundo de Mixed Reality y Hololens. Antes de la gran noticia que dio Microsoft ayer, comienzo con una que es bastante interesante:

LimpidArmor ha presentado un casco que integra unas Hololens y se integra con cámaras fuera de un tanque. El usuario del caso, gracias a Hololens tiene una visión de 360 grados tanto de sistemas ópticos como termales sin exponerlos a riesgos adicionales.

La foto es impresionante, y en su página de Facebook se pueden ver un par de fotos con pruebas de campo.

i3

Ahora vamos a la noticia principal después de las declaraciones y anuncios que ha hecho el equipo de Microsoft en la keynote del evento IFS en Berlín, Alemania.

Microsoft Mixed Reality to get support for SteamVR, Minecraft and Halo starting this holiday season

Wow! En las referencias dejo un par de artículos donde se pueden encontrar muchos detalles, pero mejore vamos por partes que hay un par de bullets más que interesantes:

  • Microsoft ha anunciado que están trabajando con 343 Industries para traer a Halo al mundo de Mixed Reality.
  • También se ha presentado un plan de ordenadores para poder utilizar los VR Devices:
    • Windows Mixed Reality PCs: will consist of desktops and laptops with integrated graphics.  When plugged into these devices, our immersive headsets will run at 60 frames per second.
    • Windows Mixed Reality Ultra PCs: will consist of desktops and laptops with discrete graphics. When plugged into these devices, our immersive headsets will run at 90 frames per second.
  • Finalmente el partnership que promete y mucho es el que han presentado con Steam. Si conoces Steam, seguramente sabes que desde hace un tiempo están trabajando en SteamVR. El mundo de realidad virtual no es nuevo para Steam, ya tienen muchos títulos que se pueden utilizar con HTC Vive o con Oculus Rift. Veremos que tal integra la realidad mixta.

Estos partnerships, proyectos y devices nos dan a entender que tal vez no tengamos que esperar mucho tiempo para poder disfrutar de un Minecraft VR en los nuevos devices de Windows 10 Mixed Reality. Y tal vez en 4K. ¿Todavia no sabes que es Minecraft VR? Pues toma un video

Saludos @ Burlington

El Bruno

References

 

#Hololens – Getting Started with #MixedRealityToolkit #MRToolkit

Hi!

So, HoloToolkit is gone (until you see the code on the new toolkit) and now it’s time to start using the new Mixed Reality Toolkit. There are a couple of ways to do this, IMHO the best one is to import a custom package in Unity3D with all the contents of the Mixed Reality Toolkit.

I’ve used to create and maintain my own custom packages for HoloToolkit, however I’ll follow the guidelines and now I’ll start to use the official ones. We can find them in the Release section on the GitHub repository (link). Then we need to click on [Edit / Packages / Custom package] and we have all the assets in our project.

In the releases page

  • We are going to see the status of the latests releases. That’s mean current bug, new features, fixed bugs and more.
  • We will have 3 different packages to download and import in Unity3D
    • Toolkit
    • Toolkit + Tests
    • Toolkit + Tests + Examples
  • We will be able to download the source code of the toolkit (if you don’t want to clone the git repository)

If you already know how to work with the Mixed Reality Toolkit, the 1st package is all you need. On the other hand, if you want to see some examples on how to use it, maybe the 3rd package is the best option. In example, the 3rd package contains several samples, one of them is a interaction demo such as the following animation.

2017 08 15 Holo MRToolkit New Samples 02

Among the innovations included in this version, we can find some very useful such as a keyboard to be used in AR / VR applications. We have a Prefab that we can use in our applications in [Assets / HoloToolkit / UI / Prefabs]

I1

Important: this new virtual keyboard does not automatically appear when you focus or gaze in a TextBox. There is some code required to do this, I”ll write a post in the near future on this.

Other elements to consider are the assets that allow us to manage the transitions between Scenes. In [Assets / HoloToolkit / Utilities / Prefabs] we can find a button to launch the navigation to a new Scene and also a prefab to return to the previous scene.

Finally I have to highlight the gallery of examples that are included in this release. In [Assets / HoloToolkit-Examples] we can find all the samples. At first glance I detected that there are a couple new ones, like the scenes of “Medical” and the scenes of “Prototyping”.

The 2nd one is especially interesting since it shows different forms of interaction with elements in a world of MR. The animation at the beginning of the post corresponds to the “CycleArray” scene.

Happy Holocoding!

Greetings @ Burlington

El Bruno

References

El Bruno, my posts

#Hololens – Empezando con #MixedRealityToolkit #MRToolkit

Hola!

Si bien hay varias opciones, la mejor forma de utilizar Mixed Reality Toolkit en nuestros proyectos de Unity3D es importando un Custom Package. Durante un tiempo, me dedique a mantener un repositorio con los paquetes que iba creando para uso personal. Sin embargo, en el mismo repositorio en GitHub podemos encontrar el listado de Releases (link), donde podremos

  • Ver el estado de la última Release activa, incluidos nuevas funcionalidades, bug solucionados y problemas existentes
  • Descargar 3 paquetes para importar Unity con las siguientes combinaciones
    • Toolkit
    • Toolkit + Tests
    • Toolkit + Tests + Examples
  • Descargar Código Fuente del Toolkit

Si ya sabes cómo manejarte con Mixed Reality Toolkit, el 1er paquete es suficiente. En cambio, si quieres ver algunos ejemplos sobre cómo utilizar el mismo, tal vez el 3er paquete sea la mejor opción. En este último package puedes encontrar ejemplos completos de interacción como el de la siguiente animación.

2017 08 15 Holo MRToolkit New Samples 02

Dentro de las novedades que se incluyen en esta version, podemos encontrar algunas muy utiles como por ejemplo un teclado para aplicaciones AR/VR. Tenemos un Prefab que podremos utilizar en nuestras aplicaciones en [Assets / HoloToolkit / UI / Prefabs]

I1

Importante: el teclado virtual no aparece automáticamente cuando hacemos foco en una caja de texto. Es necesario agregar un poco de código para utilizar el mismo, estoy creando un HowTo sobre esto.

Otros elementos a tener en cuenta son los assets que nos permiten gestionar las transiciones entre Scenes. En [Assets / HoloToolkit / Utilities / Prefabs] podemos encontrar un botón para lanzar la navegación a una nueva Scene y además un prefab para volver a la escena anterior.

Finalmente tengo que remarcar la galería de ejemplos que se incluyen en este release. En [Assets / HoloToolkit-Examples] podemos encontrar los mismos. A simple vista he detectado que hay un par nuevos, como las escenas de “Medical” y las escenas de “Prototyping”. El 2do es especialmente interesante ya que muestra diferentes formas de interacción con elementos en un mundo de MR. La animación del principio del post corresponde a la escena “CycleArray”.

Happy Holocoding!

Saludos @ Burlington

El Bruno

References

El Bruno, my posts