#Office – #Clippy is back in Office for Windows and Mac (powered by #AI)

Hi !

So, after my post yesterday, about the suggestions in Outlook based on an email content, somebody asked me why they don’t see this in Outlook.

I was sharing my experience in Outlook for Mac, in Windows we can find a [Suggested Meeting] option on top of the email content.

And then, when we expand the section, we may find the related meeting information

Happy coding!

Greetings @ Toronto

El Bruno

My posts

Advertisements

#Office – Clippy is back in Office (powered by #AI)

Hi !

I’ve already share some of the new AI features in Office, like Excel Ideas or real-time captions / subtitles in PowerPoint.

Today I’ll switch to Office editor, to share something amazing. I added Clippy to this post title and the idea is similar to the Clippy intents a long time ago:

An AI assistant will analyze the text while you are writing it or reading it, and it will suggest actions related to this text.

A single image is the best way to show this

I received this email last week and I notice that part of the text was automatically highlighted and show an amazing suggestion. It seems that the Word, Outlook editor is becoming more and more intelligent.

And I’m really looking forward to some extensibility SDK, so someone can really bring back Clippy at 100%

Happy coding!

Greetings @ Toronto

El Bruno

My posts

#Flow – Detener el salva pantallas utilizando botones Flic, procesos con @microsoftflow, @powershell y mucho mas ! (gracias @ShortcutLabs)

Buenas !

Llevar adelante un Centro de Innovación es una experiencia que es 50% de problemas y 50% de diversión. En estos momentos estamos actualizando nuestro IC en Toronto, y esto significa que tenemos muchas nuevas screens (entre otras cosas) y esto también supone un desafío al momento de pensar en ahorro de energía.

Por suerte, las TVs se apagan automáticamente cuando no hay signal. El desafío era comprender como enviar signal desde dispositivos con Windows 10, y claro, antes de llegar a este punto, pasamos por la experiencia del Screen Saver.

Hay soluciones completas y complejas para manejar estos escenarios de Digital Signage. En nuestro caso, la idea era ver de hacer algo utilizando parte de la tecnologia que conocemos para solucionar este escenario.  @ChloeCondon escribió un par de tweets donde menciono a los Flic buttons, y se me ocurrió que era el momento de probarlos. Inicialmente pensé en este proceso

  • Clic en un botón Flic
  • El boton lanza un MS Flow
  • MS Flow apaga el screen saver

Parece muy simple, ademas la conexion entre MS Flow y Flic esta disponible en Preview! podemos suscribir los eventos clic, doble clic y hold como triggers de Microsoft Flow.

F1 settings for a button

El siguiente paso es crear un Flow para procesar el evento del botón. De alguna manera tengo que procesar el trigger de MS Flow para detener el screen saver en un ordenador con Windows 10. Una forma casi natural de trabajar con MSFlow es crear un archivo en el ordenador con Windows 10. Inicialmente hice algunas pruebas con Microsoft Flow Gateways, que parece ser la herramienta ideal para compartir información entre origenes de datos on premises y MS Flows en el cloud.

Sobre un MS Flow en blanco, agregue el trigger de Flic button. Las settings del trigger permiten elegir el botón y el tipo de acción para la suscripción.

01 microsoft flow flic button trigger02 microsoft flow flic button settings

Llego el momento de configurar Gateways. Y, básicamente, no fue de mi agrado. Path en modo hard-code, lo mismo para guardar credenciales, algo no me parecía una buena idea. Este fue el momento de salir a correr un poco para tener nuevas ideas.

03 microsoft flow gateway file system configuration

Por suerte, ya tenemos un sistema de archivos distribuido, que es seguro y bastante potente. Tambien tengo la suerte de que el mismo esta funcionando en todos los ordenadores de demo de Windows 10: OneDrive. Y como bonus, OneDrive tiene muchos triggers y acciones para trabajar con Microsoft Flow!

Llego el momento de actualizar mi escenario

  • Clic en un botón Flic
  • Lanzar un MS Flow
  • MS Flow crea un archivo en OneDrive
  • De alguna manera esto deshabilita el screen saver

El MS Flow es mas bien simple

04 microsoft flow flic button trigger and save a OneDrive file

Un par de apuntes.

La expresión convertTimeZone() esta pensada para poder tener nombres de archivos unicos para cada nuevo archivo:

convertTimeZone(utcnow(),’UTC’,’W. Europe Standard Time’,’yyyy MM dd HH mm ss’)

El contenido del archivo no es importante (por ahora). He aquí un ejemplo de archivos creados con este Flow.

onedrive files created by flic clic

Esto va tomando forma. Despues de presionar el boton Flic, en pocos segundos, OneDrive sincroniza el nuevo archivo en todos los ordenadores con Windows 10. Es el momento de crear un File System Watcher para detectar los cambios y detener el screen saver. Como era de esperar, mi primera idea fue crear una .Net Core Console App para hacer esto, sin embargo recorde que mis amigos IT Pro hacen magia con PowerShell.

Una busqueda en internet me llevo a leer el siguiente post de Jessica Cook: Flow of the Week: Local code execution, donde cierra un escenario muy parecido al que comento en este post.  Ella si ha utilizado gateways, y yo me dedique a modificar un poco su PS Script:

$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "<Folder to Watch>"
$watcher.Filter = "*.*"
$watcher.IncludeSubdirectories = $true
$watcher.EnableRaisingEvents = $true 
$action = {
Start-Process "taskkill /f /im bubbles.scr"
} 
Register-ObjectEvent $watcher "Created" -Action $action
while ($true) {sleep 5}

Listo! Una vez agregado el PowerShell script al inicio de Windows, los ordenadores de demo, ya estan en modo “sync to stop screen saver”. Y es presionar el boton y ver como se deshabilitan los screen savers!

El proceso

  • Clic en un botón Flic
  • Lanzar un MS Flow
  • MS Flow crea un archivo en OneDrive
  • PS Script para detectar nuevos archivos en un directorio
  • Cuando se detecta un archivo, se detiene el proceso del Screen Saver

Hay mucho margen de mejora, como por ejemplo detener el screen saver, en lugar de matar el proceso del mismo. He intentado con SendKeys y moviendo el mouse con PS, pero lo unico que me ha funcionado es el kill process!

Importante tambien es recordar que esto no es realtime, algunas veces tarda 1 segundo y otras 10 segundos. Lo importante es que funciona y es una buena historia para contar en el IC.

Happy coding!

Saludos @ Toronto

El Bruno

References

#Hololens – Developer Resources for #Hololens2

Hi !

Hololens 2 are on the way to some selected group of developers, so it’s time to start to collect and group some dev resources for the not so near future.

Let’s start with a couple of tweets

There is also a new Mixed Reality Toolkit, available as usual in GitHub https://microsoft.github.io/MixedRealityToolkit-Unity/README.html

What is the Mixed Reality Toolkit

MRTK is a Microsoft driven open source project.

MRTK-Unity provides a set of foundational components and features to accelerate MR app development in Unity. The latest Release of MRTK (V2) supports HoloLens/HoloLens 2, Windows Mixed Reality, and OpenVR platforms.

  • Provides the basic building blocks for unity development on HoloLens, Windows Mixed Reality, and OpenVR.
  • Showcases UX best practices with UI controls that match Windows Mixed Reality and HoloLens Shell.
  • Enables rapid prototyping via in-editor simulation that allows you to see changes immediately.
  • Is extensible. Provides devs ability to swap out core components and extend the framework.
  • Supports a wide range of platforms, including
    • Microsoft HoloLens
    • Microsoft HoloLens 2
    • Microsoft Immersive headsets (IHMD)
    • Windows Mixed Reality headsets
    • OpenVR headsets (HTC Vive / Oculus Rift)

Input Simulation Service

The Input Simulation Service emulates the behaviour of devices and platforms that may not be available in the Unity editor. Examples include:

  • HoloLens or VR device head tracking
  • HoloLens hand gestures
  • HoloLens 2 articulated hand tracking

As usual, a video is the best way to describe this

Unreal Engine 4.22 released

New: HoloLens Remote Streaming Support

Unreal Engine 4 now supports Holographic Remoting through the Windows Mixed Reality plugin! This allows Unreal applications to run on a Windows desktop PC and stream the rendered result wirelessly to HoloLens over a Wi-Fi connection in real time.

Happy coding!
Greetings @ Toronto

Resources

#Flow – Stop a screen saver using Flic buttons, @microsoftflow processes, @powershell and much more ! (Thanks @ShortcutLabs)

Hi !

Running an Innovation Center is half challenge, half fun. Everybody expect to have amazing experiences, and they must also be up and ready all the time! We are now updating / upgrading out Toronto IC, with all the new screens, we have some challenges and the screen savers is a good one.

The scenario is very simple: we have several Windows 10 machines connected to screens, and they go offline automatically until the W10 device screen gets back online. That’s the current scenario, however, before this I was working with the standards Windows 10 screen savers.

And I was wondering if I can find a fun way to disable all the screen savers at once using some kind of cool technology. So, after some tweets from @ChloeCondon where she described a cool scenario using Flic buttons, I decided to give them a try. And in my mind this is the process to follow

  • Click on a Flic button
  • Trigger a MS Flow
  • MS Flow will disable the screen saver

It’s seems easy and the connection between Flic and Microsoft Flow is already in preview! We can connect the events click, double click and hold to Microsoft Flow.

F1 settings for a button

The next step is to create a flow to process the event. Somehow I need to send a signal to the physical Windows 10 device to stop the screen saver. In my scenario, the signal will be the creation or update of a file in the remote machine (this is important, so I bolded it!) My 1st idea was to use Microsoft Flow Gateways, which seems to be the perfect tool to share data on-premise with Flows on the cloud.

So I created a blank Flow and added the Flic button trigger. The trigger settings allows me to select the type of event to hookup in the Flow.

01 microsoft flow flic button trigger02 microsoft flow flic button settings

And now it was time to setup my next step, the amazing Gateways. And, I didn’t like it. Hard coding paths, user names or password is usually a bad idea. And, after some minutes installing the gateway in a single machine, and this configuration screen, I decided to run some KMs and look for other options.

03 microsoft flow gateway file system configuration

Lucky me, I already have a distributed cloud File System, very reliable and secure running in all my demo machines: OneDrive. And OneDrive have tons of triggers and actions available for Microsoft Flow!

So I updated my scenario with the following steps

  • Click on a Flic button
  • Trigger a MS Flow
  • MS Flow will save a file in OneDrive
  • Somehow this file will disable the screen saver

And the Flow was very simple

04 microsoft flow flic button trigger and save a OneDrive file

Some notes here. The convertTimeZone() is a custom expression I created to have a unique File Name for each created file:

convertTimeZone(utcnow(),’UTC’,’W. Europe Standard Time’,’yyyy MM dd HH mm ss’)

The file content is Flic button information. I really don’t need this, but you never know :D. A sample list of created files:

onedrive files created by flic clic

I’m almost there. Every time I click my Flic button, after a couple of seconds I got a new file in each Windows 10 machine. So now I need some kind of File System Watcher to detect changes and stop the screen saver. I was thinking to create a simple .Net Core Console App to do this, and then I remember:

My ITPro friends usually do magic using PowerShell

A quick search returned this amazing post from Jessica Cook: Flow of the Week: Local code execution and she got almost everything I need. She is using gateways to save a local file and then she uses PowerShell to look for changes into a specific folder. I made some modifications to her PS script and I got my working one:

$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "<Folder to Watch>"
$watcher.Filter = "*.*"
$watcher.IncludeSubdirectories = $true
$watcher.EnableRaisingEvents = $true 
$action = {
Start-Process "taskkill /f /im bubbles.scr"
} 
Register-ObjectEvent $watcher "Created" -Action $action
while ($true) {sleep 5}

And done! I added my PowerShell script to the startup of the Windows 10 machine and it was amazing. A couple of seconds later after the clic, OneDrive sync the files, the PS script detect a new file and it stop the screen saver!

At the end my full process is

  • Click on a Flic button
  • Trigger a MS Flow
  • MS Flow will save a file in OneDrive
  • PowerShell script look for new files in the OD folder
  • When a new file is detected, PS script will kill the Screen Saver process

Of course, there is some room for improvement. Like in example, try to do something nicer than kill a windows process. I’ve tried sending keys or moving the mouse using PowerShell but it didn’t work. The kill process is the one!

Also, this is not real-time. Sometimes it sync and works in 1 second and sometimes it tooks 10 seconds, but it’s a great hack to connect dots and use some cool tech.

Happy coding!

Greetings @ Toronto

El Bruno

References

#Event – Materiales utilizados durante #GlobalAINight con los amigos de @metrotorontoUG

 

Buenas !

Después de una noche genial con los amigos de Metro Toronto UG, llega el momento de compartir los materiales que utilice durante la sesión. La idea inicial era hablar un poco de Azure Notebooks, y de alguna manera terminamos hablando también de Cognitive Services y Custom Vision, fue genial!

Para comenzar, los 15 min con el video de la Keynote:

Mis Slides

Y algunos de los links que utilicé durante la sesión

My posts on Custom Vision and ONNX

  1. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  2. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  3. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, drawing frames
  4. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, calculate FPS
  5. Can’t install Docker on Windows 10 Home, need Pro or Enterprise
  6. Running a Custom Vision project in a local Docker Container
  7. Analyzing images in a Console App using a Custom Vision project in a Docker Container
  8. Analyzing images using PostMan from a Custom Vision project hosted in a Docker Container
  9. Building the CustomVision.ai project in Docker in a RaspberryPi
  10. Container dies immediately upon successful start in a RaspberryPi. Of course, it’s all about TensorFlow dependencies
  11. About ports, IPs and more to access a container hosted in a Raspberry Pi
  12. Average response times using a CustomVision.ai docker container in a RaspberryPi and a PC

Windows 10 and YOLOV2 for Object Detection Series

Saludos @ Toronto

El Bruno

#Event – Resources used during the #GlobalAINight @metrotorontoUG

 

Hi  !

After an amazing event with my friends from Metro Toronto UG, it’s time to share some resources. It was initially supposed to be focused only on Azure Notebooks, but somehow we spend a lot of time talking about Cognitive Services and Custom Vision, that was great!

Let’s start with the 15 min Keynote video:

My Slides

And some interesting online resources

My posts on Custom Vision and ONNX

  1. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  2. Object recognition with Custom Vision and ONNX in Windows applications using WinML
  3. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, drawing frames
  4. Object recognition with Custom Vision and ONNX in Windows applications using Windows ML, calculate FPS
  5. Can’t install Docker on Windows 10 Home, need Pro or Enterprise
  6. Running a Custom Vision project in a local Docker Container
  7. Analyzing images in a Console App using a Custom Vision project in a Docker Container
  8. Analyzing images using PostMan from a Custom Vision project hosted in a Docker Container
  9. Building the CustomVision.ai project in Docker in a RaspberryPi
  10. Container dies immediately upon successful start in a RaspberryPi. Of course, it’s all about TensorFlow dependencies
  11. About ports, IPs and more to access a container hosted in a Raspberry Pi
  12. Average response times using a CustomVision.ai docker container in a RaspberryPi and a PC

Windows 10 and YOLOV2 for Object Detection Series

Greetings @ Toronto

El Bruno

#Office – Ideas Tool para Microsoft Excel basado en AI (#Excel, #AI)

Buenas !

Hoy voy a cambiar a Excel para compartir algunas de las otras increíbles características de Inteligencia Artificial en Microsoft Office: Excel Ideas.

Excel Ideas es esta nueva herramienta que tenemos en Excel que nos permitirá obtener algunas ideas sobre información seleccionada en una hoja de trabajo. Para utilizar esta herramienta, necesitamos seleccionar un conjunto de datos y hacer clic en el botón [ideas] en la cinta de opciones

Microsoft Excel Ideas.jpg

Para escribir este post, he descargado un conjunto de muestra de datos espaciales de SpatialKey (ver referencias). El archivo de seguro de muestra contiene 36.634 registros en Florida para 2012 de una compañía de muestra que implementó un plan de crecimiento agresivo en 2012. Hay columnas de valor asegurado total (TIV) que contienen TIV de 2011 y 2012, por lo que este conjunto de datos es ideal para probar la característica de comparación. Este archivo tiene información de dirección que puede elegir para geocodificar, o puede usar la latitud/longitud existente en el archivo.

Una vez que seleccioné el conjunto completo de datos, es bueno ver las siguientes ideas

Si usted es un usuario de PowerBI, este tipo de información y análisis será familiar. Para mí es bueno ver que tenemos estas capacidades en un solo clic en una herramienta popular como Excel.

Hay más de + 40 Insights y es agradable para navegar por ellos y encontrar que en el Condado de Miami Dade, el deducible del sitio es mayor para las construcciones de marco de acero y hormigón armado.

Microsoft Excel Ideas Sample 03

Con 2 clic podemos analizar algunos datos abiertos relacionados con el tráfico en Ontario y ver cómo el tráfico aumenta lentamente en Hwy 69 desde 1988 hasta 2010.

Microsoft Excel Ideas Sample 04

Una herramienta increíble dentro de Excel para obtener rápidamente información sobre nuestros datos.

Happy coding!

Saludos @ Toronto

El Bruno

References

My posts

#Office – Ideas Tool for Microsoft Excel powered by AI (#Excel, #AI)

Hi !

Today I’ll switch to Excel to share some of the other amazing AI features embedded in Microsoft Office: Excel Ideas.

Excel Ideas is this new tool we have in Excel which will allow us to get some insights about the selected information in a worksheet. In order to use this tool, we need to select a set of data and click the [Ideas] button in the Ribbon.

Microsoft Excel Ideas.jpg

In order to write this post, I downloaded a sample set of spatial data from SpatialKey (see references). The sample insurance file contains 36,634 records in Florida for 2012 from a sample company that implemented an agressive growth plan in 2012. There are total insured value (TIV) columns containing TIV from 2011 and 2012, so this dataset is great for testing out the comparison feature. This file has address information that you can choose to geocode, or you can use the existing latitude/longitude in the file.

Once I selected the full set of Data, is nice to see the following insights

If you are a PowerBI user, this type of information and analysis will be familiar. To me is nice to see that we have this capabilities in a single click in a popular tool like Excel.

There are over +40 insights and it’s nice to browse them and find that in Miami Dade County, the site deductible is higher for Steel Frame and Reinforced Concrete constructions.

Microsoft Excel Ideas Sample 03

With 2 click we can analyze some Open Data related to traffic on Ontario and see how the traffic slowly increases in Hwy 69.

Microsoft Excel Ideas Sample 04

An amazing tool inside Excel to quickly get insights on our data.

Happy coding!

Greetings @ Toronto

El Bruno

References

My posts