Loci Voice and Gaze Commands

You use Loci by saying commands and looking at nodes or locations in the scene.  Some commands are done only using your voice, and some include where you are looking, your center of gaze.

It is important to understand that you use your gaze to identify a node that you act on the node with your voice.    This is gaze-based selection combined with voice actions (commands).

Generally commands to change the graph are just voice, and commands to change nodes or links include you focusing your gaze onto a node, and seeing the donut cursor on that object.

Our demonstration video, six minutes long, provides examples of using voice and gaze commands: Demo Video

The image of Loci commands in this post is also included as the default image for every node created in Loci.  It appears when you start Loci as part of the default scene.

Basic movement of a node is done by looking at it to place the cursor, and saying “Select” which Windows HoloShell uses as a voice command like a left mouse button select, or using an air tap with your index finger.  Once you select a node, it locks onto your center of gaze and you can walk and move your head to place it.  If you have it where you want it, say select or tap again to place it.  It will be locked there for subsequent graph layout, since your chosen locations are very important for using Loci.

When starting  out, a very useful voice-only command is “Load Example” which will load an example mind map graph describing the Loci application.

The primary voice commands to use are:

“add node” creates a node at your center of gaze and allows you to use voice to name it and gaze to place it. If you look in mid air, it creates a new unattached node. If you look at an existing node, it adds a new node and conveniently also adds a link to it, and then Loci allows you to name and place the newly added node.

You can add links between nodes by focusing your gaze on a node and saying “Link This” then moving your gaze to another node and saying “To That” and then a link between them will appear.  You can delete a link by saying “Unlink This” while gazing at the first node, and then “From That” while gazing at the second node.

“delete node” deletes the node you are looking at (cursor follows gaze onto node).  This includes deleting links to that node.

“undo” allows you to undo a prior editing action, such as deleting a node

“select” is a HoloLens standard voice command that is equivalent to a finger tap for selecting anything, in Loci it allows you to say “select” when gazing at a node and then place it with your gaze.  When you have the node in location, you say “select” again.

Say “name node” when you look at a node and Loci will ask you to provide a short name using your voice for dictation, and change the node name.

“capture image” will use the camera to capture what you are looking at, create a node with this image shown below, and allow you to place it.

A very useful set of commands are those for copy and paste.

You center your gaze on a node and say “Copy Node”.  This will take all the properties of a node and put them in the copy buffer in a text form.  Then you can look at a place in open space and say “Paste Node” and a copy will appear one meter in front of where you were looking.  You can also focus your gaze on an existing node, and say the same command “Paste Node” and it will update that node with the properties of the node you have previously copied (Color, scale, name, etc.).

Sometimes you only want to paste certain properties.  So you do the command “Copy Node” as usual, and then focus your gaze (cursor) on another node and say paste with a property name, such as “Paste Name” or “Paste Image” or “Paste Color”.

 

Keith Boesky Of ODG validates Loci’s Mind Mapping and Date Visualization in AR

“This is a huge one for me, Data Visualization. How much more do you see when you go to a 3D model on your screen than a flat screen?  How much more are you going to see when you’re able to put your glasses on with full awareness of what’s around you, with full awareness of everybody in the room with you, and be able to walk amongst your data set? I think that’s huge! I think it is going to change things as much is the spreadsheet did.

 

Loci Features Explained

This post describes important features of Loci, and compares Loci features with related work by others to give you a better idea of what Loci does.

First, Loci supports mind map interaction on graphs.  So it is easy to add a node to the graph in augmented reality, but it does not have to have a central node.

In Loci, nodes have locations.   If you place a node somewhere in your room, it will show up there next time you view that graph.  This is not the same for the other mind map applications, they will load the same mind map, but it is not anchored to a place in the real world.

 FeatureLociHolo MindHolo Mind VectorScapeHopHolo Graph
_URLLinkLinkLinkLinkLink
Price$19.99FreeFreeFreeFree
In App PurchasesNoNoYesNoNo
MS HoloLensXXXX
MS Mixed Reality
Google ARCoreX
Google TangoX
Apple iOS ARKit
MindMap InteractionXXXX
Graph Multiple StructuresXX
Graph Links and NodesXX
Control GazeX
Control TouchX
Control VoiceX
Control GestureXXX
Input TypedXXX?
Input Voice DictationXX
Edit Undo / RedoXX
Edit Copy / PasteXX
Drive File Load / SaveXAppAppAppX
File Data FormatJSON???Graphml, csv
File Image Formatjpg, pngjpg, pngjpg, png
Force Directed LayoutXX
Spatial Map Locations (Loci)X
Directional LinksXX
Bilboarded Node DataX
Hyperlink NodesX
Edit LinksXX
Take Picture as NodeX
Edit NodesXXXX

Loci Available on Windows Store for HoloLens

Buy Loci On Windows Store

On April 15, 2017 we published version 1.0 of the Loci universal windows application for HoloLens on the Windows Store.  Loci lets you create mind maps  in mixed reality where you place the mind map nodes in your home or office in locations that have meaning to you.   Using locations in this way, as a form of method of loci, where you associate your thoughts and ideas with locations, has been shown to increase your recall of them later.

Get App on Microsoft Store

Our Loci application allows you to create and edit mind maps in mixed reality. You save and load these using OneDrive.  The primary way to interact with Loci is using voice and gaze, and the current supported language is English. To understand how Loci works, it has a free one day trial.

Graphs

Loci is a general graph editor that supports mind map graphs. Graphs consist of nodes and edges. We call nodes with locations loci, and edges associating loci we call links.

node + location in real world = locus (loci is plural)
edge = relation = association = link

Many items in life can be understood using graphs. From molecules to web page links to friends of your friends, you can think of things as the nodes, and relations between them as the links.

Graphs are used in mathematics, physics, and informatics as underlying representations for solving problems.  Graph data structures are used for 3D games to manage the application of transformations, and Loci itself uses such a scene graph to draw all the things it shows.

Most mind map software requires a central concept node, and links to other nodes, with no node being unlinked, and all being directly, or indirectly connected to the central node.  Loci does not require that.   Nodes can be solitary, or multiply connected.   In this way we support the analysis process from end to end.   You can make unlinked notes for things as you observe them, or your ideas as you come upon them, then orient your thoughts by linking them together to make sense of them.