Personal Informatics is an approach to managing your own personal information so that it will be of use to you, part of an overall field called Informatics. Most often people think of this as monitoring your exercise data, weight, and other health information, most often referred to as the quantified self, but our approach is much broader than that. We help you make use of your general information to make decisions. This means we have a way to structure your information and a way to help you analyze and make sense of it so you can act on it.
Increasingly companies are collecting information about you in the form of a knowledge graph. They are using Enterprise Informatics for their businesses. In Facebook and LinkedIn, the nodes in the graph can be people such as yourself, and the edges are how you are related to those other people. By organizing information about you this way, they can provide useful recommendations to you, or improve their advertising. Knowledge graphs are also used to run search engines such as Google and Bing, in this case the nodes are web pages, and edges are links from one page to another.
In the course of your day, you interact with people, use web pages and collect information about things and events in your life. You have a need to own and manage your information beyond Facebook and Google. You need your own personal knowledge graph. And you need a way to make sense of it which is why we support visual interaction with minds map graphs using our Loci software.
Most mind maps are shown as 2D views of a graph with central nodes. It is generally better for comprehension to see and interact with a 3D version of the same graph to understand the relations between its nodes.
In the paper Using augmented reality for visualizing complex graphs in three dimensions by D. Belcher, M. Billinghurst, S Hayes, and R. Stiles, published in the International Society for Mixed and Augmented Reality (ISMAR) 2003, the study results (link) showed that there is a significant advantage with reduced error for 3D views over 2D views of the same graph when trying to determine if nodes are connected (linked) to each other, which is a key capability when trying to comprehend a graph (or mind map).
The study shows that this improvement of 3D views and interaction with complex graphs also transfers to augmented reality, and not just desktop 3D. In the graph from the paper, you can see significantly reduced node link comprehension error for 3D desktop and AR over the conventional 2D case, and this relationship holds as the number of nodes in the scene scales up.
A study of the method of loci, entitled “Mnemonic Training Reshapes Brain Networks to Support Superior Memory” published in the March 2017 issue of Neuron by scientists at Stanford University, the Max Planck Institute of Psychiatry, and the Donders Institute shows the effectiveness of mnemonic training, specifically the method of loci, which results in significant recall improvement across time periods of twenty minutes, one day, and four months. Even after four months, the group using the method of loci had more than a 20% level of improvement in recall over the active and passive control groups.
The study included using functional magnetic resonance imaging (fMRI) of the subjects, testing their recall of words, and then comparing their before and after MRIs with memory athletes. They found the method of loci subject’s connections across brain centers came to resemble the memory athlete’s. In their study they show with a figure of the brain’s MRI connections the noticeable difference in connections between memory athletes and controls and indicate this improvement from using the method of loci may result from improved connections (links) across brain areas.
This study does not include AR or VR elements, but it does emphasize the use of the method of loci, where mental images of locations are used to recall items. In our Loci software, we functionally support placing objects in places for later recall consistent with the method of loci.
Link In YOUR Mind – You Organize, Understand and Recall
Loci currently is provided as two variants; Loci Memory Palace in Virtual Reality on the PC and Loci AR Mind Map in Augmented Reality on the HoloLens. Both have a free seven day trial period and can import mind maps.
Loci uses three core concepts to help you organize, understand and recall; mind maps lets you break a problem down into component parts to organize and do analysis, mixed reality interaction with mind maps lets you visualize in 3D to understand how they are related, and the method of loci persistent placement of mind map nodes lets you improve your recall of the mind map, even when you are not using the Loci software. In this way you can organize your problem, understand it, and remember it later when you need to use that information for decisions.
PC/MR-HMD
Loci Memory Palace puts you in a 3D Memory Palace where you can make mind maps using the Windows Mixed Reality headset, and handheld controllers. You can use hand controllers to move, scale or rotate nodes or designate them for voice commands, or to select a place for moving nodes.
HoloLens
Loci AR Mind Map helps you put nodes and links in your own real settings using the HoloLens, such as your home, where you can place your notes and ideas with real items to help you remember and think about them. You can use one or two hand gestures with your own hands to move, scale or rotate nodes.
Both versions of Loci support gaze and voice interaction combined, so that the use of hands or controllers is not required at all times.
Both versions of Loci share the same mind map graph format, *.loci, and both have initial import capability for MindManager, Freemind, and GraphML files. This allows you to bring in your previous mind maps and graphs, as well as mind maps or graph data from other people.
Loci currently imports three file types; MindManager, Freemind / Freeplane, and GraphML. Here we provide an overview of the formats, our import of the format, and usage in Loci.
Loci has its own file format based on JSON, which has the file extension *.loci. It supports concepts of graph, node and edges for general graphs. Our initial import capability imports nodes, implicit MindMap relations (links), explicit relations, colors, urls, notes, and titles, and 2D positions where possible.
The Loci voice command to import is to say “Load Graph” which will use the system file browser to find files with extensions Loci can import. Once imported, you may wish to use “Graph Reset” followed by “Graph Layout” to use our force directed layout in 3D.
The MindManager format has the extension *.mmap and is actually zipped (compressed) xml based text. It is generally a strict mind map format and not a full graph format, in that it expects one or more central nodes. MindManager is a capable 2D mind mapping tool for Windows, Mac, iOS, and Android that has scheduling capabilities, and different 2D views.
The Freemind and associated Freeplane format has the extension *.mm and is an unzipped xml format file. Like MindManager, it expects central nodes, with implicit links (edges) to nodes nested within it. Freemind and Freeplane are java based open source 2D mind mapping tools available for Windows, Mac and Linux, and are widely used as an intermediate mind map format for exchange between systems. We tested our import using Freeplane, which is a fork and likely superset of the Freemind app and format.
The GraphML format has the extension *.graphml and is xml based. It primarily uses the abstraction of nodes and edges (links). It allows for a wide variety of graphs, a superset of mind maps. This includes freestanding nodes, graphs with cycles, and bi-directional graphs. And of course it allows for typed edges (links, relations) between nodes, while in Mind Map applications such relations are an exception. The variety of GraphML we have tested on has been produced by yEd, a free 2D java based graph editor available for Windows, Mac and Linux with nice automated layout capabilities.
At the University of Maryland College Park, Krokos et al recently published a paper “Virtual memory places: immersion aids recall“, which shows more than 8% recall improvement of a VR memory palace over a desktop memory palace.
The authors provide a nice overview of related work and then describe their study which compared desktop 3D scenes and virtual reality scenes (Head Mount Display – HMD) of a palace and a medieval town as locations to use the method of loci for recall.
The memory task the study participants were to do was recalling two sets of faces and names they saw in these two scenes after two minutes. Strangely, they allowed study participants to rotate their view, but not to translate (move around). Even with this limitation in their study, they found there was a statistically significant effect favoring the HMD (VR) display over the desktop display. Study participants were better able to recall items by 8.8% overall average difference using the HMD over the desktop display.
We have released Loci Memory Palace app for Windows Mixed Reality, which is a virtual reality memory palace populated with nodes and links that are part of your mind map graphs. With Loci Memory Palace, you can move around your memory palace and make mind maps by placing nodes in locations that help you recall. Our Loci app is available now on the Microsoft Store here.
PC/MR-HMD
With Loci Memory Palace, you can load images from files and place them in the memory palace, moving yourself and the nodes around in full six degree of freedom. When compared to the conditions of this study, there is more involvement of your body in placing, naming, editing, and associating nodes, and because of this embodiment, involving more of your senses and thoughts, an even greater improvement in recall from using Loci than this study is likely. Your recall of your notes and ideas is available to you even when you are not using our software.
Abstract:Virtual reality displays, such as head-mounted displays (HMD), afford us a superior spatial awareness by leveraging our vestibular and proprioceptive senses, as compared to traditional desktop displays. Since classical times, people have used memory palaces as a spatial mnemonic to help remember information by organizing it spatially and associating it with salient features in that environment. In this paper, we explore whether using virtual memory palaces in a head-mounted display with head-tracking (HMD condition) would allow a user to better recall information than when using a traditional desktop display with a mouse-based interaction (desktop condition). We found that virtual memory palaces in HMD condition provide a superior memory recall ability compared to the desktop condition. We believe this is a first step in using virtual environments for creating more memorable experiences that enhance productivity through better recall of large amounts of information organized using the idea of virtual memory palaces.
Image of training and test memory palaces for this post is from this paper, used under creative commons license.
