Sharing neighboring map data across devices

{
  "publication_number": "US2020042278A1",
  "country": "US",
  "kind": "A1",
  "title": "Sharing neighboring map data across devices",
  "abstract": "A computing device and method are provided for transmitting a relevant subset of map data, called a neighborhood, to enable mutual spatial understanding by multiple display devices around a target virtual location to display a shared hologram in the same exact location in the physical environment at the same moment in time. The computing device may comprise a processor, a memory operatively coupled to the processor, and an anchor transfer program stored in the memory and executed by the processor.",
  "claims": [
    "1. A first computing device operated by a first user, networked with a second computing device and a third computing device, the first computing device comprising: a processor; a memory operatively coupled to the processor and storing local map data of the first computing device; a first display operatively coupled to the memory and the processor; and an anchor transfer program stored in the memory and executed by the processor to be configured to receive first anchor data causing the first display to display one or more holograms to the first user at a first virtual place-located anchor at a first target virtual location from a vantage point of the first user, and further configured to execute an export anchor mode, wherein in the export anchor mode, the anchor transfer program is configured to: receive a first neighboring map data of a neighborhood around the first target virtual location; and send the first anchor data and the first neighboring map data to the second computing device operated by a second user, thereby causing a second display of the second computing device to display the one or more holograms at the first virtual place-located anchor at the first target virtual location from a vantage point of the second user based on the first neighboring map data and local map data of the second computing device. 2. The first computing device of claim 1, wherein the anchor transfer program is further configured to execute an import anchor mode; and in the import anchor mode, the anchor transfer program is configured to: receive, from the third computing device, second neighboring map data of a neighborhood around a second virtual place-located anchor at a second target virtual location, integrate the second neighboring map data into the local map data of the first computing device to create an integrated map data, and cause the first display to display the one or more holograms at the second virtual place-located anchor at the second target virtual location from the vantage point of the first user based on the integrated map data. 3. The first computing device of claim 1, wherein the anchor transfer program is configured to send an instruction to a server computing device to generate the first virtual place-located anchor at the first target virtual location, and receive the first virtual place-located anchor from the server computing device. 4. The first computing device of claim 1, wherein the anchor transfer program is configured to receive the first neighboring map data in a serialized format; and the anchor transfer program is configured to send the first neighboring map data to the second computing device in a deserialized format. 5. The first computing device of claim 1, wherein the first target virtual location is world-locked to a position that is fixed in a three-dimensional coordinate space overlaid upon a real world three-dimensional environment. 6. The first computing device of claim 1, wherein the first target virtual location is world-locked to a position relative to an object in a real world three-dimensional environment. 7. The first computing device of claim 1, wherein the neighboring map data comprises keyframes and at least a portion of a pose-graph describing rotational motion and translational motion of the first computing device and the second computing device through a real world three-dimensional environment. 8. The first computing device of claim 7, further comprising: visual sensors and/or inertial measurement sensors, wherein the visual sensors and/or inertial measurement sensors track the rotational motion and translational motion of the first computing device and second computing device for the keyframes and pose-graphs. 9. The first computing device of claim 7, wherein the keyframes comprise at least one of a fingerprint of a Wi-Fi beacon, gravity data, temperature data, global positioning data, and calibration data. 10. A method for a first computing device operated by a first user, networked with a second computing device and a third computing device, the method comprising: receiving first anchor data causing a first display of the first computing device to display one or more holograms to the first user at a first virtual place-located anchor at a first target virtual location from a vantage point of the first user, and executing an export anchor mode, wherein in the export anchor mode, a first neighboring map data of a neighborhood around the first target virtual location is received; and the first anchor data and the first neighboring map data are sent to the second computing device operated by a second user, thereby causing a second display of the second computing device to display the one or more holograms at the first virtual place-located anchor at the first target virtual location from a vantage point of the second user based on the first neighboring map data and local map data of the second computing device. 11. The method of claim 10, further comprising: executing an import anchor mode, wherein in the import anchor mode, second neighboring map data of a neighborhood around a second virtual place-located anchor at a second target virtual location is received from the third computing device; the second neighboring map data is integrated into the local map data of the first computing device to create an integrated map data; and the first display is caused to display the one or more holograms at the second virtual place-located anchor at the second target virtual location from the vantage point of the first user based on the integrated map data. 12. The method of claim 10, further comprising: sending an instruction to a server computing device to generate the first virtual place-located anchor at the first target virtual location; and receiving the first virtual place-located anchor from the server computing device. 13. The method of claim 10, wherein the first neighboring map data is received in a serialized format; and the first neighboring map data is sent to the second computing device in a deserialized format. 14. The method of claim 10, wherein the first target virtual location is world-locked to a position that is fixed in a three-dimensional coordinate space overlaid upon a real world three-dimensional environment. 15. The method of claim 10, wherein the first target virtual location is world-locked to a position relative to an object in a real world three-dimensional environment. 16. The method of claim 10, wherein the neighboring map data comprises keyframes and at least a portion of a pose-graph describing rotational motion and translational motion of the first computing device and the second computing device through a real world three-dimensional environment. 17. The method of claim 16, wherein visual sensors and/or inertial measurement sensors track the rotational motion and translational motion of the first computing device and the second computing device for the keyframes and pose-graphs. 18. The method of claim 16, wherein the keyframes comprise at least one of a fingerprint of a Wi-Fi beacon, gravity data, temperature data, global positioning data, and calibration data. 19. A first computing device networked with a second computing device and a third computing device, the first computing device comprising: a processor; a non-volatile storage device operatively coupled to the processor and storing local map data of the first computing device; a first display operatively coupled to the non-volatile storage device and the processor; and an anchor transfer program stored in the non-volatile storage device and executed by the processor to receive first anchor data causing the first display to align movements in aligned coordinate space based on a first virtual place-located anchor at a first target virtual location from a vantage point of the first computing device, and further configured to execute an export anchor mode, wherein in the export anchor mode, the anchor transfer program is configured to: receive a first neighboring map data of a neighborhood around the first target virtual location, and send the first anchor data and the first neighboring map data to the second computing device, thereby causing the second computing device to incorporate the neighboring map data into existing map data of the second computing device to create an integrated map data; and causing the second computing device to align movements in aligned coordinate space with the first computing device based on the integrated map data of the second computing device; and the first computing device and the second computing device are configured to be autonomous robots. 20. The first computing device of claim 19, wherein the anchor transfer program is further configured to: cause the second computing device to render and interact with one or more virtual objects and/or virtual spaces at the first virtual place-located anchor at the first target virtual location from a vantage point of the second computing device based on the integrated map data of the second computing device."
  ],
  "description_excerpt": "6-DoF tracking, also known as six degrees of freedom tracking, is a method by which a device (e.g. mixed-reality head-mounted device (HMD), robot, smartphone, etc.) uses sensors (e.g. cameras, inertial measurement units, etc.) to determine its position relative to its surrounding physical environment. For example, a mixed-reality HMD or smartphone can use this positional understanding to place holograms or digital content so as to appear to be world-locked to a position in the physical world, and a robot can use this positional understanding to navigate itself relative to its surroundings. Recently, scenarios have arisen in which it is useful to have two or more such devices operating with a common understanding of their positions relative to a physical environment, and thus relative to each other. As discussed in detail below, there are several general approaches to developing this common understanding of positions between such devices, each with significant challenges recognized by the inventors.\n\nTo address these issues, a computing device and method are provided for transmitting a relevant subset of map data, called a neighborhood, to enable mutual spatial understanding by multiple display devices around a target virtual location to display a shared hologram in the same exact location in the physical environment at the same moment in time. The computing device may comprise a processor, a memory operatively coupled to the processor, and an anchor transfer program stored in the memory and executed by the processor.",
  "cpc": [
    "G06F 3/1454",
    "B65G 1/0492",
    "G02B 2027/0138",
    "G02B 2027/0141",
    "G02B 27/017",
    "G03H 1/0005",
    "G03H 1/0808",
    "G03H 1/2249",
    "G03H 2001/0088",
    "G03H 2001/2252",
    "G03H 2226/02",
    "G03H 2226/04",
    "G03H 2226/05",
    "G05B 2219/40126",
    "G05D 2201/0216",
    "G06F 3/011",
    "G06F 3/012",
    "G06F 3/04815",
    "G06F 3/1423",
    "G06F 3/147",
    "G06T 19/006",
    "G09G 3/002",
    "G09G 3/003",
    "Y10S 901/01"
  ],
  "ipc": [
    "B65G 1/04",
    "G02B 27/01",
    "G03H 1/00",
    "G03H 1/22",
    "G06F 3/01",
    "G06F 3/0481",
    "G06F 3/14",
    "G06F 3/147",
    "G06T 19/00",
    "G09G 3/00"
  ],
  "assignees": [
    "Microsoft Technology Licensing LLC"
  ],
  "inventors": [
    "Ethan Eade",
    "Jeroen Vanturennout",
    "Jonathan LYONS",
    "David Fields",
    "Gavin Dean Lazarow",
    "Tushar Cyril BHATNAGAR"
  ],
  "filing_date": "2019-10-11",
  "publication_date": "2020-02-06",
  "priority_date": "2017-03-30",
  "application_number": "US-201916600340-A",
  "family_id": "63670655",
  "cited_by_count": 8
}