Heads-up displays for augmented reality network in a medical environment

{
  "publication_number": "US9645785B1",
  "country": "US",
  "kind": "B1",
  "title": "Heads-up displays for augmented reality network in a medical environment",
  "abstract": "A system for providing networked communications includes a plurality of head-mountable devices, each in communication with a control system via a communication network. Each of the plurality of head-mountable devices includes a display, and may also include an image-capture device and/or a microphone. The control system receives, via the communication network, surgical data of a patient obtained from at least one source of surgical data. Wearers of the head-mountable devices may select aspects of the surgical data and the control system causes those selected aspects to be displayed on the respective wearable device. The control system may also generate alerts and cause the alerts to be displayed on the wearable devices. An alert may include a notification that additional surgical data is available for access by the wearer of the head-mountable device.",
  "claims": [
    "1. A method comprising: receiving, by a control system, surgical data of a patient obtained from at least one source of surgical data, wherein the control system is in communication with a plurality of wearable devices via a communication network, wherein a first of the plurality of wearable devices comprises a head-mountable device that comprises a display; receiving, by the control system from the first of the plurality of wearable devices, via the communication network, a selection by a wearer of the first of the plurality of wearable devices of at least one aspect of the surgical data; transmitting to the first of the plurality of wearable devices via the communication network, at least one instruction configured to cause the display of the first of the plurality of wearable devices to display the at least one aspect of surgical data; generating, by the control system, at least one alert, wherein the at least one alert comprises a notification that a wearer of a second of the plurality of wearable devices has made additional surgical data available for access by the first of the plurality of wearable devices; and transmitting to the first of the plurality of wearable devices via the communication network, at least one instruction configured to cause the display of the first of the plurality of wearable devices to display the at least one alert.",
    "2. The method of claim 1, wherein the at least one alert further comprises the name of the wearer of the second one of the plurality of wearable devices.",
    "3. The method of claim 1, wherein the at least one alert further comprises a notification that additional surgical data is available for access by the first of the plurality of wearable devices.",
    "4. The method of claim 1, further comprising: receiving, by the control system from the first of the plurality of wearable devices via the communication network, an instruction to make additional surgical data available for access by the second of the plurality of wearable devices.",
    "5. The method of claim 1, wherein the at least one alert further comprises a notification that one of the at least one sources of surgical data has made additional surgical data available for access by the first of the plurality of wearable devices.",
    "6. The method of claim 1, wherein the surgical data comprises surgical image data captured by a camera provided on at least one of the plurality of wearable devices.",
    "7. The method of claim 1, wherein the surgical data comprises speech captured by a microphone provided on at least one of the plurality of wearable devices.",
    "8. The method of claim 1, wherein the surgical data comprises pre-surgical images of a patient.",
    "9. The method of claim 1, wherein the surgical data comprises surgical parameter data collected from at least one surgical instrument.",
    "10. The method of claim 1, further comprising: transmitting to at least one of the plurality of wearable devices via the communication network, at least one instruction configured to cause a display of the at least one of the plurality of wearable devices to display an action item; receiving, by the control system from at least one of the plurality of wearable devices, via the communication network, a response to the action item; and transmitting to at least one of the plurality of wearable devices via the communication network, at least one instruction configured to cause the display of the at least one of the plurality of wearable devices to display the response to the at least one action item.",
    "11. The method of claim 1, further comprising: transforming, by the control system, at least one aspect of the surgical data of a patient obtained from at least one source of surgical data to generate transformed surgical data; and transmitting to the first of the plurality of wearable devices via the communication network, at least one instruction configured to cause the display of the first of the plurality of wearable devices to display the transformed surgical data.",
    "12. The method of claim 11, wherein the at least one aspect of the surgical data comprises surgical video data of a patient, and further wherein transforming, by the control system, at least one aspect of the surgical data of a patient obtained from at least one source of surgical data to generate transformed surgical data comprises: generating a layered surgical image comprising an additional aspect of the surgical data superimposed on the surgical video data.",
    "13. The method of claim 12, wherein the additional aspect of the surgical data comprises one or more of fluorescent image data, spectral image data, and hyperspectral image data.",
    "14. A method comprising: receiving, by a first of a plurality of wearable devices, an instruction by a wearer of the first of the plurality of wearable devices to select at least one aspect of surgical data of a patient obtained from at least one source of surgical data, wherein the first of the plurality of wearable devices comprises a head-mountable device that comprises a display; displaying the selected at least one aspect of surgical data on the display of the first of the plurality of wearable devices; receiving, by the first of the plurality of wearable devices, an alert that additional surgical data is available for access by the first of the plurality of wearable devices, wherein the alert comprises a notification that a wearer of a second of the plurality of wearable devices has made the additional surgical data available for access by the first of the plurality of wearable devices; and displaying the alert on the display of the first of the plurality of wearable devices.",
    "15. The method of claim 14, further comprising: receiving, by the first of the plurality of wearable devices, an instruction by the wearer of the first of the plurality of wearable devices to select the additional surgical data available for access; and displaying the additional surgical data on the display of the first of the plurality of wearable devices.",
    "16. The method of claim 14, wherein the surgical data comprises surgical image data captured by a camera provided on at least one of the plurality of wearable devices.",
    "17. A system comprising: a plurality of wearable devices, wherein a first of the plurality of head-mountable devices comprises a head-mountable device that comprises a display; and a control system in communication with each of the plurality of wearable devices via a communication network, the control system comprising: a processor; a non-transitory computer-readable medium; and program instructions stored in the non-transitory computer readable medium, wherein the program instructions are executable by the processor to cause the control system to perform functions comprising: receiving, via the communication network, surgical data of a patient obtained from at least one source of surgical data; receiving from a first of the plurality of wearable devices, via the communication network, a selection of at least one aspect of the surgical data made by a wearer of the first of the plurality of wearable devices; transmitting to the first of the plurality of wearable devices via the communication network, at least one instruction configured to cause the display of the first of the plurality of wearable devices to display the at least one aspect of surgical data; generating at least one alert, wherein the at least one alert comprises a notification that a wearer of a second of the plurality of wearable devices has made additional surgical data available for access by the first of the plurality of wearable devices; and transmitting to the first of the plurality of wearable devices via the communication network, at least one instruction configured to cause the display of the first of the plurality of wearable devices to display the at least one alert.",
    "18. The system of claim 17, wherein at least one of the plurality of wearable devices comprises an image capture device and wherein the surgical data comprises surgical image data obtained by the image capture device.",
    "19. The system of claim 17, wherein at least one of the plurality of wearable devices comprises a microphone and wherein the surgical data comprises speech captured by the microphone.",
    "20. The system of claim 17, wherein the at least one alert further comprises the name of the wearer of the second one of the plurality of wearable devices that is within the field of view of the wearer of the first of the plurality of wearable devices."
  ],
  "description_excerpt": "Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.\n\nA surgical team may comprise a number of medical professionals, each playing an assigned role and each managing various aspects of surgical data captured during, or even before, a surgical procedure. This surgical data may include surgical images and video, surgical instrument parameters, patient health parameters (e.g., vital signs), lab results, etc. A variety of technologies exists to extract this data from the surgical setting and save it in a medical records database, or elsewhere for later analysis. However, during a surgical procedure, these various aspects of surgical data often remain under the control of the surgical team member that captured or generated the data and may not be readily and immediately available to others on the team. For example, in a surgical robotics scenario, such as a laparoscopic surgery, only the surgeon may have an optical visual interface to the surgical field. Further, some members of the surgical team, such as a patient's general practitioner or a radiologist, may not be located within the operating room during a surgical procedure, but may benefit from being aware of the status of the surgery and/or being in communication with those within the operating room.\n\nOpen communication and data sharing among surgical team members can play an important role in reducing medical errors in the operating room. In some cases, however, team communication can be impeded by user interface technologies.",
  "cpc": [
    "A61B 5/7445",
    "A61B 2034/252",
    "A61B 2034/254",
    "A61B 2034/256",
    "A61B 34/25",
    "G02B 2027/014",
    "G02B 27/017",
    "G06F 19/321",
    "G06F 3/1423",
    "G06F 3/1454",
    "G08B 21/18",
    "G16H 20/40",
    "G16H 30/20",
    "G16H 40/63"
  ],
  "ipc": [
    "A61B 34/00",
    "G06F 19/00",
    "G06F 3/14",
    "G08B 21/18"
  ],
  "assignees": [
    "Verily Life Sciences LLC"
  ],
  "inventors": [
    "Blake Hannaford",
    "Joelle Karine Barral",
    "Eden Rephaeli",
    "Christine Denise Ching",
    "Vikram Singh Bajaj"
  ],
  "filing_date": "2016-12-05",
  "publication_date": "2017-05-09",
  "grant_date": "2017-05-09",
  "priority_date": "2014-12-31",
  "application_number": "US-201615369484-A",
  "family_id": "57705772",
  "cited_by_count": 47,
  "citations": [
    "US20140247926A1",
    "US20140081659A1"
  ]
}