Processing of gesture-based user interactions using volumetric zones

{
  "publication_number": "US9811166B2",
  "country": "US",
  "kind": "B2",
  "title": "Processing of gesture-based user interactions using volumetric zones",
  "abstract": "Systems and methods for processing gesture-based user interactions within an interactive display area are provided. The display of one or more virtual objects and user interactions with the one or more virtual objects may be further provided. Multiple interactive areas may be created by partitioning an area proximate a display into multiple volumetric spaces or zones. The zones may be associated with respective user interaction capabilities. A representation of a user on the display may change as the ability of the user to interact with one or more virtual object changes.",
  "claims": [
    "1. A method comprising: receiving, by a computing system, information corresponding to a physical space from one or more cameras, wherein the physical space comprises a plurality of zones; and generating, by the computing system, an image comprising one or more virtual objects and a first status indicator indicating a first portion of a user in the physical space that is capable of providing input to the computing system, wherein the first status indicator is not reactive to movement of the user within a first zone of the plurality of zones unless the user approaches a boundary to either another of the plurality of zones or to an outside physical space outside of any of the plurality of zones, and wherein, in response to detection of the at least the first portion of the user approaching an inter-zone boundary between the first zone and a second zone from a first direction, adjusting a display characteristic of the first status indicator to indicate that a change to portions of the user that are capable of interacting with the one or more virtual objects will occur if the user continues to move in the first direction.",
    "2. The method of claim 1, wherein the first zone is positioned less than a first distance from a display device upon which the image is displayed, the second zone is positioned at a distance greater than the first distance from the display device and less than a second distance from the display device, and wherein the plurality of zones further comprises a third zone that is positioned greater than the second distance from the display device and less than a third distance from the display device.",
    "3. The method of claim 1, further comprising selecting the first status indicator from a plurality of status indicators in response to detecting the at least the first portion of the user within the first zone, wherein the first status indicator indicates that the user is not capable of interacting with the one or more virtual objects with hands or body of the user.",
    "4. The method of claim 3, wherein the first status indicator comprises at least one of: a disappearing puff of smoke; or a fading image of at least a portion of the user.",
    "5. The method of claim 1, further comprising selecting a second status indicator in response to detecting at least hands and body of the user within the second zone, wherein the second status indicator indicates that the user is capable of interacting with the one or more virtual objects in the second zone with the user's hands and body and the second status indicator does not indicate specific input from the user that is used to interact with the one or more virtual objects.",
    "6. The method of claim 5, further comprising selecting a third status indicator in response to detecting at least the portion of the user within a third zone, wherein the third status indicator indicates that the user is capable of interacting with the one or more virtual objects in the third zone with the body of the user.",
    "7. The method of claim 1, wherein said detection of the at least the first portion of the user approaching the inter-zone boundary is based on at least one of a distance of the at least the first portion of the user from a display device or a distance of the least the first portion of the user from one or more of the one or more cameras.",
    "8. The method of claim 1, wherein the first status indicator indicates a third portion of the user that is not capable of providing input to the computing system in order to interact with the one or more virtual objects.",
    "9. The method of claim 1, wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to a display device upon which the image is displayed, the method further comprising adjusting the boundary between the first zone and a second zone using the display distance.",
    "10. The method of claim 1, further comprising: determining an activation level of the user based on a first distance from a first position of the first portion of the user to a position of another portion of the user in the physical space; and in response to detection of the at least the first portion of the user approaching the inter-zone boundary between the first zone and the second zone from the first direction, updating the activation level of the user based on a second distance from a second position of a second portion of the user in the physical space to the position of another portion of the user in the physical space.",
    "11. A non-transitory computer readable storage having executable instructions stored thereon, the executable instructions configured to cause a computing system having one or more hardware processors to perform operations comprising: receiving information indicative of a physical space having a plurality of zones; generating an image, the image comprising: a virtual object; and a first status indicator indicating a first portion of a user in the physical space; wherein the first status indicator is not reactive to movement of the user within a first zone of the plurality of zones unless the user approaches a boundary to either another of the plurality of zones or to an outside physical space outside of any of the plurality of zones; and adjusting, in response to detection of the at least the first portion of the user approaching an inter-zone boundary between the first zone and a second zone from a first direction, a display characteristic of the first status indicator to indicate that a change to portions of the user that are capable of interacting with the one or more virtual objects will occur if the user continues to move in the first direction.",
    "12. The non-transitory computer readable storage of claim 11, wherein the user is capable of different interactions with the virtual object in each of the plurality of zones.",
    "13. The non-transitory computer readable storage of claim 11, wherein the first zone is positioned less than a first distance from a display device upon which the image is displayed, the second zone is positioned at a distance greater than the first distance from the display device and less than a second distance from the display device, and wherein the plurality of zones further comprises a third zone that is positioned greater than the second distance from the display device and less than a third distance from the display device.",
    "14. The non-transitory computer readable storage of claim 11, wherein the computer system further performs operations comprising selecting the first status indicator from a plurality of comprises in response to detecting the at least the first portion of the user within the first zone, wherein the first status indicator indicates that the user is not capable of interacting with the one or more virtual objects with hands or body of the user.",
    "15. The non-transitory computer readable storage of claim 11, wherein the computer system further performs operations comprising: determining an activation level of the user based on a first distance from a first position of the first portion of the user to a position of another portion of the user in the physical space; and updating, in response to detection of the at least the first portion of the user approaching the inter-zone boundary between the first zone and a second zone from the first direction, the activation level of the user based on a second distance from a second position of a second portion of the user in the physical space to the position of another portion of the user in the physical space.",
    "16. A computing system, comprising: a camera; and one or more hardware processors configured to execute instructions in order to cause the computing system to: receive information corresponding to a physical space from the camera, wherein the physical space comprises a plurality of zones; and generate an image comprising one or more virtual objects and a first status indicator indicating a first portion of a user that is capable of providing input to the computing system, wherein the first status indicator is not reactive to movement of the user within a first zone of the plurality of zones unless the user approaches a boundary to either another of the plurality of zones or to an outside physical space outside of any of the plurality of zones, and wherein, in response to detection of the at least the first portion of the user approaching an inter-zone boundary between the first zone and a second zone from a first direction, adjusting a display characteristic of the first status indicator to indicate that a change in which portions of the user are capable of interacting with the one or more virtual objects will occur if the user continues to move in the first direction.",
    "17. The system of claim 16, wherein the first zone is positioned less than a first distance from a display device upon which the image is displayed, the second zone is positioned at a distance greater than the first distance from the display device and less than a second distance from the display device, and a third zone is positioned greater than the second distance from the display device and less than a third distance from the display device.",
    "18. The system of claim 16, wherein the instructions further cause the computer system to select the first status indicator in response to detection of the at least the first portion of the user within the first zone, wherein the first status indicator indicates that the user is not capable of interacting with the one or more virtual objects with hands or body of the user.",
    "19. The system of claim 16, wherein the information corresponding to the physical space includes a display distance indicating a distance of the user to a display device upon which the image is displayed, wherein the instructions further cause the computer system to adjust the inter-zone boundary between the first zone and a second zone using the display distance.",
    "20. The system of claim 16, further comprising instructions to cause the computer system to: determine an activation level of the user based on a first distance from a first position of the first portion of the user to a position of another portion of the user in the physical space; and in response to detection of the at least the first portion of the user approaching the inter-zone boundary between the first zone and a second zone from the first direction, update the activation level of the user based on a second distance from a second position of a second portion of the user in the physical space to the position of another portion of the user in the physical space."
  ],
  "description_excerpt": "Field of the Invention\n\nThe present invention generally relates to interactive display systems. More specifically, the present invention concerns user interfaces for processing gesture-based user interactions with an interactive display in a three-dimensional environment.\n\nDescription of the Related Art\n\nMany existing interactive technologies rely on the likes of two-dimensional cameras. The two-dimensional data generated by these cameras is difficult to reliably parse for the purpose of separating a foreground from a background. As a result, it is difficult to isolate the hands or other appendages of a user and track corresponding movements of the same.\n\nPoor isolation and tracking data leads to difficulties with respect to providing data to an interactive display system. If an interactive display system is unable to gather data for controlling or manipulating information rendered through the same, a user may be forced to rely upon button presses or manipulation of physical objects, such as keyboards, joysticks, and actuators. Once a user is forced to rely upon physical instruments for providing input, many of the benefits offered by an interactive display system are lost.\n\nThere is a need for processing of gesture-based user interactions with respect to interactive display systems.\n\nIn a first embodiment, a method for processing gesture-based user interactions with an interactive display is disclosed. Through the method, three-dimensional information about an object in a defined three-dimensional space in gathered.",
  "cpc": [
    "G06F 3/017",
    "A63F 2300/1087",
    "G01B 11/14",
    "G06F 3/011",
    "G06F 3/0304",
    "G06F 3/04812",
    "G06F 3/04883",
    "G06K 9/00335",
    "G06V 40/20"
  ],
  "ipc": [
    "G01B 11/14",
    "G06F 3/01",
    "G06F 3/03",
    "G06K 9/00",
    "G06F 3/0481",
    "G06F 3/0488"
  ],
  "assignees": [
    "Intellectual Ventures Holding 81 LLC"
  ],
  "inventors": [
    "Matthew Bell",
    "Tipatat Chennavasin",
    "Charles H. Clanton",
    "Michael Hulme",
    "Eyal Ophir",
    "Matthew Vieta"
  ],
  "filing_date": "2015-06-15",
  "publication_date": "2017-11-07",
  "grant_date": "2017-11-07",
  "priority_date": "2007-09-14",
  "application_number": "US-201514739922-A",
  "family_id": "40452370",
  "cited_by_count": 8,
  "citations": [
    "US2917980A",
    "US3068754A",
    "US3763468A",
    "US4053208A",
    "US4275395A",
    "JPS5794672A",
    "EP0055366A2",
    "US4573191A",
    "US4725863A",
    "US5001558A",
    "US4791572A",
    "US4843568A",
    "US4769697A",
    "US5900982A",
    "US4887898A",
    "US4948371A",
    "US5276609A",
    "US5325472A",
    "US5138304A",
    "US6732929B2",
    "US5239373A",
    "US5151718A",
    "US5534917A",
    "US5325473A",
    "US5861881A",
    "US6097369A",
    "US5528297A",
    "US5418583A",
    "US5497269A",
    "US6008800A",
    "US5982352A",
    "US7084859B1",
    "US5442252A",
    "US5319496A",
    "US5771307A",
    "US5526182A",
    "US5436639A",
    "EP0626636A2",
    "US5454043A",
    "US5473396A",
    "US5423554A",
    "US5703637A",
    "US5510828A",
    "US5426474A",
    "US5528263A",
    "US5846086A",
    "US5808784A",
    "US6198487B1",
    "US5548694A",
    "US5594469A",
    "US6125198A",
    "US6191773B1",
    "US5633691A",
    "US5882204A",
    "US5591972A",
    "US6477239B1",
    "US5923380A",
    "US6308565B1",
    "US5662401A",
    "US6278418B1",
    "US5953152A",
    "US6084979A",
    "US6337699B1",
    "US6270403B1",
    "US6400374B2",
    "US5923475A",
    "US5969754A",
    "US5978136A",
    "US6128003A",
    "JPH10207619A",
    "US6353428B1",
    "US6118888A",
    "WO1998038533A1",
    "US6088612A",
    "US6097390A",
    "US6058397A",
    "US6335977B1",
    "US6323895B1",
    "US6304267B1",
    "US6075895A",
    "US6377298B1",
    "US20010012001A1",
    "US6414672B2",
    "JPH1157216A",
    "US7042440B2",
    "US6720949B1",
    "US6522312B2",
    "US6215890B1",
    "US6072494A",
    "EP0913790A1",
    "US6339748B1",
    "US6166744A",
    "US6611241B1",
    "US6217449B1",
    "US6176782B1",
    "US6195104B1",
    "US6388657B1",
    "US6198844B1",
    "US6349301B1",
    "US6464375B2",
    "US20020032697A1",
    "US20010033675A1",
    "US5966696A",
    "US6445815B1",
    "US6254246B1",
    "US6792398B1",
    "JP2000105583A",
    "US20060013440A1",
    "US6263339B1",
    "US20020006583A1",
    "WO2000016562A1",
    "US6677969B1",
    "US6359612B1",
    "US6501515B1",
    "US6106119A",
    "US6351222B1",
    "US6661918B1",
    "US6222465B1",
    "US7158676B1",
    "US6552760B1",
    "US6513953B1",
    "US6333735B1",
    "US6712476B1",
    "US6292171B1",
    "US6476834B1",
    "US20030032484A1",
    "US6545706B1",
    "US6965693B1",
    "US7331856B1",
    "US6791700B2",
    "US6346933B1",
    "EP1087327A2",
    "CN1289086A",
    "US6407870B1",
    "US20040046744A1",
    "US6826727B1",
    "US20030065563A1",
    "US6658150B2",
    "US6975360B2",
    "JP2003517642A",
    "US7069516B2",
    "US6480267B2",
    "USRE41685E1",
    "US7289130B1",
    "US6394896B2",
    "US6747666B2",
    "US20020140633A1",
    "US6491396B2",
    "US7367887B2",
    "US20040005924A1",
    "WO2001063916A1",
    "US7129927B2",
    "US6454419B2",
    "US20020046100A1",
    "JP2002014997A",
    "US20030137494A1",
    "US20020032906A1",
    "US7262874B2",
    "US6752720B1",
    "WO2002001537A2",
    "US6873710B1",
    "US7149262B1",
    "US20020041327A1",
    "US20080018595A1",
    "US7227526B2",
    "US20080030460A1",
    "US6598978B2",
    "US6754370B1",
    "US6707444B1",
    "US20060242145A1",
    "US6654734B1",
    "US20020078441A1",
    "JP2002092023A",
    "US7000200B1",
    "US20020081032A1",
    "US20020073417A1",
    "US7058204B2",
    "US20020064382A1",
    "US7095401B2",
    "US20020103617A1",
    "JP2002171507A",
    "US20020105623A1",
    "US20050091614A1",
    "US20030091724A1",
    "US8085293B2",
    "US20020130839A1",
    "US20020178440A1",
    "US20020140682A1",
    "US6912313B2",
    "US7834846B1",
    "US7259747B2",
    "WO2002100094A2",
    "EP1689172A1",
    "US20080062123A1",
    "US20020186200A1",
    "US7088508B2",
    "US6808293B2",
    "US7190832B2",
    "US7274800B2",
    "US20030113018A1",
    "US7068274B2",
    "US7015894B2",
    "US20030078840A1",
    "US20030093784A1",
    "US20030098819A1",
    "US7054068B2",
    "US20030103030A1",
    "US20030122839A1",
    "US20060256382A1",
    "US20030126013A1",
    "KR20030058894A",
    "US6902310B2",
    "US20030156756A1",
    "US7340077B2",
    "US7339521B2",
    "US20030161502A1",
    "JP2003271084A",
    "US6607275B1",
    "US20030178549A1",
    "US6707054B2",
    "US6831664B2",
    "US7050177B2",
    "US7006236B2",
    "US20030218760A1",
    "US20060132432A1",
    "US7170492B2",
    "US8035612B2",
    "US7710391B2",
    "US20080150890A1",
    "US7348963B2",
    "US20050122308A1",
    "US8035624B2",
    "US20040073541A1",
    "US20040015783A1",
    "US20040165006A1",
    "US6882480B2",
    "US6996460B1",
    "US20040091110A1",
    "US20050147135A1",
    "US20100026624A1",
    "US7576727B2",
    "US8199108B2",
    "WO2004055776A1",
    "US20060132725A1",
    "US6871982B2",
    "US6999600B2",
    "US20050195598A1",
    "US20040155962A1",
    "JP2004246578A",
    "US6877882B1",
    "US20040212725A1",
    "US7738725B2",
    "US7665041B2",
    "US8745541B2",
    "US20040193413A1",
    "US7593552B2",
    "US20050147282A1",
    "WO2004097741A1",
    "US20050185828A1",
    "US7193608B2",
    "US8072470B2",
    "WO2005003948A1",
    "US20060187545A1",
    "US20050028188A1",
    "US7001023B2",
    "US20050039206A1",
    "US7724280B2",
    "US20050086695A1",
    "US20050088407A1",
    "US7809167B2",
    "US20090235295A1",
    "WO2005041579A2",
    "WO2005041578A2",
    "US7536032B2",
    "US8487866B2",
    "US7619824B2",
    "US20050104506A1",
    "US7268950B2",
    "US20050132266A1",
    "WO2005057398A2",
    "WO2005057399A2",
    "JP2007514242A",
    "WO2005057921A2",
    "US20080090484A1",
    "US7468742B2",
    "WO2005091651A2",
    "US7379563B2",
    "US7382897B2",
    "US7394459B2",
    "US20050265587A1",
    "US7432917B2",
    "US20070259716A1",
    "US20060001760A1",
    "US20060010400A1",
    "US20070285419A1",
    "US20060031786A1",
    "US7559841B2",
    "US20060078015A1",
    "US7330584B2",
    "US20060111182A1",
    "US8085994B2",
    "US7671321B2",
    "US20060168515A1",
    "US7598942B2",
    "US20060187196A1",
    "US20060184993A1",
    "US7317836B2",
    "US20060227099A1",
    "US20060258397A1",
    "US7428542B1",
    "US8081822B1",
    "US7431253B2",
    "US20080123109A1",
    "US20090179733A1",
    "US20060294247A1",
    "US20060294258A1",
    "US20070002039A1",
    "US20070019066A1",
    "WO2007019443A1",
    "US8018579B1",
    "US8098277B1",
    "US7480870B2",
    "US20070199035A1",
    "US8611667B2",
    "US20070209009A1",
    "US20090256801A1",
    "US20080040692A1",
    "US20080013826A1",
    "US20080284925A1",
    "US20080179507A2",
    "US20080036732A1",
    "US7907117B2",
    "US20080062257A1",
    "US8064704B2",
    "US20080089587A1",
    "US7737636B2",
    "US20110041100A1",
    "US7769205B2",
    "US8121352B2",
    "US7728280B2",
    "US8384753B1",
    "US20110157316A1",
    "US7961906B2",
    "US7971156B2",
    "US20080170123A1",
    "US20080212306A1",
    "US7745771B2",
    "US20080252596A1",
    "WO2008124820A1",
    "US20080267447A1",
    "US7873906B2",
    "US20080320394A1",
    "US20090031240A1",
    "US20090027337A1",
    "US8230367B2",
    "WO2009035705A1",
    "US20090079813A1",
    "US20090106785A1",
    "US20090102788A1",
    "US8159682B2",
    "US20120200843A1",
    "US20090172606A1",
    "US20100039500A1",
    "US20090228841A1",
    "US8259163B2",
    "US20120317511A1",
    "US20100060722A1",
    "US20100121866A1",
    "US20120080411A1",
    "US20120293625A1"
  ]
}