MLchartDataset catalogue

Patent · US9696427B2 · B2 · US

Wide angle depth detection

(11) Publication number
US9696427B2
(21) Application number
13/585,606
(22) Filing date
2012-08-14
(30) Priority date
2012-08-14
(43) Publication date
2017-07-04
(45) Date of grant
2017-07-04
(51) IPC
G01S 17/894; H04N 13/02; G01B 11/25
(52) CPC
  • G01B Measuring length, thickness or similar linear dimensions; measuring angles; measuring areas; measuring irregularities of surfaces or contours: 11/2513
  • G01S Radio direction-finding; radio navigation; determining distance or velocity by use of radio waves; locating or presence-detecting by use of the reflection or reradiation of radio waves; analogous arrangements using other waves: 17/89, 17/894
  • H04N Pictorial communication, e.g. television: 13/0253, 13/0271, 13/254, 13/271
(73) Assignee
Microsoft Technology Licensing LLC
(72) Inventors
Andrew Wilson; Hrvoje Benko; Jay Kapur; Stephen Edward Hodges
(54) Title
Wide angle depth detection
(57) Abstract

Embodiments for a depth sensing camera with a wide field of view are disclosed. In one example, a depth sensing camera comprises an illumination light projection subsystem, an image detection subsystem configured to acquire image data having a wide angle field of view, a logic subsystem configured to execute instructions, and a data-holding subsystem comprising stored instructions executable by the logic subsystem to control projection of illumination light and to determine depth values from image data acquired via the image sensor. The image detection subsystem comprises an image sensor and one or more lenses.

Full text
View on Google Patents

Claims (20)

  1. A depth sensing camera, comprising: an illumination light projection subsystem; an image detection subsystem configured to acquire image data including one or more images, each of the one or more images having a field of view of 100 degrees or greater, the image detection subsystem comprising an image sensor and one or more lenses, wherein the image sensor comprises a plurality of non-uniformly-sized pixels; a logic subsystem configured to execute instructions; and a data-holding subsystem comprising stored instructions executable by the logic subsystem to control projection of illumination light and to determine depth values from image data acquired via the image sensor.
  2. The depth sensing camera of claim 1, wherein the illumination light projection subsystem comprises one or more coherent infrared light sources, and wherein the instructions are executable to determine depth values based on a round-trip time-of-flight of light emitted by the one or more coherent infrared light sources and reflected back to the image sensor.
  3. The depth sensing camera of claim 2, wherein the instructions are further executable to correct for radial distortion.
  4. The depth sensing camera of claim 2, wherein the instructions are executable to correct for radial distortion by performing one or more of upsampling and downsampling regions of an image detected by the image detection subsystem and converting image data to Cartesian coordinates.
  5. The depth sensing camera of claim 1, wherein the illumination light projection subsystem comprises a projector configured to project a structured light pattern, and wherein the instructions are executable to determine depth values based on an image of the structured light pattern as reflected from one or more objects.
  6. The depth sensing camera of claim 5, wherein the projector is configured to project a structured light pattern with more widely spaced pattern elements at a periphery of the structured light pattern than at a center of the structured light pattern.
  7. The depth sensing camera of claim 5, wherein the instructions are executable to control a density of structured light features in the structured light pattern.
  8. The depth sensing camera of claim 5, wherein the instructions are further executable to correct for radial distortion and convert image data to Cartesian coordinates.
  9. The depth sensing camera of claim 1, wherein the image detecting subsystem comprises a mechanically scanning camera and the image sensor comprises a one-dimensional array of pixels.
  10. The depth sensing camera of claim 1, wherein the illumination light projection subsystem is configured to control a field of illumination by dynamically directing illumination light to a region of interest.
  11. The depth sensing camera of claim 1, wherein the plurality of non-uniformly-sized pixels provides lower resolution towards the periphery of the sensor surface as compared to the resolution at the center of the sensor surface.
  12. The depth sensing camera of claim 1, wherein the depth sensing camera is configured to be mounted in a light socket.
  13. The depth sensing camera of claim 12, further comprising a communication subsystem configured to communicate via a wireless network.
  14. A time-of-flight depth camera, comprising: an illumination light projection subsystem comprising one or more infrared light sources; an image detection subsystem configured to acquire image data having a field of view of 170 degrees or greater, the image detection subsystem comprising an image sensor comprising a plurality of non-uniformly-sized pixels, and one or more lenses; a logic subsystem configured to execute instructions; and a data-holding subsystem comprising stored instructions executable by the logic sub system to: control projection of the one or more infrared light sources; acquire image data via the image detection subsystem; correct for radial distortion in the image data to generate corrected image data; and determine depth values from the corrected image data.
  15. The time-of-flight depth camera of claim 14, wherein the instructions are further executable to correct for radial distortion by converting the image data to Cartesian coordinates.
  16. The time-of-flight depth camera of claim 14, wherein the instructions are executable to correct for radial distortion by performing one or more of upsampling and downsampling regions of an image detected by the image detection subsystem.
  17. The time-of-flight depth camera of claim 14, wherein the plurality of non-uniformly-sized pixels provides lower resolution towards the periphery of the sensor surface as compared to the resolution at the center of the sensor surface.
  18. In a structured light depth camera including a projector, an image sensor comprising a plurality of non-uniformly-sized pixels, and one or more lenses, a method of acquiring image data comprising: projecting a pre-distorted structured light pattern including a plurality of non-uniformly-spaced pattern elements having a lower resolution at a periphery of the structured light pattern than at a center of the structured light pattern; acquiring image data via the plurality of non-uniformly-sized pixels of the image sensor, the image data capturing the structured light pattern as reflected from a depth imaging environment; correcting for radial distortion in the image data to generate corrected image data; and determining depth values from the corrected image data.
  19. The method of claim 18, wherein the structured light pattern has a projected field of view of 100 degrees or greater, and wherein the plurality of non-uniformly-spaced pattern elements have a wider spacing at the periphery of the structured light pattern than at the center of the structured light pattern.
  20. The method of claim 18, wherein correcting for radial distortion includes converting image data to Cartesian coordinates.

Description

Depth cameras may be used in a variety of different environments, and may utilize various technologies, including but not limited to time-of-flight and structured light depth sensing technologies, to determine depth from image data. Structured light depth cameras project a pre-defined light pattern onto a target and then determine depth values for an image of the target based upon distortions in an image of the light pattern. Time-of-flight depth cameras project pulses of light onto the target, and then determine depth values based upon differences in how long it takes for light reflected from different parts of the target to reach the image sensor.

Embodiments are disclosed herein that relate to wide angle depth sensing cameras. For example, one disclosed embodiment provides a depth sensing camera comprising an illumination light projection subsystem and an image detection subsystem comprising an image sensor and one or more lenses and being configured to acquire image data having a field of view of 100 degrees or greater. The depth sensing camera further comprises a logic subsystem configured to execute instructions, and a data-holding subsystem comprising stored instructions executable by the logic subsystem to control the projection of illumination light and to determine depth values from image data acquired via the image sensor.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description.

Citations (193)

  • US4695953A
  • US4630910A
  • US4627620A
  • US4645458A
  • US4702475A
  • US4843568A
  • US4711543A
  • US4796997A
  • US5184295A
  • US4751642A
  • US4809065A
  • US4817950A
  • US5239464A
  • US5239463A
  • US4901362A
  • US4893183A
  • US5288078A
  • US4925189A
  • US5229756A
  • US5469740A
  • US5229754A
  • US5101444A
  • US5148154A
  • US5534917A
  • US5295491A
  • US6054991A
  • WO1993010708A1
  • US5875108A
  • US5417210A
  • EP0583061A2
  • US7222078B2
  • US5320538A
  • US5715834A
  • US5495576A
  • US5690582A
  • US5704837A
  • US5405152A
  • US5454043A
  • US5423554A
  • US5980256A
  • US5617312A
  • US5347306A
  • US5616078A
  • US5577981A
  • US5580249A
  • US5597309A
  • US5385519A
  • US5524637A
  • JPH0844490A
  • US5563988A
  • US6714665B1
  • US5516105A
  • US5638300A
  • US5703367A
  • US5594469A
  • US5682229A
  • US5913727A
  • US6229913B1
  • US5682196A
  • US6066075A
  • US6765726B2
  • US6073489A
  • US6430997B1
  • WO1997017598A1
  • US7359121B2
  • US7038855B2
  • US6308565B1
  • US6098458A
  • US6876496B2
  • US6283860B1
  • US5933125A
  • US5641288A
  • US6152856A
  • US6173066B1
  • US5989157A
  • US6005548A
  • US5995649A
  • US6128003A
  • US6009210A
  • US6100896A
  • US5877803A
  • US6215898B1
  • US6226396B1
  • US6188777B1
  • US6289112B1
  • US7042440B2
  • US6496598B1
  • US6215890B1
  • US6141463A
  • US6411744B1
  • USRE42256E1
  • US6101289A
  • US6130677A
  • US7746345B2
  • US6384819B1
  • US6256033B1
  • US6072494A
  • US7184048B2
  • US6181343B1
  • WO1999044698A2
  • US6159100A
  • US6077201A
  • US7684592B2
  • US7036094B1
  • US6681031B2
  • US7460690B2
  • US7121946B2
  • US7668340B2
  • US6950534B2
  • US6316934B1
  • US6256400B1
  • US20090046152A1
  • US6661918B1
  • US6147678A
  • US7202898B1
  • US6570555B1
  • US6363160B1
  • US7003134B1
  • US6299308B1
  • US6503195B1
  • US6476834B1
  • US6873723B1
  • EP1067362A1
  • US6738066B1
  • US7760182B2
  • US7113918B1
  • US6801637B2
  • US7050606B2
  • US7367887B2
  • US6633294B1
  • US7060957B2
  • US6640202B1
  • US6731799B1
  • US6788809B1
  • US7227526B2
  • US7898522B2
  • US7555142B2
  • US7421093B2
  • US7058204B2
  • US7039676B1
  • US6539931B2
  • US7259747B2
  • US7834846B1
  • US7452275B2
  • US7680298B2
  • US6937742B2
  • US7607509B2
  • US8035624B2
  • US8035614B2
  • US8035612B2
  • US7710391B2
  • US7170492B2
  • US7348963B2
  • US7489812B2
  • US7576727B2
  • US7436496B2
  • US20040196282A1
  • US7593552B2
  • US7590262B2
  • US8072470B2
  • US7620202B2
  • US7809167B2
  • US7536032B2
  • US7379563B2
  • US7308112B2
  • US20080024594A1
  • US20070285419A1
  • US7704135B2
  • US7702130B2
  • US7570805B2
  • US7430312B2
  • US7379566B2
  • US7574020B2
  • US7598942B2
  • US7317836B2
  • US7389591B2
  • US7560701B2
  • US20080026838A1
  • US7450736B2
  • US20100194862A1
  • US7701439B2
  • US7683954B2
  • US20100063681A1
  • US7412077B2
  • US7729530B2
  • US20080239109A1
  • US7852262B2
  • US20090059041A1
  • CN201254344Y
  • EP2378310A1
  • US20110310226A1
  • KR20120018915A
  • US20130266174A1
Record as JSON 
{
  "publication_number": "US9696427B2",
  "country": "US",
  "kind": "B2",
  "title": "Wide angle depth detection",
  "abstract": "Embodiments for a depth sensing camera with a wide field of view are disclosed. In one example, a depth sensing camera comprises an illumination light projection subsystem, an image detection subsystem configured to acquire image data having a wide angle field of view, a logic subsystem configured to execute instructions, and a data-holding subsystem comprising stored instructions executable by the logic subsystem to control projection of illumination light and to determine depth values from image data acquired via the image sensor. The image detection subsystem comprises an image sensor and one or more lenses.",
  "claims": [
    "1. A depth sensing camera, comprising: an illumination light projection subsystem; an image detection subsystem configured to acquire image data including one or more images, each of the one or more images having a field of view of 100 degrees or greater, the image detection subsystem comprising an image sensor and one or more lenses, wherein the image sensor comprises a plurality of non-uniformly-sized pixels; a logic subsystem configured to execute instructions; and a data-holding subsystem comprising stored instructions executable by the logic subsystem to control projection of illumination light and to determine depth values from image data acquired via the image sensor.",
    "2. The depth sensing camera of claim 1, wherein the illumination light projection subsystem comprises one or more coherent infrared light sources, and wherein the instructions are executable to determine depth values based on a round-trip time-of-flight of light emitted by the one or more coherent infrared light sources and reflected back to the image sensor.",
    "3. The depth sensing camera of claim 2, wherein the instructions are further executable to correct for radial distortion.",
    "4. The depth sensing camera of claim 2, wherein the instructions are executable to correct for radial distortion by performing one or more of upsampling and downsampling regions of an image detected by the image detection subsystem and converting image data to Cartesian coordinates.",
    "5. The depth sensing camera of claim 1, wherein the illumination light projection subsystem comprises a projector configured to project a structured light pattern, and wherein the instructions are executable to determine depth values based on an image of the structured light pattern as reflected from one or more objects.",
    "6. The depth sensing camera of claim 5, wherein the projector is configured to project a structured light pattern with more widely spaced pattern elements at a periphery of the structured light pattern than at a center of the structured light pattern.",
    "7. The depth sensing camera of claim 5, wherein the instructions are executable to control a density of structured light features in the structured light pattern.",
    "8. The depth sensing camera of claim 5, wherein the instructions are further executable to correct for radial distortion and convert image data to Cartesian coordinates.",
    "9. The depth sensing camera of claim 1, wherein the image detecting subsystem comprises a mechanically scanning camera and the image sensor comprises a one-dimensional array of pixels.",
    "10. The depth sensing camera of claim 1, wherein the illumination light projection subsystem is configured to control a field of illumination by dynamically directing illumination light to a region of interest.",
    "11. The depth sensing camera of claim 1, wherein the plurality of non-uniformly-sized pixels provides lower resolution towards the periphery of the sensor surface as compared to the resolution at the center of the sensor surface.",
    "12. The depth sensing camera of claim 1, wherein the depth sensing camera is configured to be mounted in a light socket.",
    "13. The depth sensing camera of claim 12, further comprising a communication subsystem configured to communicate via a wireless network.",
    "14. A time-of-flight depth camera, comprising: an illumination light projection subsystem comprising one or more infrared light sources; an image detection subsystem configured to acquire image data having a field of view of 170 degrees or greater, the image detection subsystem comprising an image sensor comprising a plurality of non-uniformly-sized pixels, and one or more lenses; a logic subsystem configured to execute instructions; and a data-holding subsystem comprising stored instructions executable by the logic sub system to: control projection of the one or more infrared light sources; acquire image data via the image detection subsystem; correct for radial distortion in the image data to generate corrected image data; and determine depth values from the corrected image data.",
    "15. The time-of-flight depth camera of claim 14, wherein the instructions are further executable to correct for radial distortion by converting the image data to Cartesian coordinates.",
    "16. The time-of-flight depth camera of claim 14, wherein the instructions are executable to correct for radial distortion by performing one or more of upsampling and downsampling regions of an image detected by the image detection subsystem.",
    "17. The time-of-flight depth camera of claim 14, wherein the plurality of non-uniformly-sized pixels provides lower resolution towards the periphery of the sensor surface as compared to the resolution at the center of the sensor surface.",
    "18. In a structured light depth camera including a projector, an image sensor comprising a plurality of non-uniformly-sized pixels, and one or more lenses, a method of acquiring image data comprising: projecting a pre-distorted structured light pattern including a plurality of non-uniformly-spaced pattern elements having a lower resolution at a periphery of the structured light pattern than at a center of the structured light pattern; acquiring image data via the plurality of non-uniformly-sized pixels of the image sensor, the image data capturing the structured light pattern as reflected from a depth imaging environment; correcting for radial distortion in the image data to generate corrected image data; and determining depth values from the corrected image data.",
    "19. The method of claim 18, wherein the structured light pattern has a projected field of view of 100 degrees or greater, and wherein the plurality of non-uniformly-spaced pattern elements have a wider spacing at the periphery of the structured light pattern than at the center of the structured light pattern.",
    "20. The method of claim 18, wherein correcting for radial distortion includes converting image data to Cartesian coordinates."
  ],
  "description_excerpt": "Depth cameras may be used in a variety of different environments, and may utilize various technologies, including but not limited to time-of-flight and structured light depth sensing technologies, to determine depth from image data. Structured light depth cameras project a pre-defined light pattern onto a target and then determine depth values for an image of the target based upon distortions in an image of the light pattern. Time-of-flight depth cameras project pulses of light onto the target, and then determine depth values based upon differences in how long it takes for light reflected from different parts of the target to reach the image sensor.\n\nEmbodiments are disclosed herein that relate to wide angle depth sensing cameras. For example, one disclosed embodiment provides a depth sensing camera comprising an illumination light projection subsystem and an image detection subsystem comprising an image sensor and one or more lenses and being configured to acquire image data having a field of view of 100 degrees or greater. The depth sensing camera further comprises a logic subsystem configured to execute instructions, and a data-holding subsystem comprising stored instructions executable by the logic subsystem to control the projection of illumination light and to determine depth values from image data acquired via the image sensor.\n\nThis Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description.",
  "cpc": [
    "G01B 11/2513",
    "G01S 17/89",
    "G01S 17/894",
    "H04N 13/0253",
    "H04N 13/0271",
    "H04N 13/254",
    "H04N 13/271"
  ],
  "ipc": [
    "G01S 17/894",
    "H04N 13/02",
    "G01B 11/25"
  ],
  "assignees": [
    "Microsoft Technology Licensing LLC"
  ],
  "inventors": [
    "Andrew Wilson",
    "Hrvoje Benko",
    "Jay Kapur",
    "Stephen Edward Hodges"
  ],
  "filing_date": "2012-08-14",
  "publication_date": "2017-07-04",
  "grant_date": "2017-07-04",
  "priority_date": "2012-08-14",
  "application_number": "US-201213585606-A",
  "family_id": "49029209",
  "cited_by_count": 36,
  "citations": [
    "US4695953A",
    "US4630910A",
    "US4627620A",
    "US4645458A",
    "US4702475A",
    "US4843568A",
    "US4711543A",
    "US4796997A",
    "US5184295A",
    "US4751642A",
    "US4809065A",
    "US4817950A",
    "US5239464A",
    "US5239463A",
    "US4901362A",
    "US4893183A",
    "US5288078A",
    "US4925189A",
    "US5229756A",
    "US5469740A",
    "US5229754A",
    "US5101444A",
    "US5148154A",
    "US5534917A",
    "US5295491A",
    "US6054991A",
    "WO1993010708A1",
    "US5875108A",
    "US5417210A",
    "EP0583061A2",
    "US7222078B2",
    "US5320538A",
    "US5715834A",
    "US5495576A",
    "US5690582A",
    "US5704837A",
    "US5405152A",
    "US5454043A",
    "US5423554A",
    "US5980256A",
    "US5617312A",
    "US5347306A",
    "US5616078A",
    "US5577981A",
    "US5580249A",
    "US5597309A",
    "US5385519A",
    "US5524637A",
    "JPH0844490A",
    "US5563988A",
    "US6714665B1",
    "US5516105A",
    "US5638300A",
    "US5703367A",
    "US5594469A",
    "US5682229A",
    "US5913727A",
    "US6229913B1",
    "US5682196A",
    "US6066075A",
    "US6765726B2",
    "US6073489A",
    "US6430997B1",
    "WO1997017598A1",
    "US7359121B2",
    "US7038855B2",
    "US6308565B1",
    "US6098458A",
    "US6876496B2",
    "US6283860B1",
    "US5933125A",
    "US5641288A",
    "US6152856A",
    "US6173066B1",
    "US5989157A",
    "US6005548A",
    "US5995649A",
    "US6128003A",
    "US6009210A",
    "US6100896A",
    "US5877803A",
    "US6215898B1",
    "US6226396B1",
    "US6188777B1",
    "US6289112B1",
    "US7042440B2",
    "US6496598B1",
    "US6215890B1",
    "US6141463A",
    "US6411744B1",
    "USRE42256E1",
    "US6101289A",
    "US6130677A",
    "US7746345B2",
    "US6384819B1",
    "US6256033B1",
    "US6072494A",
    "US7184048B2",
    "US6181343B1",
    "WO1999044698A2",
    "US6159100A",
    "US6077201A",
    "US7684592B2",
    "US7036094B1",
    "US6681031B2",
    "US7460690B2",
    "US7121946B2",
    "US7668340B2",
    "US6950534B2",
    "US6316934B1",
    "US6256400B1",
    "US20090046152A1",
    "US6661918B1",
    "US6147678A",
    "US7202898B1",
    "US6570555B1",
    "US6363160B1",
    "US7003134B1",
    "US6299308B1",
    "US6503195B1",
    "US6476834B1",
    "US6873723B1",
    "EP1067362A1",
    "US6738066B1",
    "US7760182B2",
    "US7113918B1",
    "US6801637B2",
    "US7050606B2",
    "US7367887B2",
    "US6633294B1",
    "US7060957B2",
    "US6640202B1",
    "US6731799B1",
    "US6788809B1",
    "US7227526B2",
    "US7898522B2",
    "US7555142B2",
    "US7421093B2",
    "US7058204B2",
    "US7039676B1",
    "US6539931B2",
    "US7259747B2",
    "US7834846B1",
    "US7452275B2",
    "US7680298B2",
    "US6937742B2",
    "US7607509B2",
    "US8035624B2",
    "US8035614B2",
    "US8035612B2",
    "US7710391B2",
    "US7170492B2",
    "US7348963B2",
    "US7489812B2",
    "US7576727B2",
    "US7436496B2",
    "US20040196282A1",
    "US7593552B2",
    "US7590262B2",
    "US8072470B2",
    "US7620202B2",
    "US7809167B2",
    "US7536032B2",
    "US7379563B2",
    "US7308112B2",
    "US20080024594A1",
    "US20070285419A1",
    "US7704135B2",
    "US7702130B2",
    "US7570805B2",
    "US7430312B2",
    "US7379566B2",
    "US7574020B2",
    "US7598942B2",
    "US7317836B2",
    "US7389591B2",
    "US7560701B2",
    "US20080026838A1",
    "US7450736B2",
    "US20100194862A1",
    "US7701439B2",
    "US7683954B2",
    "US20100063681A1",
    "US7412077B2",
    "US7729530B2",
    "US20080239109A1",
    "US7852262B2",
    "US20090059041A1",
    "CN201254344Y",
    "EP2378310A1",
    "US20110310226A1",
    "KR20120018915A",
    "US20130266174A1"
  ]
}

Record 4,033 of 8,000 in Patents full text (MLC-0201). Request the full dataset.