MLchartDataset catalogue

Patent · US10924638B2 · B2 · US

Compact, low cost VCSEL projector for high performance stereodepth camera

(11) Publication number
US10924638B2
(21) Application number
15/193,956
(22) Filing date
2016-06-27
(30) Priority date
2016-06-27
(43) Publication date
2021-02-16
(45) Date of grant
2021-02-16
(51) IPC
F21V 5/00; G02B 3/00; H01S 5/42; H04N 13/239; H04N 23/20; G01B 11/25; G01C 3/08; G01S 3/00; G06F 3/01; G06F 3/03; G06F 3/0354; G06F 3/0488; H01S 5/00; H01S 5/022; H04N 13/257
(52) CPC
  • H04N Pictorial communication, e.g. television: 13/239, 13/257, 23/20, 23/81, 5/217, 5/33
  • F21V Functional features or details of lighting devices or systems thereof; structural combinations of lighting devices with other articles, not otherwise provided for: 5/007, 5/008
  • G01B Measuring length, thickness or similar linear dimensions; measuring angles; measuring areas; measuring irregularities of surfaces or contours: 11/2513
  • G01C Measuring distances, levels or bearings; surveying; navigation; gyroscopic instruments; photogrammetry or videogrammetry: 3/08
  • G01S Radio direction-finding; radio navigation; determining distance or velocity by use of radio waves; locating or presence-detecting by use of the reflection or reradiation of radio waves; analogous arrangements using other waves: 3/00
  • G02B Optical elements, systems or apparatus: 3/0056
  • G06F Electric digital data processing: 3/011, 3/012, 3/017, 3/0304, 3/03547, 3/04883
  • H01S Devices using the process of light amplification by stimulated emission of radiation [laser] to amplify or generate light; devices using stimulated emission of electromagnetic radiation in wave ranges other than optical: 5/005, 5/02255, 5/02292, 5/423, 5/426
(73) Assignee
Intel Corp
(72) Inventors
Krishna Swaminathan; Anders Grunnet-Jepsen; Leonid Keselman
(54) Title
Compact, low cost VCSEL projector for high performance stereodepth camera
(57) Abstract

A VCSEL projector and method for using the same are disclosed. In one embodiment, the apparatus comprises a vertical cavity surface emitting laser (VCSEL) array comprising a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array; and a projection lens coupled to the micro-lens array (MLA), where light emitted by the VCSEL array is projected as a sequence of patterns onto an object by the projection lens.

Full text
View on Google Patents

Claims (26)

  1. An apparatus comprising: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is positioned at an offset with respect to a centerline of a corresponding VCSEL and the lens is directly coupled to the corresponding VCSEL; and a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as a sequence of patterns onto an object by the projection lens.
  2. The apparatus defined in claim 1, wherein the plurality of lenses of the MLA cause light emitted by VCSELs in the VCSEL array to bend so that more light emitted from the VCSELs travels through the projection lens.
  3. The apparatus defined in claim 2, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array.
  4. The apparatus defined in claim 3, wherein the plurality of lenses of the MLA bend light emitted by the VCSELs in an increasing amount as a lens of the plurality of lenses is closer to an edge of a field of view.
  5. The apparatus defined in claim 2, wherein the center of each lens of the plurality of lenses and a corresponding VCSEL are misaligned to cause emitted light from the corresponding VCSEL to be bent.
  6. The apparatus defined in claim 5, wherein a lens and a corresponding VCSEL with more misalignment bends light more than a lens and corresponding VCSEL with less misalignment.
  7. The apparatus defined in claim 1, wherein the light includes infra-red (IR) light and the projection lens includes an IR projection lens.
  8. An apparatus comprising: a projector configured to project a sequence of light patterns on an object, wherein the projector includes: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is at an offset with respect to a centerline of a corresponding VCSEL; a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as the sequence of light patterns onto the object by the projection lens, wherein the offset between the centerline of the lens and the centerline of the corresponding VCSEL is dependent on a location of the projection lens with respect to the VCSEL array; a first camera configured to capture a sequence of images of the object illuminated with the projected light patterns; and a processing unit to receive the sequence of images and reconstruct depth information using triangulation.
  9. The apparatus defined in claim 8, wherein the plurality of lenses of the MLA cause light emitted by VCSELs in the VCSEL array to bend so that more light emitted from the VCSELs travels through the projection lens.
  10. The apparatus defined in claim 9, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array.
  11. The apparatus defined in claim 10, wherein the plurality of lenses of the MLA bend light emitted by the VCSELs in an increasing amount as a lens of the plurality of lenses is closer to an edge of a field of view.
  12. The apparatus defined in claim 9, wherein the center of each lens of the plurality of lenses and a corresponding VCSEL are misaligned to cause emitted light from the corresponding VCSEL to be bent.
  13. The apparatus defined in claim 12, wherein a lens and a corresponding VCSEL with more misalignment bends light more than a lens and corresponding VCSEL with less misalignment.
  14. The apparatus defined in claim 8, wherein the projector includes an infra-red (IR) projector and the projection lens includes an IR projection lens.
  15. The apparatus defined in claim 8, inlcuding a second camera configured to capture a second image of the object illuminated with the projected light patterns and configured to capture a texture image of the object in the second image.
  16. The apparatus defined in claim 15, wherein the projector includes an infrared (IR) projector, the first camera includes an IR camera, and the second camera includes a red, green, and blue (RGB) camera.
  17. A method comprising: projecting a sequence of light patterns on an object using a projector, wherein the projector includes: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is at an offset with respect to a centerline of a corresponding VCSEL and the lens is directly coupled to the corresponding VCSEL; a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as the sequence of light patterns onto the object by the projection lens, wherein the offset between the centerline of the lens and the centerline of the corresponding VCSEL is dependent on a location of the projection lens with respect to the VCSEL array; capturing, using a camera, a sequence of images of the sequence of light patterns; and generating depth information via triangulation using the sequence of images.
  18. The method defined in claim 17, including bending light emitted by VCSELs in the VCSEL array using the plurality of lenses of the MLA so that more light emitted from the VCSELs travels through the projection lens.
  19. The method defined in claim 18, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array.
  20. The method defined in claim 19, wherein the plurality of lenses of the MLA bend light emitted by the VCSELs in an increasing amount as a lens of the plurality of lenses is closer to an edge of a field of view.
  21. The method defined in claim 18, wherein the center of each lens of the plurality of lenses and a corresponding VCSEL are misaligned to cause emitted light from the corresponding VCSEL to be bent.
  22. The method defined in claim 21, wherein a lens and a corresponding VCSEL with more misalignment bends light more than a lens and corresponding VCSEL with less misalignment.
  23. The method defined in claim 17, wherein the light includes infra-red (IR) light and the projection lens includes an IR projection lens.
  24. An article of manufacture having one or more non-transitory computer readable storage media comprising instructions which, when executed by a system, cause the system to perform operations including: receiving a sequence of images of a sequence of light patterns captured by a camera, wherein the light patterns are projected by a projector that includes: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is at an offset with respect to a centerline of a corresponding VCSEL and the lens is directly coupled to the corresponding VCSEL; a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as a sequence of patterns onto an object by the projection lens, wherein the offset between the centerline of the lens and the centerline of the corresponding VCSEL is dependent on a location of the projection lens with respect to the VCSEL array; and generating depth information via triangulation using the sequence of images.
  25. The article of manufacture defined in claim 24, wherein the method further includes bending light emitted by VCSELs in the VCSEL array using the plurality of lenses of the MLA so that more light emitted from the VCSELs travels through the projection lens.
  26. The article of manufacture defined in claim 25, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array.

Description

Embodiments of the present invention related to camera systems; more particularly, embodiments of the present invention relate to systems with VCSEL camera systems that use lenses to bend light.

Stereo depth cameras are well-known and are often used to measure a distance from an object. One such measurement device includes a projector and a camera. In such a device, the projector projects a known pattern image on an object (e.g., a scene), and an image of the object upon which the image is projected is captured by the camera. From the captured images, depth information may be determined. One technique for determining depth in such devices is through the use of triangulation. Thus, images of objects are captured and measurements are taken to determine depth information.

It is well known that use of an infra-red (IR) laser projector to project a textured pattern onto the target provides a significant boost to the performance of stereoscopic depth cameras. The projected pattern adds texture to the scene and allows high accuracy depth imaging of even targets with minimal or no texture such as a wall. In the case of stereo cameras using structured light approach, the knowledge of the size and distance between the features in the projected pattern is even more important and acts as the main mechanism to achieve accurate depth maps. Due to these reasons, an IR laser pattern projector has been widely used in almost all stereoscopic depth cameras.

Citations (26)

  • US20020043561A1
  • US20040218875A1
  • US20070030570A1
  • US20080187013A1
  • US20100118123A1
  • US20110243178A1
  • US20120051588A1
  • US20160197452A1
  • US20150292709A1
  • US20140375775A1
  • KR20130037152A
  • US20140158862A1
  • US20140176958A1
  • US20140240464A1
  • US20160109575A1
  • US20160072256A1
  • US20150097947A1
  • US9443310B2
  • US20160131318A1
  • US20180131450A1
  • US20180225866A1
  • US20170188012A1
  • US20180006431A1
  • US20190249984A1
  • US20180197842A1
  • US20190107604A1
Record as JSON 
{
  "publication_number": "US10924638B2",
  "country": "US",
  "kind": "B2",
  "title": "Compact, low cost VCSEL projector for high performance stereodepth camera",
  "abstract": "A VCSEL projector and method for using the same are disclosed. In one embodiment, the apparatus comprises a vertical cavity surface emitting laser (VCSEL) array comprising a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array; and a projection lens coupled to the micro-lens array (MLA), where light emitted by the VCSEL array is projected as a sequence of patterns onto an object by the projection lens.",
  "claims": [
    "1. An apparatus comprising: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is positioned at an offset with respect to a centerline of a corresponding VCSEL and the lens is directly coupled to the corresponding VCSEL; and a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as a sequence of patterns onto an object by the projection lens.",
    "2. The apparatus defined in claim 1, wherein the plurality of lenses of the MLA cause light emitted by VCSELs in the VCSEL array to bend so that more light emitted from the VCSELs travels through the projection lens.",
    "3. The apparatus defined in claim 2, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array.",
    "4. The apparatus defined in claim 3, wherein the plurality of lenses of the MLA bend light emitted by the VCSELs in an increasing amount as a lens of the plurality of lenses is closer to an edge of a field of view.",
    "5. The apparatus defined in claim 2, wherein the center of each lens of the plurality of lenses and a corresponding VCSEL are misaligned to cause emitted light from the corresponding VCSEL to be bent.",
    "6. The apparatus defined in claim 5, wherein a lens and a corresponding VCSEL with more misalignment bends light more than a lens and corresponding VCSEL with less misalignment.",
    "7. The apparatus defined in claim 1, wherein the light includes infra-red (IR) light and the projection lens includes an IR projection lens.",
    "8. An apparatus comprising: a projector configured to project a sequence of light patterns on an object, wherein the projector includes: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is at an offset with respect to a centerline of a corresponding VCSEL; a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as the sequence of light patterns onto the object by the projection lens, wherein the offset between the centerline of the lens and the centerline of the corresponding VCSEL is dependent on a location of the projection lens with respect to the VCSEL array; a first camera configured to capture a sequence of images of the object illuminated with the projected light patterns; and a processing unit to receive the sequence of images and reconstruct depth information using triangulation.",
    "9. The apparatus defined in claim 8, wherein the plurality of lenses of the MLA cause light emitted by VCSELs in the VCSEL array to bend so that more light emitted from the VCSELs travels through the projection lens.",
    "10. The apparatus defined in claim 9, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array.",
    "11. The apparatus defined in claim 10, wherein the plurality of lenses of the MLA bend light emitted by the VCSELs in an increasing amount as a lens of the plurality of lenses is closer to an edge of a field of view.",
    "12. The apparatus defined in claim 9, wherein the center of each lens of the plurality of lenses and a corresponding VCSEL are misaligned to cause emitted light from the corresponding VCSEL to be bent.",
    "13. The apparatus defined in claim 12, wherein a lens and a corresponding VCSEL with more misalignment bends light more than a lens and corresponding VCSEL with less misalignment.",
    "14. The apparatus defined in claim 8, wherein the projector includes an infra-red (IR) projector and the projection lens includes an IR projection lens.",
    "15. The apparatus defined in claim 8, inlcuding a second camera configured to capture a second image of the object illuminated with the projected light patterns and configured to capture a texture image of the object in the second image.",
    "16. The apparatus defined in claim 15, wherein the projector includes an infrared (IR) projector, the first camera includes an IR camera, and the second camera includes a red, green, and blue (RGB) camera.",
    "17. A method comprising: projecting a sequence of light patterns on an object using a projector, wherein the projector includes: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is at an offset with respect to a centerline of a corresponding VCSEL and the lens is directly coupled to the corresponding VCSEL; a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as the sequence of light patterns onto the object by the projection lens, wherein the offset between the centerline of the lens and the centerline of the corresponding VCSEL is dependent on a location of the projection lens with respect to the VCSEL array; capturing, using a camera, a sequence of images of the sequence of light patterns; and generating depth information via triangulation using the sequence of images.",
    "18. The method defined in claim 17, including bending light emitted by VCSELs in the VCSEL array using the plurality of lenses of the MLA so that more light emitted from the VCSELs travels through the projection lens.",
    "19. The method defined in claim 18, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array.",
    "20. The method defined in claim 19, wherein the plurality of lenses of the MLA bend light emitted by the VCSELs in an increasing amount as a lens of the plurality of lenses is closer to an edge of a field of view.",
    "21. The method defined in claim 18, wherein the center of each lens of the plurality of lenses and a corresponding VCSEL are misaligned to cause emitted light from the corresponding VCSEL to be bent.",
    "22. The method defined in claim 21, wherein a lens and a corresponding VCSEL with more misalignment bends light more than a lens and corresponding VCSEL with less misalignment.",
    "23. The method defined in claim 17, wherein the light includes infra-red (IR) light and the projection lens includes an IR projection lens.",
    "24. An article of manufacture having one or more non-transitory computer readable storage media comprising instructions which, when executed by a system, cause the system to perform operations including: receiving a sequence of images of a sequence of light patterns captured by a camera, wherein the light patterns are projected by a projector that includes: a vertical cavity surface emitting laser (VCSEL) array including a plurality of VCSELs; a micro-lens array coupled to the VCSEL array and having a plurality of lenses, and each of the plurality of lenses is positioned over a VCSEL in the VCSEL array, wherein a centerline of a lens of the plurality of lenses is at an offset with respect to a centerline of a corresponding VCSEL and the lens is directly coupled to the corresponding VCSEL; a projection lens coupled to the micro-lens array (MLA), wherein light emitted by the VCSEL array is projected as a sequence of patterns onto an object by the projection lens, wherein the offset between the centerline of the lens and the centerline of the corresponding VCSEL is dependent on a location of the projection lens with respect to the VCSEL array; and generating depth information via triangulation using the sequence of images.",
    "25. The article of manufacture defined in claim 24, wherein the method further includes bending light emitted by VCSELs in the VCSEL array using the plurality of lenses of the MLA so that more light emitted from the VCSELs travels through the projection lens.",
    "26. The article of manufacture defined in claim 25, wherein the plurality of lenses of the MLA that are farther from a center of the VCSEL array are operable to bend the light more than those closer to the center of the VCSEL array."
  ],
  "description_excerpt": "Embodiments of the present invention related to camera systems; more particularly, embodiments of the present invention relate to systems with VCSEL camera systems that use lenses to bend light.\n\nStereo depth cameras are well-known and are often used to measure a distance from an object. One such measurement device includes a projector and a camera. In such a device, the projector projects a known pattern image on an object (e.g., a scene), and an image of the object upon which the image is projected is captured by the camera. From the captured images, depth information may be determined. One technique for determining depth in such devices is through the use of triangulation. Thus, images of objects are captured and measurements are taken to determine depth information.\n\nIt is well known that use of an infra-red (IR) laser projector to project a textured pattern onto the target provides a significant boost to the performance of stereoscopic depth cameras. The projected pattern adds texture to the scene and allows high accuracy depth imaging of even targets with minimal or no texture such as a wall. In the case of stereo cameras using structured light approach, the knowledge of the size and distance between the features in the projected pattern is even more important and acts as the main mechanism to achieve accurate depth maps. Due to these reasons, an IR laser pattern projector has been widely used in almost all stereoscopic depth cameras.",
  "cpc": [
    "H04N 13/239",
    "F21V 5/007",
    "F21V 5/008",
    "G01B 11/2513",
    "G01C 3/08",
    "G01S 3/00",
    "G02B 3/0056",
    "G06F 3/011",
    "G06F 3/012",
    "G06F 3/017",
    "G06F 3/0304",
    "G06F 3/03547",
    "G06F 3/04883",
    "H01S 5/005",
    "H01S 5/02255",
    "H01S 5/02292",
    "H01S 5/423",
    "H01S 5/426",
    "H04N 13/257",
    "H04N 23/20",
    "H04N 23/81",
    "H04N 5/217",
    "H04N 5/33"
  ],
  "ipc": [
    "F21V 5/00",
    "G02B 3/00",
    "H01S 5/42",
    "H04N 13/239",
    "H04N 23/20",
    "G01B 11/25",
    "G01C 3/08",
    "G01S 3/00",
    "G06F 3/01",
    "G06F 3/03",
    "G06F 3/0354",
    "G06F 3/0488",
    "H01S 5/00",
    "H01S 5/022",
    "H04N 13/257"
  ],
  "assignees": [
    "Intel Corp"
  ],
  "inventors": [
    "Krishna Swaminathan",
    "Anders Grunnet-Jepsen",
    "Leonid Keselman"
  ],
  "filing_date": "2016-06-27",
  "publication_date": "2021-02-16",
  "grant_date": "2021-02-16",
  "priority_date": "2016-06-27",
  "application_number": "US-201615193956-A",
  "family_id": "60677904",
  "cited_by_count": 7,
  "citations": [
    "US20020043561A1",
    "US20040218875A1",
    "US20070030570A1",
    "US20080187013A1",
    "US20100118123A1",
    "US20110243178A1",
    "US20120051588A1",
    "US20160197452A1",
    "US20150292709A1",
    "US20140375775A1",
    "KR20130037152A",
    "US20140158862A1",
    "US20140176958A1",
    "US20140240464A1",
    "US20160109575A1",
    "US20160072256A1",
    "US20150097947A1",
    "US9443310B2",
    "US20160131318A1",
    "US20180131450A1",
    "US20180225866A1",
    "US20170188012A1",
    "US20180006431A1",
    "US20190249984A1",
    "US20180197842A1",
    "US20190107604A1"
  ]
}

Record 1,751 of 8,000 in Patents full text (MLC-0201). Request the full dataset.