Patent · US9736459B2 · B2 · US

Generation of patterned radiation

(11) Publication number

US9736459B2

(21) Application number

13/765,706

(22) Filing date

2013-02-13

(30) Priority date

2010-02-02

(43) Publication date

2017-08-15

(45) Date of grant

2017-08-15

(51) IPC

G03B 15/06; H04N 13/02; H04N 23/75; G02B 19/00; G02B 27/00; H01S 5/00; H01S 5/022; H01S 5/40; H01S 5/42

(52) CPC

G03B Apparatus or arrangements for taking photographs or for projecting or viewing them; apparatus or arrangements employing analogous techniques using waves other than optical waves; accessories therefor: 21/2033, 15/06, 21/2066, 21/28
G02B Optical elements, systems or apparatus: 19/0028, 19/0057, 27/0944, 27/0972, 27/0983
H01L Semiconductor devices; electric solid state devices not otherwise provided for: 33/60
H01S Devices using the process of light amplification by stimulated emission of radiation [laser] to amplify or generate light; devices using stimulated emission of electromagnetic radiation in wave ranges other than optical: 5/005, 5/0071, 5/02248, 5/02253, 5/02255, 5/02257, 5/02315, 5/02325, 5/4031, 5/423
H04N Pictorial communication, e.g. television: 13/0207, 13/207
H10H Inorganic light-emitting semiconductor devices having potential barriers: 20/856

(73) Assignee

Apple Inc

(72) Inventors

Zafrir Mor; Alexander Shpunt

(54) Title

Generation of patterned radiation

(57) Abstract

Imaging apparatus includes an illumination assembly, including a plurality of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene. An imaging assembly includes an image sensor and objective optics configured to form an optical image of the scene on the image sensor, which includes an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form an electronic image of the scene. A controller is coupled to actuate the radiation sources sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter.

Full text

View on Google Patents

Claims (16)

Optical apparatus, comprising: a plurality of radiation sources, mounted on a substrate and configured to emit optical radiation; a patterning element, comprising a transparency; collection optics, comprising an array of micro-optics, which are aligned with the radiation sources so that a respective micro-optic collects the radiation emitted by each of the radiation sources and directs the radiation toward the patterning element, while causing the optical radiation emitted by each of the radiation sources to pass through a different, respective region of the transparency so as to form different, respective patterns; projection optics, which are configured to project the patterns of the radiation from the patterning element onto a scene; an imaging assembly, comprising an image sensor and objective optics configured to form an optical image of the scene on the image sensor, so that the image sensor forms electronic images of the scene; and a controller, which is coupled to actuate the radiation sources sequentially in a pulsed mode, so as to form the different, respective patterns in succession, and to analyze the patterns in the electronic images so as to generate a depth map of the scene by analyzing multiple images, generated in succession by the image sensor, containing the different, respective patterns formed due to emission from each of the plurality of radiation sources.
The apparatus according to claim 1, wherein the transparency comprises a micro-lens array.
The apparatus according to claim 1, wherein the image sensor comprises an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene, and wherein the controller is coupled to actuate the radiation sources in synchronization with the rolling shutter.
The apparatus according to claim 1, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction perpendicular to the substrate.
The apparatus according to claim 1, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction parallel to the substrate.
The apparatus according to claim 5, wherein the collection optics comprise a reflector disposed on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the patterning element.
The apparatus according to claim 6, wherein the reflector comprises a prism.
A method for imaging, comprising: arranging a plurality of radiation sources on a substrate so as to emit optical radiation; directing the radiation emitted by the radiation sources toward a patterning element, comprising a transparency, using an array of micro-optics that are aligned with the radiation sources so that a respective micro-optic collects the radiation emitted by each of the radiation sources so as to cause the optical radiation emitted by each of the radiation sources to pass through a different, respective region of the transparency, thus forming different, respective patterns; projecting the patterns of the radiation from the patterning element onto a scene; actuating the radiation sources sequentially in a pulsed mode so as to form the different, respective patterns in succession; forming an optical image of the scene on an image sensor, so that the image sensor forms a succession of electronic images of the scene containing the different, respective patterns formed due to emission from each of the plurality of radiation sources; and analyzing the patterns in the succession of electronic images so as to generate a depth map of the scene, wherein analyzing the patterns comprises analyzing images containing the different, respective patterns formed due to emission from each of the plurality of radiation sources in succession.
The method according to claim 8, wherein the transparency comprises a micro-lens array.
The method according to claim 8, wherein the image sensor comprises an array of sensor elements arranged in multiple groups, in which the groups of the sensor elements receive the radiation from different, respective areas of the scene, wherein the method comprises triggering the groups of the sensor elements with a rolling shutter to capture the radiation from the scene in successive, respective exposure periods so as to form the electronic image of the scene, and wherein actuating the radiation sources sequentially in the pulsed mode comprises operating the radiation sources in synchronization with the rolling shutter.
The method according to claim 8, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction perpendicular to the substrate.
The method according to claim 8, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction parallel to the substrate.
The method according to claim 12, wherein arranging the plurality of the radiation sources comprises providing a reflector on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the scene.
The apparatus according to claim 13, wherein the reflector comprises a prism.
The apparatus according to claim 1, wherein the micro-optics comprises microlenses.
The method according to claim 8, wherein the micro-optics comprise microlenses.

Description

The present invention relates generally to systems and methods for electronic imaging, and specifically to methods of illumination for enhancing the quality of captured images.

Most low-cost CMOS image sensors use a rolling shutter, in which successive rows of sensor elements are triggered sequentially to capture light. This method of image acquisition thus records each individual frame not as a single snapshot at a point in time, but rather as a sequence of image stripes scanning across the frame. The result of the rolling shutter is that not all parts of the optical image are recorded at exactly the same time (although the frame is stored as a single electronic image).

The use of a rolling shutter introduces a temporal shear in the image frame, which can create artifacts in imaging of moving objects. Bradley et al. address this problem in “Synchronization and Rolling Shutter Compensation for Consumer Video Camera Arrays,” IEEE International Workshop on Projector-Camera Systems - PROCAMS 2009 (Miami Beach, Fla., 2009), which is incorporated herein by reference. The authors propose to solve the problem using synchronized stroboscopic illumination.

Embodiments of the present invention that are described hereinbelow provide apparatus and methods for illuminating an object that can be advantageous when the object is imaged using a sensor with a rolling shutter.

Citations (73)

US3796498A
JPS6211286A
US4850673A
US5648951A
US5691989A
US5477383A
US5406543A
US5742262A
US5606181A
US5781332A
JPH10123512A
JPH10301201A
US6002520A
US6031611A
US6560019B2
US20040082112A1
US6927852B2
US6583873B1
US6707027B2
US6611000B2
US20050088644A1
US20040012958A1
JP2002372701A
US20030090900A1
US20040184270A1
US20040258354A1
US6940583B2
US7112774B2
US20050178950A1
US20060001055A1
US7227618B1
US7304735B2
US7335898B2
US20060044803A1
US20060252169A1
US7952781B2
US20060252167A1
US20060269896A1
CN1725042A
US20070019909A1
WO2007043036A1
US20080106746A1
US20090096783A1
US20100013860A1
WO2007105205A2
US20100142014A1
US20080198355A1
US20080212835A1
WO2008120217A2
US20080240502A1
US20080278572A1
US7700904B2
US20090090937A1
US20090183125A1
US20090185274A1
US8384997B2
US20110075259A1
US20110069389A1
US20110019258A1
WO2010004542A1
US20100007717A1
US20110114857A1
JP2011118178A
US20110188054A1
US20110187878A1
US20110295331A1
US20120038986A1
WO2012020380A1
WO2012066501A1
US20120140109A1
US20120140094A1
US20130038941A1
US20130038881A1

Record as JSON

{
  "publication_number": "US9736459B2",
  "country": "US",
  "kind": "B2",
  "title": "Generation of patterned radiation",
  "abstract": "Imaging apparatus includes an illumination assembly, including a plurality of radiation sources and projection optics, which are configured to project radiation from the radiation sources onto different, respective regions of a scene. An imaging assembly includes an image sensor and objective optics configured to form an optical image of the scene on the image sensor, which includes an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene so as to form an electronic image of the scene. A controller is coupled to actuate the radiation sources sequentially in a pulsed mode so that the illumination assembly illuminates the different, respective areas of the scene in synchronization with the rolling shutter.",
  "claims": [
    "1. Optical apparatus, comprising: a plurality of radiation sources, mounted on a substrate and configured to emit optical radiation; a patterning element, comprising a transparency; collection optics, comprising an array of micro-optics, which are aligned with the radiation sources so that a respective micro-optic collects the radiation emitted by each of the radiation sources and directs the radiation toward the patterning element, while causing the optical radiation emitted by each of the radiation sources to pass through a different, respective region of the transparency so as to form different, respective patterns; projection optics, which are configured to project the patterns of the radiation from the patterning element onto a scene; an imaging assembly, comprising an image sensor and objective optics configured to form an optical image of the scene on the image sensor, so that the image sensor forms electronic images of the scene; and a controller, which is coupled to actuate the radiation sources sequentially in a pulsed mode, so as to form the different, respective patterns in succession, and to analyze the patterns in the electronic images so as to generate a depth map of the scene by analyzing multiple images, generated in succession by the image sensor, containing the different, respective patterns formed due to emission from each of the plurality of radiation sources.",
    "2. The apparatus according to claim 1, wherein the transparency comprises a micro-lens array.",
    "3. The apparatus according to claim 1, wherein the image sensor comprises an array of sensor elements arranged in multiple groups, which are triggered by a rolling shutter to capture the radiation from the scene in successive, respective exposure periods from different, respective areas of the scene, and wherein the controller is coupled to actuate the radiation sources in synchronization with the rolling shutter.",
    "4. The apparatus according to claim 1, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction perpendicular to the substrate.",
    "5. The apparatus according to claim 1, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction parallel to the substrate.",
    "6. The apparatus according to claim 5, wherein the collection optics comprise a reflector disposed on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the patterning element.",
    "7. The apparatus according to claim 6, wherein the reflector comprises a prism.",
    "8. A method for imaging, comprising: arranging a plurality of radiation sources on a substrate so as to emit optical radiation; directing the radiation emitted by the radiation sources toward a patterning element, comprising a transparency, using an array of micro-optics that are aligned with the radiation sources so that a respective micro-optic collects the radiation emitted by each of the radiation sources so as to cause the optical radiation emitted by each of the radiation sources to pass through a different, respective region of the transparency, thus forming different, respective patterns; projecting the patterns of the radiation from the patterning element onto a scene; actuating the radiation sources sequentially in a pulsed mode so as to form the different, respective patterns in succession; forming an optical image of the scene on an image sensor, so that the image sensor forms a succession of electronic images of the scene containing the different, respective patterns formed due to emission from each of the plurality of radiation sources; and analyzing the patterns in the succession of electronic images so as to generate a depth map of the scene, wherein analyzing the patterns comprises analyzing images containing the different, respective patterns formed due to emission from each of the plurality of radiation sources in succession.",
    "9. The method according to claim 8, wherein the transparency comprises a micro-lens array.",
    "10. The method according to claim 8, wherein the image sensor comprises an array of sensor elements arranged in multiple groups, in which the groups of the sensor elements receive the radiation from different, respective areas of the scene, wherein the method comprises triggering the groups of the sensor elements with a rolling shutter to capture the radiation from the scene in successive, respective exposure periods so as to form the electronic image of the scene, and wherein actuating the radiation sources sequentially in the pulsed mode comprises operating the radiation sources in synchronization with the rolling shutter.",
    "11. The method according to claim 8, wherein the radiation sources comprise a matrix of light-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction perpendicular to the substrate.",
    "12. The method according to claim 8, wherein the radiation sources comprise a row of edge-emitting elements, which are arranged on the substrate and are configured to emit the radiation in a direction parallel to the substrate.",
    "13. The method according to claim 12, wherein arranging the plurality of the radiation sources comprises providing a reflector on the substrate so as to turn the radiation emitted by the edge-emitting elements away from the substrate and toward the scene.",
    "14. The apparatus according to claim 13, wherein the reflector comprises a prism.",
    "15. The apparatus according to claim 1, wherein the micro-optics comprises microlenses.",
    "16. The method according to claim 8, wherein the micro-optics comprise microlenses."
  ],
  "description_excerpt": "The present invention relates generally to systems and methods for electronic imaging, and specifically to methods of illumination for enhancing the quality of captured images.\n\nMost low-cost CMOS image sensors use a rolling shutter, in which successive rows of sensor elements are triggered sequentially to capture light. This method of image acquisition thus records each individual frame not as a single snapshot at a point in time, but rather as a sequence of image stripes scanning across the frame. The result of the rolling shutter is that not all parts of the optical image are recorded at exactly the same time (although the frame is stored as a single electronic image).\n\nThe use of a rolling shutter introduces a temporal shear in the image frame, which can create artifacts in imaging of moving objects. Bradley et al. address this problem in “Synchronization and Rolling Shutter Compensation for Consumer Video Camera Arrays,” IEEE International Workshop on Projector-Camera Systems - PROCAMS 2009 (Miami Beach, Fla., 2009), which is incorporated herein by reference. The authors propose to solve the problem using synchronized stroboscopic illumination.\n\nEmbodiments of the present invention that are described hereinbelow provide apparatus and methods for illuminating an object that can be advantageous when the object is imaged using a sensor with a rolling shutter.",
  "cpc": [
    "G03B 21/2033",
    "G02B 19/0028",
    "G02B 19/0057",
    "G02B 27/0944",
    "G02B 27/0972",
    "G02B 27/0983",
    "G03B 15/06",
    "G03B 21/2066",
    "G03B 21/28",
    "H01L 33/60",
    "H01S 5/005",
    "H01S 5/0071",
    "H01S 5/02248",
    "H01S 5/02253",
    "H01S 5/02255",
    "H01S 5/02257",
    "H01S 5/02315",
    "H01S 5/02325",
    "H01S 5/4031",
    "H01S 5/423",
    "H04N 13/0207",
    "H04N 13/207",
    "H10H 20/856"
  ],
  "ipc": [
    "G03B 15/06",
    "H04N 13/02",
    "H04N 23/75",
    "G02B 19/00",
    "G02B 27/00",
    "H01S 5/00",
    "H01S 5/022",
    "H01S 5/40",
    "H01S 5/42"
  ],
  "assignees": [
    "Apple Inc"
  ],
  "inventors": [
    "Zafrir Mor",
    "Alexander Shpunt"
  ],
  "filing_date": "2013-02-13",
  "publication_date": "2017-08-15",
  "grant_date": "2017-08-15",
  "priority_date": "2010-02-02",
  "application_number": "US-201313765706-A",
  "family_id": "44341315",
  "cited_by_count": 57,
  "citations": [
    "US3796498A",
    "JPS6211286A",
    "US4850673A",
    "US5648951A",
    "US5691989A",
    "US5477383A",
    "US5406543A",
    "US5742262A",
    "US5606181A",
    "US5781332A",
    "JPH10123512A",
    "JPH10301201A",
    "US6002520A",
    "US6031611A",
    "US6560019B2",
    "US20040082112A1",
    "US6927852B2",
    "US6583873B1",
    "US6707027B2",
    "US6611000B2",
    "US20050088644A1",
    "US20040012958A1",
    "JP2002372701A",
    "US20030090900A1",
    "US20040184270A1",
    "US20040258354A1",
    "US6940583B2",
    "US7112774B2",
    "US20050178950A1",
    "US20060001055A1",
    "US7227618B1",
    "US7304735B2",
    "US7335898B2",
    "US20060044803A1",
    "US20060252169A1",
    "US7952781B2",
    "US20060252167A1",
    "US20060269896A1",
    "CN1725042A",
    "US20070019909A1",
    "WO2007043036A1",
    "US20080106746A1",
    "US20090096783A1",
    "US20100013860A1",
    "WO2007105205A2",
    "US20100142014A1",
    "US20080198355A1",
    "US20080212835A1",
    "WO2008120217A2",
    "US20080240502A1",
    "US20080278572A1",
    "US7700904B2",
    "US20090090937A1",
    "US20090183125A1",
    "US20090185274A1",
    "US8384997B2",
    "US20110075259A1",
    "US20110069389A1",
    "US20110019258A1",
    "WO2010004542A1",
    "US20100007717A1",
    "US20110114857A1",
    "JP2011118178A",
    "US20110188054A1",
    "US20110187878A1",
    "US20110295331A1",
    "US20120038986A1",
    "WO2012020380A1",
    "WO2012066501A1",
    "US20120140109A1",
    "US20120140094A1",
    "US20130038941A1",
    "US20130038881A1"
  ]
}

Record 3,949 of 8,000 in Patents full text (MLC-0201). Request the full dataset.