MLchartDataset catalogue

Patent · US8996228B1 · B1 · US

Construction zone object detection using light detection and ranging

(11) Publication number
US8996228B1
(21) Application number
13/603,618
(22) Filing date
2012-09-05
(30) Priority date
2012-09-05
(43) Publication date
2015-03-31
(45) Date of grant
2015-03-31
(51) IPC
G01C 21/34; G01S 17/86; G01S 17/89; G01S 17/931; G05D 1/00
(52) CPC
  • B60W Conjoint control of vehicle sub-units of different type or different function; control systems specially adapted for hybrid vehicles; road vehicle drive control systems for purposes not related to the control of a particular sub-unit: 30/08, 2420/403, 2552/53, 2554/20, 30/09, 30/0956, 30/18163, 40/06, 60/00184
  • B25J Manipulators; chambers provided with manipulation devices: 19/023, 9/16, 9/1697
  • G01C Measuring distances, levels or bearings; surveying; navigation; gyroscopic instruments; photogrammetry or videogrammetry: 21/28, 21/3461
  • G01S Radio direction-finding; radio navigation; determining distance or velocity by use of radio waves; locating or presence-detecting by use of the reflection or reradiation of radio waves; analogous arrangements using other waves: 13/86, 13/865, 13/867, 17/86, 17/89, 17/931, 2013/93185, 2013/9319, 2013/9327, 2013/93271, 2013/93273, 2013/93275, 2013/93277, 7/4808
  • G05D Systems for controlling or regulating non-electric variables: 1/021, 1/0246, 1/0251, 1/027
  • G06V Image or video recognition or understanding: 20/10, 20/58, 20/582, 20/588
  • G08G Traffic control systems: 1/16, 1/165, 1/166, 1/167
  • Y10S Technical subjects covered by former uspc cross-reference art collections [xracs] and digests: 901/47
(73) Assignee
FERGUSON DAVID IAN; HAEHNEL DIRK; MAHON IAN; GOOGLE INC
(72) Inventors
FERGUSON DAVID IAN; HAEHNEL DIRK; MAHON IAN
(54) Title
Construction zone object detection using light detection and ranging
(57) Abstract

Methods and systems for construction zone object detection are described. A computing device may be configured to receive, from a LIDAR, a 3D point cloud of a road on which a vehicle is travelling. The 3D point cloud may comprise points corresponding to light reflected from objects on the road. Also, the computing device may be configured to determine sets of points in the 3D point cloud representing an area within a threshold distance from a surface of the road. Further, the computing device may be configured to identify construction zone objects in the sets of points. Further, the computing device may be configured to determine a likelihood of existence of a construction zone, based on the identification. Based on the likelihood, the computing device may be configured to modify a control strategy of the vehicle; and control the vehicle based on the modified control strategy.

Full text
View on Google Patents

Claims (19)

  1. A method, comprising: receiving, at a computing device configured to control a vehicle, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from one or more objects on the road; selecting, using the computing device, a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road; identifying one or more construction zone objects in the selected portion; determining, using the computing device, a number and locations of the one or more construction zone objects; determining, using the computing device, a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects; in response to the likelihood exceeding a threshold likelihood, determining a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects; modifying, using the computing device, a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and controlling, using the computing device, the vehicle based on the modified control strategy.
  2. The method of claim 1, wherein the vehicle is in an autonomous operation mode.
  3. The method of claim 1, wherein the one or more construction zone objects are construction cones or construction barrels.
  4. The method of claim 1, wherein determining the likelihood of the existence of the construction zone comprises determining that the one or more construction zone objects are within a predetermined distance of each other.
  5. The method of claim 1, wherein determining the likelihood of the existence of the construction zone comprises determining, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.
  6. The method of claim 1, wherein identifying the one or more construction zone objects in the selected portion comprises determining, for each identified construction zone object, a respective likelihood of the identification.
  7. The method of claim 6, wherein determining the respective likelihood of the identification comprises: identifying a shape in the selected portion; and matching the shape to one or more shapes of standard construction zone objects.
  8. The method of claim 6, wherein determining the likelihood of the existence of the construction zone comprises processing information relating to the respective likelihoods, the number, and locations of the one or more construction zone objects by a classifier trained by previously collected training data, wherein training the classifier comprises: receiving training data for a plurality of driving situations of the vehicle, wherein the training data includes respective LIDAR-based information relating to a respective 3D point cloud of a respective road; receiving positive or negative indication of respective existence of a respective construction zone corresponding to respective training data for each of the driving situations; correlating, for each driving situation, the positive or negative indication with the respective training data; and determining parameters of the classifier based on the correlations for the plurality of driving situations.
  9. The method of claim 1, wherein controlling the vehicle based on the modified control strategy comprises one or more of: (i) utilizing sensor information received from on-board or off-board sensors in making a navigation decision rather than preexisting map information, (ii) utilizing the sensor information to estimate lane boundaries rather than the preexisting map information, (iii) determining locations of construction zone markers rather than lane markers on the road to estimate and follow the lane boundaries, (iv) activating one or more sensors for detection of construction workers and making the navigation decision based on the detection, (v) following another vehicle, (vi) maintaining a predetermined safe distance with other vehicles, (vii) turning-on lights, (viii) reducing a speed of the vehicle, (ix) stopping the vehicle.
  10. A non-transitory computer readable medium having stored thereon instructions executable by a computing device of a vehicle to cause the computing device to perform functions comprising: receiving, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from objects on the road; selecting a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road; identifying one or more construction zone objects in the selected portion; determining, for each identified construction zone object, a respective likelihood of the identification; determining, based on the respective likelihoods, a number and locations of the one or more construction zone objects; determining a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects; in response to the likelihood exceeding a threshold likelihood, determining a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects; modifying a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and controlling the vehicle based on the modified control strategy.
  11. The non-transitory computer readable medium of claim 10, wherein the vehicle is in an autonomous operation mode.
  12. The non-transitory computer readable medium of claim 10, wherein the one or more construction zone objects are construction cones or construction barrels.
  13. The non-transitory computer readable medium of claim 10, wherein the function of determining the likelihood of the existence of the construction zone comprises determining that the one or more construction zone objects are within a predetermined distance of each other.
  14. The non-transitory computer readable medium of claim 10, wherein the function of determining the likelihood of the existence of the construction zone comprises determining, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.
  15. The non-transitory computer readable medium of claim 10, wherein the function of determining the respective likelihood of the identification comprises: identifying a shape in the selected portion; and matching the shape to one or more shapes of standard construction zone objects.
  16. A control system for a vehicle, comprising: a light detection and ranging (LIDAR) sensor configured to provide LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR sensor and reflected from objects on the road; and a computing device in communication with the LIDAR sensor and configured to: receive the LIDAR-based information; select a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road; identify one or more construction zone objects in the selected portion; determine, for each identified construction zone object, a respective likelihood of the identification; determine, based on the respective likelihoods, a number and locations of the one or more construction zone objects; determine a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects; in response to the likelihood exceeding a threshold likelihood, determine a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects; modify a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and control the vehicle based on the modified control strategy.
  17. The system of claim 16, wherein the computing device is further configured to control the vehicle in an autonomous operation mode.
  18. The system of claim 16, wherein the one or more construction zone objects are construction cones or construction barrels.
  19. The system of claim 16, wherein, to determine the likelihood of the existence of the construction zone, the computing device is configured to determine, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.

Description

Autonomous vehicles use various computing systems to aid in transporting passengers from one location to another. Some autonomous vehicles may require some initial input or continuous input from an operator, such as a pilot, driver, or passenger. Other systems, for example autopilot systems, may be used only when the system has been engaged, which permits the operator to switch from a manual mode (where the operator exercises a high degree of control over the movement of the vehicle) to an autonomous mode (where the vehicle essentially drives itself) to modes that lie somewhere in between.

The present application discloses embodiments that relate to detection of a construction zone object detection using light detection and ranging. In one aspect, the present application describes a method. The method may comprise receiving, at a computing device configured to control a vehicle, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling. The 3D point cloud may comprise points corresponding to light emitted from the LIDAR and reflected from one or more objects on the road. The method also may comprise determining, using the computing device, one or more sets of points in the 3D point cloud representing an area within a threshold distance from a surface of the road. The method further may comprise identifying one or more construction zone objects in the one or more sets of points.

Citations (37)

  • EP2072316A2
  • JP2010221909A
  • US2006184297A1
  • US2008125972A1
  • US2008189040A1
  • US2009088916A1
  • US2009149990A1
  • US2010099353A1
  • US2010100268A1
  • US2010104199A1
  • US2010164701A1
  • US2010256867A1
  • US2010274430A1
  • US2011149064A1
  • US2011150348A1
  • US2011216198A1
  • US2011280026A1
  • US2011282581A1
  • US2012022764A1
  • US2012046820A1
  • US2012083960A1
  • US2012098968A1
  • US2012150425A1
  • US2012176234A1
  • US2013066511A1
  • US2013101174A1
  • US2013158796A1
  • US2013197804A1
  • US5220497A
  • US6058339A
  • US6064926A
  • US6970779B2
  • US8060271B2
  • US8311274B2
  • US8311695B2
  • US8332134B2
  • WO2012047743A2
Record as JSON 
{
  "publication_number": "US8996228B1",
  "country": "US",
  "kind": "B1",
  "title": "Construction zone object detection using light detection and ranging",
  "abstract": "Methods and systems for construction zone object detection are described. A computing device may be configured to receive, from a LIDAR, a 3D point cloud of a road on which a vehicle is travelling. The 3D point cloud may comprise points corresponding to light reflected from objects on the road. Also, the computing device may be configured to determine sets of points in the 3D point cloud representing an area within a threshold distance from a surface of the road. Further, the computing device may be configured to identify construction zone objects in the sets of points. Further, the computing device may be configured to determine a likelihood of existence of a construction zone, based on the identification. Based on the likelihood, the computing device may be configured to modify a control strategy of the vehicle; and control the vehicle based on the modified control strategy.",
  "claims": [
    "1. A method, comprising: receiving, at a computing device configured to control a vehicle, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from one or more objects on the road; selecting, using the computing device, a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road; identifying one or more construction zone objects in the selected portion; determining, using the computing device, a number and locations of the one or more construction zone objects; determining, using the computing device, a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects; in response to the likelihood exceeding a threshold likelihood, determining a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects; modifying, using the computing device, a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and controlling, using the computing device, the vehicle based on the modified control strategy.",
    "2. The method of claim 1, wherein the vehicle is in an autonomous operation mode.",
    "3. The method of claim 1, wherein the one or more construction zone objects are construction cones or construction barrels.",
    "4. The method of claim 1, wherein determining the likelihood of the existence of the construction zone comprises determining that the one or more construction zone objects are within a predetermined distance of each other.",
    "5. The method of claim 1, wherein determining the likelihood of the existence of the construction zone comprises determining, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.",
    "6. The method of claim 1, wherein identifying the one or more construction zone objects in the selected portion comprises determining, for each identified construction zone object, a respective likelihood of the identification.",
    "7. The method of claim 6, wherein determining the respective likelihood of the identification comprises: identifying a shape in the selected portion; and matching the shape to one or more shapes of standard construction zone objects.",
    "8. The method of claim 6, wherein determining the likelihood of the existence of the construction zone comprises processing information relating to the respective likelihoods, the number, and locations of the one or more construction zone objects by a classifier trained by previously collected training data, wherein training the classifier comprises: receiving training data for a plurality of driving situations of the vehicle, wherein the training data includes respective LIDAR-based information relating to a respective 3D point cloud of a respective road; receiving positive or negative indication of respective existence of a respective construction zone corresponding to respective training data for each of the driving situations; correlating, for each driving situation, the positive or negative indication with the respective training data; and determining parameters of the classifier based on the correlations for the plurality of driving situations.",
    "9. The method of claim 1, wherein controlling the vehicle based on the modified control strategy comprises one or more of: (i) utilizing sensor information received from on-board or off-board sensors in making a navigation decision rather than preexisting map information, (ii) utilizing the sensor information to estimate lane boundaries rather than the preexisting map information, (iii) determining locations of construction zone markers rather than lane markers on the road to estimate and follow the lane boundaries, (iv) activating one or more sensors for detection of construction workers and making the navigation decision based on the detection, (v) following another vehicle, (vi) maintaining a predetermined safe distance with other vehicles, (vii) turning-on lights, (viii) reducing a speed of the vehicle, (ix) stopping the vehicle.",
    "10. A non-transitory computer readable medium having stored thereon instructions executable by a computing device of a vehicle to cause the computing device to perform functions comprising: receiving, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR and reflected from objects on the road; selecting a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road; identifying one or more construction zone objects in the selected portion; determining, for each identified construction zone object, a respective likelihood of the identification; determining, based on the respective likelihoods, a number and locations of the one or more construction zone objects; determining a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects; in response to the likelihood exceeding a threshold likelihood, determining a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects; modifying a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and controlling the vehicle based on the modified control strategy.",
    "11. The non-transitory computer readable medium of claim 10, wherein the vehicle is in an autonomous operation mode.",
    "12. The non-transitory computer readable medium of claim 10, wherein the one or more construction zone objects are construction cones or construction barrels.",
    "13. The non-transitory computer readable medium of claim 10, wherein the function of determining the likelihood of the existence of the construction zone comprises determining that the one or more construction zone objects are within a predetermined distance of each other.",
    "14. The non-transitory computer readable medium of claim 10, wherein the function of determining the likelihood of the existence of the construction zone comprises determining, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary.",
    "15. The non-transitory computer readable medium of claim 10, wherein the function of determining the respective likelihood of the identification comprises: identifying a shape in the selected portion; and matching the shape to one or more shapes of standard construction zone objects.",
    "16. A control system for a vehicle, comprising: a light detection and ranging (LIDAR) sensor configured to provide LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling, wherein the 3D point cloud comprises points corresponding to light emitted from the LIDAR sensor and reflected from objects on the road; and a computing device in communication with the LIDAR sensor and configured to: receive the LIDAR-based information; select a portion of the 3D point cloud representing an area within a predetermined threshold distance from a surface of the road; identify one or more construction zone objects in the selected portion; determine, for each identified construction zone object, a respective likelihood of the identification; determine, based on the respective likelihoods, a number and locations of the one or more construction zone objects; determine a likelihood of existence of a construction zone based on the number and the locations of the one or more construction zone objects; in response to the likelihood exceeding a threshold likelihood, determine a severity of road changes due to the existence of the construction zone based on the number and the locations of the one or more construction zone objects; modify a control strategy associated with a driving behavior of the vehicle based on the likelihood of the existence of the construction zone on the road and the severity of the road changes; and control the vehicle based on the modified control strategy.",
    "17. The system of claim 16, wherein the computing device is further configured to control the vehicle in an autonomous operation mode.",
    "18. The system of claim 16, wherein the one or more construction zone objects are construction cones or construction barrels.",
    "19. The system of claim 16, wherein, to determine the likelihood of the existence of the construction zone, the computing device is configured to determine, based on the number and locations of the one or more construction zone objects, a given likelihood that the one or more construction zone objects define a lane boundary."
  ],
  "description_excerpt": "Autonomous vehicles use various computing systems to aid in transporting passengers from one location to another. Some autonomous vehicles may require some initial input or continuous input from an operator, such as a pilot, driver, or passenger. Other systems, for example autopilot systems, may be used only when the system has been engaged, which permits the operator to switch from a manual mode (where the operator exercises a high degree of control over the movement of the vehicle) to an autonomous mode (where the vehicle essentially drives itself) to modes that lie somewhere in between.\n\nThe present application discloses embodiments that relate to detection of a construction zone object detection using light detection and ranging. In one aspect, the present application describes a method. The method may comprise receiving, at a computing device configured to control a vehicle, from a light detection and ranging (LIDAR) sensor coupled to the computing device, LIDAR-based information relating to a three-dimensional (3D) point cloud of a road on which the vehicle is travelling. The 3D point cloud may comprise points corresponding to light emitted from the LIDAR and reflected from one or more objects on the road. The method also may comprise determining, using the computing device, one or more sets of points in the 3D point cloud representing an area within a threshold distance from a surface of the road. The method further may comprise identifying one or more construction zone objects in the one or more sets of points.",
  "cpc": [
    "B60W 30/08",
    "B25J 19/023",
    "B25J 9/16",
    "B25J 9/1697",
    "B60W 2420/403",
    "B60W 2552/53",
    "B60W 2554/20",
    "B60W 30/09",
    "B60W 30/0956",
    "B60W 30/18163",
    "B60W 40/06",
    "B60W 60/00184",
    "G01C 21/28",
    "G01C 21/3461",
    "G01S 13/86",
    "G01S 13/865",
    "G01S 13/867",
    "G01S 17/86",
    "G01S 17/89",
    "G01S 17/931",
    "G01S 2013/93185",
    "G01S 2013/9319",
    "G01S 2013/9327",
    "G01S 2013/93271",
    "G01S 2013/93273",
    "G01S 2013/93275",
    "G01S 2013/93277",
    "G01S 7/4808",
    "G05D 1/021",
    "G05D 1/0246",
    "G05D 1/0251",
    "G05D 1/027",
    "G06V 20/10",
    "G06V 20/58",
    "G06V 20/582",
    "G06V 20/588",
    "G08G 1/16",
    "G08G 1/165",
    "G08G 1/166",
    "G08G 1/167",
    "Y10S 901/47"
  ],
  "ipc": [
    "G01C 21/34",
    "G01S 17/86",
    "G01S 17/89",
    "G01S 17/931",
    "G05D 1/00"
  ],
  "assignees": [
    "FERGUSON DAVID IAN",
    "HAEHNEL DIRK",
    "MAHON IAN",
    "GOOGLE INC"
  ],
  "inventors": [
    "FERGUSON DAVID IAN",
    "HAEHNEL DIRK",
    "MAHON IAN"
  ],
  "filing_date": "2012-09-05",
  "publication_date": "2015-03-31",
  "grant_date": "2015-03-31",
  "priority_date": "2012-09-05",
  "application_number": "US-201213603618-A",
  "family_id": "52707961",
  "citations": [
    "EP2072316A2",
    "JP2010221909A",
    "US2006184297A1",
    "US2008125972A1",
    "US2008189040A1",
    "US2009088916A1",
    "US2009149990A1",
    "US2010099353A1",
    "US2010100268A1",
    "US2010104199A1",
    "US2010164701A1",
    "US2010256867A1",
    "US2010274430A1",
    "US2011149064A1",
    "US2011150348A1",
    "US2011216198A1",
    "US2011280026A1",
    "US2011282581A1",
    "US2012022764A1",
    "US2012046820A1",
    "US2012083960A1",
    "US2012098968A1",
    "US2012150425A1",
    "US2012176234A1",
    "US2013066511A1",
    "US2013101174A1",
    "US2013158796A1",
    "US2013197804A1",
    "US5220497A",
    "US6058339A",
    "US6064926A",
    "US6970779B2",
    "US8060271B2",
    "US8311274B2",
    "US8311695B2",
    "US8332134B2",
    "WO2012047743A2"
  ]
}

Record 2,087 of 5,000 in Patents full text (MLC-0201). Request the full dataset.