Enhanced roadway mark locator, inspection apparatus, and marker

{
  "publication_number": "US9784843B2",
  "country": "US",
  "kind": "B2",
  "title": "Enhanced roadway mark locator, inspection apparatus, and marker",
  "abstract": "An apparatus for locating, inspecting, or placing marks on a roadway. The apparatus includes a GPS-based machine vision locator for sampling discrete geographical location data of a pre-existing roadway mark evident on the roadway. A computer may determine a continuous smooth geographical location function fitted to the sampled geographical location data. A marker is responsive to the GPS-based locator and geographical location function for replicating automatically the pre-existing roadway mark onto the roadway. The apparatus is typically part of a moving vehicle. A related method is disclosed for locating, inspecting, and placing marks on a resurfaced roadway. A similar apparatus can be used to guide a vehicle having a snow plow along a snow-covered roadway, or a paving machine along an unpaved roadway surface. The apparatus provides for accurate determination of roadway mark locations, roadway mark inspection apparatus and roadway markers using Bayesian model-based filtered enhanced GPS geographical location data.",
  "claims": [
    "1. An apparatus for determining a geographical location of a roadway mark from a moving vehicle, the apparatus comprising: a GPS antenna mounted to a vehicle; a GPS receiver responsive to the GPS antenna and configured to determine a raw geographical location of the GPS antenna; a first imager mounted to the vehicle, wherein the first imager is (a) synchronously responsive to the GPS receiver and (b) configured to image a first roadway mark to provide a first roadway mark image; a plurality of sensors configured to collect data about the conditions of the vehicle; and a computer (a) responsive to the GPS receiver, the first imager, and the plurality of sensors and (b) configured to determine a GPS location of the first roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors.",
    "2. The apparatus according to claim 1 wherein: the GPS antenna is configured to receive GPS radio wave signals originating from a GPS satellite system or a GPS-pseudolite array, and the GPS receiver is configured to receive the GPS radio wave signals from the GPS antenna and decode the GPS signals to determine the raw geographical location of the GPS antenna.",
    "3. The apparatus according to claim 2, wherein the GPS radio wave signals comprise a real time kinematic correction signal.",
    "4. The apparatus of claim 1, wherein the computer is further configured to (i) calculate roadway mark characteristics from the first roadway mark image and (ii) compares the calculated roadway mark characteristics against a standard set of roadway mark characteristics.",
    "5. The apparatus according to claim 4, wherein the roadway mark characteristics are at least one of a width of the roadway mark, a length of the roadway mark, a grayscale reflectivity, retroreflectivity, and a fill percentage.",
    "6. The apparatus according to claim 1, wherein the first imager is mounted to the vehicle using an adjustable mount including a fixably adjustable axis of rotation parallel to a roadway surface including the first roadway mark for aligning the first imager so that the first imager can focus on the roadway mark.",
    "7. The apparatus according to claim 1, wherein the first roadway mark is located to the left of a direction of travel of the vehicle, the apparatus further comprising a second imager mounted on the vehicle and aligned to image a second roadway mark to the right of the direction of travel of the vehicle, wherein the second imager is (a) synchronously responsive to the GPS receiver and (b) configured to image a second roadway mark to provide a second roadway mark image; and, and the computer is (a) further responsive to the second imager and (b) configured to determine a GPS location of the second roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors.",
    "8. The apparatus according to claim 1, wherein the Bayesian model-based filter is a Kalman filter.",
    "9. The apparatus according to claim 1, wherein the plurality of sensors include one or more sensors selected from the group consisting of an accelerometer, a gyroscope, a magnetometer, an air pressure sensor, a temperature sensor, steering wheel angle sensor and a speed detector.",
    "10. The apparatus according to claim 1, wherein the Bayesian model-based filter further accounts for a kinematic model of the vehicle.",
    "11. The apparatus according to claim 1, wherein the Bayesian model-based filter converts the raw geographical location of the GPS antenna to a filter-enhanced geographical location of the GPS antenna by adjusting the raw geographical location of the GPS antenna based on the data collected from the plurality of sensors and a kinematic model of the vehicle.",
    "12. The apparatus according to claim 1, wherein the first imager is a calibrated imager.",
    "13. The apparatus according to claim 12 wherein the first imager is a GPS calibrated imager.",
    "14. The apparatus according to claim 1, wherein the first imager is a charge coupled device camera.",
    "15. A method for determining a geographical location of a roadway mark from a moving vehicle, comprising: determining with a GPS receiver a raw geographical location of a GPS antenna mounted to a vehicle; collecting data about the conditions of the vehicle using a plurality of sensors; imaging a first roadway mark using a first imager mounted to the vehicle to produce a first roadway mark image, wherein the first imager is synchronously responsive to the GPS receiver; and using a computer which is responsive to the GPS receiver, the first imager, and the plurality of sensors to determine a GPS location of the first roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors.",
    "16. The method according to claim 15, wherein the first imager is a GPS calibrated imager.",
    "17. The method according to claim 15, further comprising using the computer to: calculate roadway mark characteristics from the first roadway mark image; and compare the calculated roadway mark characteristics against a standard set of roadway mark characteristics.",
    "18. The method according to claim 16, wherein the roadway mark characteristics include at least one of a width of the first roadway mark, a length of the first roadway mark, a grayscale reflectivity, retroreflectivity, and a fill percentage.",
    "19. The method according to claim 15, further comprising the GPS antenna receiving GPS radio-wave signals originating from a GPS satellite system or a GPS-pseudolite array, wherein determining the raw geographical location of the GPS antenna comprises the GPS receiver receiving the GPS radio-wave signals from the GPS antenna and decoding the GPS radio-wave signals.",
    "20. The apparatus according to claim 19, wherein the GPS radio-wave signals comprise a real time kinematic correction signal.",
    "21. The method according to claim 15, wherein the first imager is mounted to the vehicle using an adjustable mount including a fixably adjustable axis of rotation parallel to the roadway surface for aligning the first imager so that the first imager can focus on the roadway mark.",
    "22. The method according to claim 15, wherein the first roadway mark is located to the left of a direction of travel of the vehicle, further comprising: imaging a second roadway mark located to the right of the direction of travel of the vehicle using a second imager mounted to the vehicle to produce a second roadway mark image, wherein the second imager is synchronously responsive to the GPS receiver; and using a computer to determine a GPS location of the second roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors.",
    "23. The method according to claim 15, wherein the Bayesian model-based filter is a Kalman filter.",
    "24. The method according to claim 15 wherein the plurality of sensors include one or more sensors selected from the group consisting of an accelerometer, a gyroscope, a magnetometer, an air pressure sensor, a temperature sensor, steering wheel angle sensor and a speed detector.",
    "25. The method according to claim 15, wherein the Bayesian model-based filter further accounts for a kinematic model of the vehicle.",
    "26. The method according to claim 15, further comprising the Bayesian model-based filter converting the raw geographical location of the GPS antenna to a filter-enhanced geographical location of the GPS antenna by adjusting the raw geographical location of the GPS antenna based on the data collected from the plurality of sensors.",
    "27. The method according to claim 15, wherein the first imager is a calibrated imager.",
    "28. The method according to claim 15, wherein the first imager is a charge coupled device camera.",
    "29. An apparatus for locating and placing marks on a roadway, the apparatus comprising: a GPS antenna mounted to a vehicle; a GPS receiver responsive to the GPS antenna and configured to determine a raw geographical location of the GPS antenna; a first imager mounted to the vehicle, wherein the first imager is (a) synchronously responsive to the GPS receiver and (b) configured to image a roadway mark to provide a roadway mark image; a plurality of sensors configured to collect data about the conditions of the vehicle; a computer (a) responsive to the GPS receiver, the first imager, and the plurality of sensors, and (b) configured to: (1) determine a GPS location of the first roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors, and (2) determine a continuous smooth geographical location function best-fitted to the GPS location of the roadway mark; and a marker responsive to the computer for replicating automatically the roadway mark onto the roadway based on the continuous smooth geographical location function.",
    "30. The apparatus according to claim 29, wherein: the GPS antenna is configured to receive GPS radio wave signals originating from a GPS satellite system or a GPS-pseudolite array, and the GPS receiver is configured to receive the GPS radio wave signals from the GPS antenna and decode the GPS signals to determine the raw geographical location of the GPS antenna.",
    "31. The apparatus according to claim 30, wherein the GPS receiver is further configured to receive real time kinematic (RTK) signals to determine the raw geographical location of the GPS antenna.",
    "32. The apparatus according to claim 29 further comprising a second imager which images the roadway in front of the vehicle.",
    "33. The apparatus according to claim 29 further comprising a speed detector and a nozzle array and control system, the speed detector informing the nozzle array and control system and enabling the nozzle array and control system to adjust the rate at which it places marking material onto the roadway.",
    "34. The apparatus according to claim 29, wherein the computer generates an additional roadway mark function by offsetting the continuous function by an amount consistent with the desired position of a second roadway mark relative to the replicated roadway mark.",
    "35. An apparatus for guiding a vehicle having a snow plow along a roadway, the apparatus comprising: a GPS antenna mounted to a vehicle; a GPS receiver responsive to the GPS antenna and configured to determine a raw geographical location of the GPS antenna; a first imager mounted to the vehicle, wherein the first imager is (a) synchronously responsive to the GPS receiver and (b) configured to image a first roadway mark to provide a first roadway mark image; a plurality of sensors configured to collect data about the conditions of the vehicle; a computer (a) responsive to the GPS receiver, the first imager, and the plurality of sensors and (b) configured to: (1) determine a GPS location of the roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors, and (2) determine a continuous smooth geographical location function best-fitted to the GPS location of the roadway mark; and a system responsive to the computer for positioning the snow plow based on the continuous smooth geographical location function.",
    "36. An apparatus for locating and placing marks on a roadway, the apparatus comprising: a GPS antenna mounted to a vehicle; a GPS receiver responsive to the GPS antenna and configured to determine a raw geographical location of the GPS antenna; an imager mounted to the vehicle, wherein the imager is (a) synchronously responsive to the GPS receiver and (b) configured to image a roadway mark to provide a roadway mark image; a plurality of sensors configured to collect data about the conditions of the vehicle; and a computer (a) responsive to the GPS receiver, the imager, and the plurality of sensors, and (b) configured to: (1) determine a GPS location of the roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors, and (2) determine a continuous smooth geographical location function best-fitted to the GPS location of the roadway mark; and a nozzle array and control system responsive to the computer for replicating automatically the roadway mark at a location defined by the continuous smooth geographical location function by placing a marking material on the roadway.",
    "37. An apparatus for determining a continuous smooth geographical location function of a roadway mark comprising: a GPS antenna mounted to a vehicle; a GPS receiver responsive to the GPS antenna and configured to determine a raw geographical location of the GPS antenna; an imager mounted to the vehicle, wherein the imager is (a) synchronously responsive to the GPS receiver and (b) configured to image a roadway mark to provide a roadway mark image; a plurality of sensors configured to collect data about the conditions of the vehicle; and a computer (a) responsive to the GPS receiver, the imager, and the plurality of sensors, and (b) configured to: (1) determine a GPS location of the roadway mark using a Bayesian model-based filter which accounts for the raw geographical location of the GPS antenna and the data collected by the plurality of sensors, and (b) calculate a smooth continuous geographical location function from the GPS location of the roadway mark.",
    "38. The apparatus according to claim 37, wherein the smooth continuous geographical location function is a best-fitted continuous geographical location function."
  ],
  "description_excerpt": "The present invention relates generally to GPS-based machine vision locating and inspecting systems and to devices for making a visual indicia in or on top of pavement. More particularly, the present invention relates to a vehicle-mounted Bayesian model-based filter-enhanced GPS-based locating and inspecting systems for determining the geographical location and condition of roadway marks, and Bayesian model-based filter-enhanced GPS-based systems used for painting or otherwise “marking” roadway traffic lane demarcation lines.\n\nNew or repaved roadway surfaces almost always require the application of roadway surface markings as a mechanism for visually providing motorists with lane demarcation lines for controlling and directing traffic. In the past, the process of applying new roadway surface markings consisted of first manually determining the center of the roadway surface and painting small dots to visually define the roadway center. A driver of a paint vehicle would then use the roadway center to guide a paint sprayer which would deposit paint along the path defined by the small dots.\n\nCurrently, this task is accomplished by determining the center of the roadway at a first location point by manually measuring the width of the roadway and placing a mark at the center point. This process is then repeated to determine the center point of the roadway at a second location point which is displaced from the first point. These two points now define the starting and ending points for a line segment which identifies the center path of the roadway.",
  "cpc": [
    "E01H 5/06",
    "B60R 1/00",
    "B60R 2300/105",
    "B60R 2300/302",
    "B60R 2300/804",
    "B62D 5/0463",
    "E01C 23/01",
    "E01C 23/163",
    "F16M 11/10",
    "F16M 11/2071",
    "F16M 13/02",
    "F16M 13/022",
    "F16M 2200/022",
    "G01C 15/02",
    "G01S 19/14",
    "G06F 18/24155",
    "G06K 9/00798",
    "G06K 9/6278",
    "G06V 10/764",
    "G06V 20/588",
    "G07C 5/08"
  ],
  "ipc": [
    "B60R 1/00",
    "E01C 23/16",
    "E01H 5/06",
    "G01S 19/14",
    "G06V 10/764",
    "G07C 5/08",
    "B60R 22/00",
    "F16M 13/02",
    "G01C 15/02"
  ],
  "assignees": [
    "Limn Tech LLC"
  ],
  "inventors": [
    "Douglas D. Dolinar",
    "William R. Haller",
    "Matthew W. Smith",
    "Charles C. Stahl"
  ],
  "filing_date": "2016-03-28",
  "publication_date": "2017-10-10",
  "grant_date": "2017-10-10",
  "priority_date": "2012-01-17",
  "application_number": "US-201615082365-A",
  "family_id": "56407703",
  "cited_by_count": 27,
  "citations": [
    "US4219092A",
    "US4460127A",
    "US4832331A",
    "US5375059A",
    "US5296256A",
    "US5220876A",
    "US20020040962A1",
    "US5540516A",
    "US5529433A",
    "US5838277A",
    "US5848373A",
    "US5836398A",
    "DE4437297A1",
    "US5857066A",
    "US5549412A",
    "US5599133A",
    "US7832762B2",
    "US5746539A",
    "US5653389A",
    "US20020044087A1",
    "US20110285849A1",
    "US5771169A",
    "US20030139879A1",
    "US20030058346A1",
    "US5990469A",
    "US5995902A",
    "US6115481A",
    "US6299934B1",
    "US6330503B1",
    "US6074693A",
    "US6723375B2",
    "US7029199B2",
    "US7552008B2",
    "US7981462B2",
    "US20040124260A1",
    "US7370818B2",
    "US20080203187A1",
    "US20070128899A1",
    "US6729706B1",
    "US20110039021A1",
    "US6951375B2",
    "EP1486799A2",
    "US7698032B2",
    "JP2005242460A",
    "US7866917B2",
    "US20120072080A1",
    "US20060215882A1",
    "US20090118994A1",
    "US7970529B2",
    "DE102005044977A1",
    "US8190362B2",
    "US7899611B2",
    "US20100185382A1",
    "US20070208497A1",
    "JP2007316767A",
    "US20080046174A1",
    "US20100091017A1",
    "JP2010506291A",
    "US7640105B2",
    "US20100286899A1",
    "US20100259609A1",
    "JP2009258651A",
    "JP2009223220A",
    "JP2009223817A",
    "JP2009259215A",
    "JP2009237901A",
    "US20100104199A1",
    "US20100025709A1",
    "US20100257029A1",
    "US20100262470A1",
    "US20100263591A1",
    "US20100256912A1",
    "JP2010175756A",
    "US20100295668A1",
    "US20120054028A1",
    "US20120098657A1",
    "US9098751B2",
    "US8935057B2",
    "US20150127223A1",
    "US9298991B2",
    "CN203047061U"
  ]
}