Autonomous vehicle visual based communication

{
  "publication_number": "US10933803B2",
  "country": "US",
  "kind": "B2",
  "title": "Autonomous vehicle visual based communication",
  "abstract": "An autonomous vehicle including a projector and a sensor system configured to output data representative of an object in an environment exterior of the autonomous vehicle. The autonomous vehicle can further include a data store that stores height map data. A computing system of the autonomous vehicle can classify an object as a person and can detect an initial position of the person. Responsive to the detection, the computing system can calibrate a notification to be projected on a ground of the environment exterior of the autonomous vehicle. The notification is based on output of a transform from a position of the projector to the ground and the transform is applied based on the predefined height map of the ground. The notification can be projected adjacent the person. The notification may be configured to inform the person that the autonomous vehicle detects a position of the person.",
  "claims": [
    "1. An autonomous vehicle, comprising: a sensor system configured to output data representative of an object in an environment exterior of the autonomous vehicle; a projector; a data store, wherein the data store includes a predefined height map of an area traversed by one or more autonomous vehicles; and a computing system that is in communication with the sensor system and the projector, wherein the computing system comprises: a processor; and memory that stores computer-executable instructions that, when executed by the processor, cause the processor to perform acts comprising: classifying the object as a person based on the data outputted by the sensor system; responsive to the classification, detecting an initial position of the person; responsive to the detection, calibrating a notification to be projected on a ground of the environment exterior of the autonomous vehicle adjacent the initial position of the person, wherein the notification is calibrated based on an output of a transform from a position of the projector to the ground, wherein the transform is applied based on the predefined height map of the ground the notification is being projected on; and responsive to the calibration, causing the projector to project the notification on the ground of the environment exterior of the autonomous vehicle.",
    "2. The autonomous vehicle of claim 1, wherein the step of classifying the object as a person comprises classifying the object according to a predefined list of person classes, wherein the acts further comprise: responsive to classifying the person class, selecting the notification to be projected based on the classification of the person class.",
    "3. The autonomous vehicle of claim 2, wherein a first notification is selected for a first person class, wherein a second notification is selected for a second person class, wherein the first notification and the second notification are different.",
    "4. The autonomous vehicle of claim 1, wherein the notification is projected near a footprint of the object.",
    "5. The autonomous vehicle of claim 1, the acts further comprising: responsive to detecting movement of the object relative to the autonomous vehicle, moving the notification along the ground simultaneous with movement of the object along a movement path of the object, wherein moving the notification comprises calibrating the notification based on a second output of the transform from the position of the projector to the ground adjacent the moved object.",
    "6. The autonomous vehicle of claim 1, the acts further comprising: responsive to the autonomous vehicle moving relative to object, calibrating the notification based on a second output of the transform from a moved position of the projector to the ground.",
    "7. The autonomous vehicle of claim 1, further comprising: a vehicle propulsion system; a braking system; and a steering system, the acts further comprising: controlling at least one of the vehicle propulsion system, the braking system, or the steering system to cause the autonomous vehicle to yield to the object.",
    "8. The autonomous vehicle of claim 1, the acts further comprising: causing the projector to project the notification until at least one of: the object exceeds a threshold distance from the autonomous vehicle; the object completes a predefined maneuver; or a predefined threshold projection time is exceeded.",
    "9. The autonomous vehicle of claim 1, wherein the step of detecting an initial position of the person further comprises detecting a gaze of the person, wherein the step of causing the projector to project the notification comprises projecting the notification when the gaze of the person is detected as being aimed at the autonomous vehicle.",
    "10. The autonomous vehicle of claim 1, wherein the notification includes an indicator of the position of the autonomous vehicle relative to the projected notification.",
    "11. The autonomous vehicle of claim 1, wherein the person is one of: a pedestrian; a cyclist; or a service animal; or an operator of a differing vehicle.",
    "12. A method performed by a computing system, the computing system comprising a processor and memory, the method comprising: classifying an object in an environment exterior of an autonomous vehicle as a person; responsive to the classification, detecting an initial position of the person; responsive to detecting the initial position, calibrating a notification to be projected on a ground of the environment exterior of the autonomous vehicle adjacent the initial position of the person, wherein the notification is calibrated based on an output of a transform from a position of a projector of the autonomous vehicle to the ground, wherein the transform is applied based on a predefined height map of the ground the notification is being projected on; and responsive to the calibration, causing the projector to project the notification on the ground of the environment exterior of the autonomous vehicle, wherein the notification is configured to inform the person that the autonomous vehicle detects a position of the person.",
    "13. The method of claim 12, further comprising: responsive to detecting movement of the object relative to the autonomous vehicle, moving the notification along the ground simultaneous with movement of the object along a movement path of the object, wherein moving the notification comprises calibrating the notification based on a second output of the transform applied based on the position of the projector to ground adjacent the moved object.",
    "14. The method of claim 12, further comprising: responsive to the autonomous vehicle moving relative to object, calibrating the notification based on a second output of the affine transform applied based on a moved position of the projector to the ground.",
    "15. The method of claim 12, further comprising: causing the projector to project the notification until at least one of: the object exceeds a threshold distance from the autonomous vehicle; the object completes a predefined maneuver; or a predefined threshold projection time is exceeded.",
    "16. The method of claim 12, wherein the step of detecting an initial position of the person further comprises detecting a gaze of the person, wherein the step of causing the projector to project the notification comprises projecting the notification when the gaze of the person is detected as being aimed at the autonomous vehicle.",
    "17. The method of claim 12, wherein the step of classifying the object as a person comprises classifying the object according to a predefined list of person classes, wherein the acts further comprise: responsive to classifying the person class, selecting the notification to be projected based on the classification of the person class.",
    "18. The method of claim 17, wherein a first notification is selected for a first person class, wherein a second notification is selected for a second person class, wherein the first notification and the second notification are different.",
    "19. An autonomous vehicle comprising: a computer-readable storage medium that comprises instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising: receiving output from a sensor system in the autonomous vehicle, wherein the output comprises data representative of an object in an environment exterior of the autonomous vehicle; responsive to receiving the output, classifying the object as a person; responsive to the classification, detecting an initial position of the person; responsive to the detection, calibrating a notification to be projected on a ground of the environment exterior of the autonomous vehicle adjacent the initial position of the person, wherein the notification is calibrated based on an output of a transform from a position of the projector to the ground, wherein the transform is applied based on the predefined height map of the ground the notification is being projected on; and responsive to the calibration, causing a projector of the autonomous vehicle to project the notification on the ground of the environment exterior of the autonomous vehicle.",
    "20. The autonomous vehicle of claim 19, wherein the step of detecting an initial position of the person further comprises detecting a gaze of the person, wherein the step of causing the projector to project the notification comprises projecting the notification when the gaze of the person is detected as being aimed at the autonomous vehicle."
  ],
  "description_excerpt": "An autonomous vehicle is a motorized vehicle that can operate without a human driver. Since the autonomous vehicle lacks a human driver, others in an environment nearby the autonomous vehicle often may have difficulty understanding actions to be taken by the autonomous vehicle because of the lack of a human driver. In contrast, others in an environment nearby a human operated vehicle may be able to interact with the driver. According to an illustration, before a person (e.g., pedestrian) performs a certain maneuver near a human operated vehicle (e.g., crossing the street), the person can attempt to determine whether the driver is aware of the person. Conventionally, this could be done via trying to make eye contact with the driver of the vehicle and/or receiving a signal from the driver indicating approval for the person to conduct the maneuver by using hand maneuvers or other signals.\n\nThe following is a brief summary of subject matter that is described in greater detail herein. This summary is not intended to be limiting as to scope of the claims.\n\nDescribed herein are various technologies pertaining to causing an autonomous vehicle to project an informative notification on the ground exterior of the autonomous vehicle. With more specificity, the autonomous vehicle can detect a location of an object and can project the notification adjacent the location of the object. The notification can inform the object that the autonomous vehicle is aware of and detects the location of the object. The notification can further inform the object that the autonomous vehicle is tracking the object over a certain duration.",
  "cpc": [
    "G06F 3/013",
    "B60Q 1/50",
    "B60Q 1/507",
    "B60Q 2400/50",
    "G05D 1/0088",
    "G05D 2201/0213",
    "G06K 9/00369",
    "G06K 9/00791",
    "G06V 20/56",
    "G06V 40/103"
  ],
  "ipc": [
    "B60Q 1/50",
    "G05D 1/00",
    "G06F 3/01",
    "G06K 9/00"
  ],
  "assignees": [
    "GM Cruise Holdings LLC"
  ],
  "inventors": [
    "Christopher Shih"
  ],
  "filing_date": "2019-03-31",
  "publication_date": "2021-03-02",
  "grant_date": "2021-03-02",
  "priority_date": "2019-03-31",
  "application_number": "US-201916371001-A",
  "family_id": "72606847",
  "cited_by_count": 5,
  "citations": [
    "US8749401B2",
    "US8954252B1",
    "US20150268665A1",
    "US20150336502A1",
    "US9483948B1",
    "US9694736B2",
    "US20170262715A1",
    "US20190156130A1"
  ]
}