Dual use of safety-capable vehicle scanner for collaborative vehicle assembly and driving surveillance

{
  "publication_number": "US11514684B2",
  "country": "US",
  "kind": "B2",
  "title": "Dual use of safety-capable vehicle scanner for collaborative vehicle assembly and driving surveillance",
  "abstract": "A vehicle assembly system includes a vehicle chassis of a vehicle; at least one object sensor mounted to the vehicle chassis, where the at least one object sensor generates sensor data based on at least one detected object; a vehicle controller mounted to the vehicle chassis and configured to receive the sensor data from the at least one object sensor, where, during assembly, the vehicle controller is configured with production control software that enables the vehicle controller to generate production object data from the sensor data, monitor for a safety event based on the production object data, and generate a safety event signal in response to detecting the safety event; and a safety controller configured to receive the safety event signal from the vehicle controller and alter a movement of a surveilled machine corresponding to the safety event.",
  "claims": [
    "1. A vehicle assembly system, comprising: a vehicle chassis of a vehicle to be assembled; at least one object sensor mounted to the vehicle chassis, wherein the at least one object sensor generates sensor data based on at least one detected object; a vehicle controller mounted to the vehicle chassis and configured to receive the sensor data from the at least one object sensor, wherein, during assembly of the vehicle, the vehicle controller is configured with production control software that enables the vehicle controller to generate production object data from the sensor data, identify a surveilled machine configured to assemble a portion of the vehicle during assembly of the vehicle based on the production object data, monitor a position of the surveilled machine based on the production object data, monitor for a safety event involving the surveilled machine based on the production object data corresponding to another detected object and the monitored position of the surveilled machine relative to the other detected object, and generate a safety event signal in response to detecting the safety event; and a safety controller configured to receive the safety event signal from the vehicle controller and alter a movement of the surveilled machine corresponding to the safety event, wherein neither the surveilled machine nor the other detected object is the vehicle chassis or a vehicle component mechanically fixed to the vehicle chassis.",
    "2. The vehicle assembly system of claim 1, wherein the surveilled machine is an articulated robot arm.",
    "3. The vehicle assembly system of claim 1, wherein the at least one object sensor includes at least one interior object sensor configured to scan an interior environment of the vehicle chassis and at least one exterior object sensor configured to scan an exterior environment of the vehicle chassis.",
    "4. The vehicle assembly system of claim 1, wherein the vehicle controller is configured to detect whether the production object data includes a human object and detect the safety event on a condition that the vehicle controller detects the human object within a predetermined threshold distance from the surveilled machine.",
    "5. The vehicle assembly system of claim 4, wherein: the vehicle controller is configured to determine a threat level of the safety event out of a plurality of threat levels based on the production object data, the safety event signal includes threat level information corresponding to the determined threat level, and the safety controller is configured to select one of a plurality of actions for altering the movement of the surveilled machine based on the threat level information.",
    "6. The vehicle assembly system of claim 5, wherein the plurality of threat levels include: a first threat level as a result of the vehicle controller detecting the human object being within the predetermined threshold distance from the surveilled machine while the surveilled machine is moving away from the human object, and a second threat level as a result of the vehicle controller detecting the human object being within the predetermined threshold distance from the surveilled machine while the surveilled machine is moving towards the human object.",
    "7. A vehicle assembly system, comprising: a vehicle chassis of a vehicle; at least one object sensor mounted to the vehicle chassis, wherein the at least one object sensor generates sensor data based on at least one detected object; a vehicle controller mounted to the vehicle chassis and configured to receive the sensor data from the at least one object sensor, wherein, during assembly, the vehicle controller is configured with production control software that enables the vehicle controller to generate production object data from the sensor data, monitor for a safety event based on the production object data, and generate a safety event signal in response to detecting the safety event; and a safety controller configured to receive the safety event signal from the vehicle controller and alter a movement of a surveilled machine corresponding to the safety event wherein the vehicle controller is configured to detect whether the production object data includes a human object and detect the safety event on a condition that the vehicle controller detects the human object within a predetermined threshold distance from the surveilled machine, wherein the vehicle controller is configured to determine a threat level of the safety event out of a plurality of threat levels based on the production object data, wherein the safety event signal includes threat level information corresponding to the determined threat level, wherein the safety controller is configured to select one of a plurality of actions for altering the movement of the surveilled machine based on the threat level information, wherein the plurality of threat levels include: a first threat level as a result of the vehicle controller detecting the human object being within the predetermined threshold distance from the surveilled machine while the surveilled machine is moving away from the human object, and a second threat level as a result of the vehicle controller detecting the human object being within the predetermined threshold distance from the surveilled machine while the surveilled machine is moving towards the human object, wherein in response to the threat level information indicating the first threat level, the safety controller is configured to slow down the movement of the surveilled machine, and wherein in response to the threat level information indicating the second threat level, the safety controller is configured to stop the movement of the surveilled machine.",
    "8. The vehicle assembly system of claim 1, wherein: the vehicle is assembled over a plurality of production stages, the production control software comprises a plurality of profiles for detecting the safety event, each of the plurality of profiles being mapped to at least one of the plurality of production stages, and the vehicle controller is configured to selectively activate and deactivate the plurality of profiles based on a current production stage to which an activated profile corresponds.",
    "9. The vehicle assembly system of claim 8, further comprising: a production controller configured to track a current production stage of the vehicle and transmit production stage information to the vehicle controller, the production stage information indicating the current production stage.",
    "10. The vehicle assembly system of claim 8, wherein: each of the plurality of profiles includes a different set of expected objects mounted to the vehicle chassis, and the vehicle controller is configured to differentiate between an unexpected object and an expected object based on the activated profile for detecting the safety event.",
    "11. The vehicle assembly system of claim 1, wherein the vehicle is assembled over a plurality of production stages, the vehicle assembly system further comprises: a software controller configured to update the production control software based on a current production stage of the plurality of production stages, wherein the updated production control software enables the vehicle controller to monitor for the safety event directed to the current production stage based on the production object data.",
    "12. The vehicle assembly system of claim 1, further comprising: a software controller configured to reflash the vehicle controller with driving control software thereby overwriting the production control software, wherein the driving control software enables the vehicle controller to generate driving object data from the sensor data and apply the driving object data to at least one driving functionality during a driving operation of the vehicle.",
    "13. The vehicle assembly system of claim 12, wherein the software controller is configured to reflash the vehicle controller with driving control software during a final phase of vehicle assembly.",
    "14. The vehicle assembly system of claim 12, wherein the at least one driving functionality includes at least one of vehicle lane assist, vehicle parking assist, adaptive cruise control, traffic sign detection, and passenger detection.",
    "15. A method of assembling a vehicle, comprising: mounting at least one object sensor to a vehicle chassis of the vehicle to be assembled; mounting a vehicle controller to the vehicle chassis; generating, by the at least one object sensor, sensor data based on at least one object detected by the at least one object sensor; installing production control software in a memory of the vehicle controller, wherein the production control software is configured for use during the assembly of the vehicle; generating, by the vehicle controller based on the production control software, production object data from the sensor data; identifying, by the vehicle controller based on the production control software, a surveilled machine configured to assemble a portion of the vehicle during assembly of the vehicle based on the production object data; monitoring, by the vehicle controller based on the production control software, a position of the surveilled machine based on the production object data; monitoring, by the vehicle controller based on the production control software, for a safety event involving the surveilled machine based on the production object data corresponding to another detected object and the monitored position of the surveilled machine relative to the other detected object; generating, by the vehicle controller based on the production control software, a safety event signal in response to detecting the safety event; and altering, by a safety controller, a movement of the surveilled machine corresponding to the safety event based on the safety event signal; wherein neither the surveilled machine nor the other detected object is the vehicle chassis or a vehicle component mechanically fixed to the vehicle chassis.",
    "16. The method of claim 15, wherein the surveilled machine is an articulated robot arm.",
    "17. The method of claim 15, wherein the at least one object sensor includes at least one interior object sensor configured to scan an interior environment of the vehicle chassis and at least one exterior object sensor configured to scan an exterior environment of the vehicle chassis.",
    "18. The method of claim 15, wherein monitoring for the safety event comprises: detecting the surveilled machine from the production object data; detecting whether the production object data includes a human object; and detecting the safety event on a condition that the vehicle controller detects the human object within a predetermined threshold distance from the surveilled machine.",
    "19. The method of claim 18, wherein: detecting the safety event comprises determining a threat level of the safety event from a plurality of threat levels based on the production object data, wherein the safety event signal includes threat level information corresponding to the determined threat level, and altering the movement of the surveilled machine corresponding to the safety event comprises selecting one of a plurality of actions for altering the movement of the surveilled machine based on the threat level information.",
    "20. The method of claim 19, wherein the plurality of threat levels include: a first threat level as a result of detecting the human object within the predetermined threshold distance from the surveilled machine while the surveilled machine is moving away from the human object, and a second threat level as a result of detecting the human object within the predetermined threshold distance from the surveilled machine while the surveilled machine is moving towards the human object.",
    "21. The method of claim 20, wherein altering the movement of the surveilled machine corresponding to the safety event comprises: in response to the threat level information indicating the first threat level, slowing down the movement of the surveilled machine, and in response to the threat level information indicating the second threat level, stopping the movement of the surveilled machine.",
    "22. The method of claim 15, further comprising: assembling the vehicle over a plurality of production stages, wherein the production control software comprises a plurality of profiles for detecting the safety event, each of the plurality of profiles being mapped to at least one of the plurality of production stages; and selectively activating and deactivating the plurality of profiles by the vehicle controller based on a current production stage to which an activated profile corresponds.",
    "23. The method of claim 22, further comprising: tracking, by a production controller, a current production stage of the vehicle; and transmitting, by the production controller, production stage information to the vehicle controller, the production stage information indicating the current production stage.",
    "24. The method of claim 15, further comprising: assembling the vehicle over a plurality of production stages; updating, by a software controller, the production control software based on a current production stage of the plurality of production stages, wherein the updated production control software enables the vehicle controller to monitor for the safety event directed to the current production stage based on the production object data.",
    "25. The method of claim 15, further comprising: reflash, by a software controller, the memory of the vehicle controller with driving control software thereby overwriting the production control software, wherein the driving control software enables the vehicle controller to generate driving object data from the sensor data and apply the driving object data to at least one driving functionality during a driving operation of the vehicle.",
    "26. The method of claim 25, wherein the at least one driving functionality includes at least one of vehicle lane assist, vehicle parking assist, adaptive cruise control, traffic sign detection, and passenger detection."
  ],
  "description_excerpt": "The present disclosure relates generally to a sensor system and to methods for monitoring an environment during manufacturing using the sensor system.\n\nVehicle production and the final driving application are considered fully separated worlds today. Vehicle production refers to vehicle manufacturing or repair. In contrast, final driving application refers to operating the vehicle outside of vehicle production (e.g., driving the vehicle).\n\nIn vehicle manufacturing, only the chassis construction is fully automated, whereas vehicle assembly still is a highly manual labor-intense task and it is only - to a certain extent - tool-assisted. In order to further increase the level of automation, there is high demand for more intense deployment of robotic systems as part of the vehicle assembly process.\n\nUnlike chassis building there is a much higher degree of flexibility in assembly for a vehicle setup. This requires a collaborative setup between human workers and smart machines. This need for flexibility and collaboration enforces an open workspace where humans and machines are no longer separated by protective fences.\n\nTherefore, so called safety Light Detection and Ranging (LIDAR) scanners have become one possible solution to protect humans who are working in the same area in which machines are operating. These safety LIDAR scanners are arranged outside of a vehicle, around a perimeter of a workstation (i.e., a production cell), and are inwardly facing towards a vehicle chassis. Thus, the safety LIDAR scanners cover some of the areas in which humans and machines are operating.",
  "cpc": [
    "G06V 20/58",
    "B25J 9/1676",
    "B60R 2300/804",
    "B60R 2300/806",
    "B62D 65/00",
    "B62D 65/005",
    "G05B 19/418",
    "G05B 19/41805",
    "G05B 2219/40202",
    "G05B 2219/40203",
    "G06V 20/582",
    "Y02P 90/02"
  ],
  "ipc": [
    "B25J 9/16",
    "B62D 65/00",
    "G05B 19/418",
    "G06V 20/58"
  ],
  "assignees": [
    "Infineon Technologies Austria AG"
  ],
  "inventors": [
    "Clemens MUELLER",
    "Harald Heinrich"
  ],
  "filing_date": "2020-02-07",
  "publication_date": "2022-11-29",
  "grant_date": "2022-11-29",
  "priority_date": "2020-02-07",
  "application_number": "US-202016784701-A",
  "family_id": "74550411",
  "cited_by_count": 0,
  "citations": [
    "EP2706384A1",
    "US20130158709A1",
    "US20140260795A1",
    "WO2018148181A1",
    "US20180276910A1",
    "US20190160675A1",
    "US20190286112A1",
    "US20190325180A1",
    "US20190329729A1",
    "US20190342152A1",
    "US20200066159A1",
    "US20200156722A1",
    "US20220088770A1",
    "US10872514B1"
  ]
}