Safety-rated multi-cell workspace mapping and monitoring

{
  "publication_number": "US11613017B2",
  "country": "US",
  "kind": "B2",
  "title": "Safety-rated multi-cell workspace mapping and monitoring",
  "abstract": "Safety systems in distributed factory workcells intercommunicate or communicate with a central controller so that when a person, robot or vehicle passes from one workcell or space into another on the same factory floor, the new workcell or space need not repeat the tasks of analysis and classification and can instead immediately integrate the new entrant into the existing workcell or space-monitoring schema. The workcell or space can also communicate attributes such as occlusions, unsafe areas, movement speed, and object trajectories, enabling rapid reaction by the monitoring system of the new workcell or space.",
  "claims": [
    "1. A safety system for identifying safe regions in a three-dimensional workspace that includes controlled machinery and a plurality of workcells distributed about the workspace, the system comprising, with respect to each workcell, a controller configured to: generate a three-dimensional representation of at least a portion of the workcell based on images obtained by a plurality of sensors; map one or more safe zones within the workcell, the safe zones being outside a safety zone of the machinery; receive, from a second controller, a three-dimensional representation of at least a portion of an adjacent workcell; and transmit, to the second controller, at least a portion of the three-dimensional representation of the at least a portion of the workcell.",
    "2. The safety system of claim 1, wherein the controller is further configured to classify three-dimensional portions within the workcell as unoccupied, occupied or unknown, the one or more safe zones each being a three-dimensional portion classified as unoccupied.",
    "3. The safety system of claim 2, wherein the three-dimensional portions within the workcell partially overlap.",
    "4. The safety system of claim 1, wherein said at least a portion of the three-dimensional representation received from the second controller is classified as unoccupied, occupied or unknown.",
    "5. The safety system of claim 1, wherein the controller is further configured to transmit, to the second controller, predicted movements of objects in the workcell into the adjacent workcell.",
    "6. The safety system of claim 5, wherein the second controller is responsive to the predicted movements and is configured to alter operation of machinery in the adjacent workcell in response to a predicted intrusion into a safe zone of the adjacent workcell.",
    "7. The safety system of claim 6, wherein the second controller is configured to computationally extend the predicted intrusion into the adjacent workcell in accordance with a model of human movement.",
    "8. The safety system of claim 6, wherein the second controller is configured to computationally extend the predicted intrusion into the adjacent workcell in accordance with a trajectory of an autonomous guided vehicle.",
    "9. The safety system of claim 8, wherein the second controller is configured to receive the trajectory from the autonomous guided vehicle.",
    "10. The safety system of claim 1, wherein the safety zone is a 3D volume surrounding at least a portion of the machinery.",
    "11. A method of identifying safe regions in a three-dimensional workspace that includes controlled machinery and a plurality of workcells distributed about the workspace, the method comprising the steps of: obtaining, by a plurality of sensors, images of at least a portion of the workcell; generating a three-dimensional digital representation of the at least a portion of the workcell based on the images; computationally mapping one or more safe zones within the workcell, the safe zones being outside a safety zone of the machinery; generating a three-dimensional representation of at least a portion of an adjacent workcell; and combining the generated three-dimensional representations.",
    "12. The method of claim 11, further comprising the step of computationally classifying three-dimensional portions within the workcell as unoccupied, occupied or unknown, the one or more safe zones each being a three-dimensional portion classified as unoccupied.",
    "13. The method of claim 12, wherein the three-dimensional portions within the workcell partially overlap.",
    "14. The method of claim 11, further comprising the step of computationally classifying three-dimensional portions within the adjacent workcell as unoccupied, occupied or unknown, one or more safe zones within the adjacent workcell each being a three-dimensional portion classified as unoccupied.",
    "15. The method of claim 11, further comprising the step of computationally predicting movements of one or more objects in the workcell into the adjacent workcell.",
    "16. The method of claim 15, further comprising the step of altering operation of the machinery in the adjacent workcell in response to a predicted intrusion into a safe zone of the adjacent workcell.",
    "17. The method of claim 16, further comprising the step of computationally extending the predicted intrusion into the adjacent workcell in accordance with a model of human movement.",
    "18. The method of claim 16, further comprising the step of computationally extending the predicted intrusion into the adjacent workcell in accordance with a trajectory of an autonomous guided vehicle.",
    "19. The method of claim 11, wherein the safety zone is a 3D volume surrounding at least a portion of the machinery.",
    "20. A safety system for identifying safe regions in a three-dimensional workspace including controlled machinery and a plurality of workcells distributed about the workspace, the system comprising: a supervisory controller; and associated with each workcell, a local controller configured to: generate a three-dimensional representation of at least a portion of the workcell based on images obtained by a plurality of sensors; map one or more safe zones within the workcell, the safe zones being outside a safety zone of the machinery; and transmit, to the supervisory controller, at least a portion of the three-dimensional representation of the workcell, wherein the supervisory controller is configured to transmit, to each local controller, predicted movements of one or more objects into the workcell associated with the local controller based on data received from one or more other local controllers.",
    "21. The safety system of claim 20, wherein the local controller of each workcell is further configured to classify three-dimensional portions within the workcell as unoccupied, occupied or unknown, the one or more safe zones each being a three-dimensional portion classified as unoccupied.",
    "22. The safety system of claim 21, wherein the three-dimensional portions within the workcell partially overlap.",
    "23. The safety system of claim 20, wherein the safety zone is a 3D volume surrounding at least a portion of the machinery.",
    "24. The safety system of claim 20, wherein the local controller of each workcell is responsive to the predicted movements and is configured to alter operation of machinery in the workcell in response to a predicted intrusion into a safe zone thereof.",
    "25. The safety system of claim 20, wherein the local controller of each workcell is configured to computationally extend a predicted intrusion into the workcell in accordance with a model of human movement.",
    "26. The safety system of claim 20, wherein the local controller of each workcell is configured to computationally extend a predicted intrusion into the workcell in accordance with a trajectory of an autonomous guided vehicle.",
    "27. The safety system of claim 26, wherein the local controller of each workcell is configured to receive the trajectory from the autonomous guided vehicle.",
    "28. The safety system of claim 26, wherein the supervisory controller is configured to receive the trajectory from the autonomous guided vehicle."
  ],
  "description_excerpt": "The field of the invention relates, generally, to monitoring of industrial environments where humans and machinery interact or come into proximity, and in particular to systems and methods for detecting unsafe conditions in a monitored multi-cell workspace.\n\nModern manufacturing generally involves the sequential execution of a set of manufacturing processes (such as welding, painting, and assembly) in fixed workcells through which the work in progress is moved by a means of transport, e.g., conveyor belts, roller stages, vehicles (guided, autonomous, or controlled or driven by a human, such a forklift, cart or dolly), or humans walking or driving between workcells, carrying the work in progress. A simple and well-known arrangement for manufacturing is the assembly line, where the workcells are arranged in a line and connected through conveyor belts or a chain line that moves the work in progress (“workpieces”) through a fixed path. A less common alternative is one where the workpieces remain in a workcell and the manufacturing processes are performed in place, with parts and tools traveling to the workcell as needed for the sequential manufacturing steps. This arrangement is common in situations where the item being manufactured is large or too unwieldy to move between workcells.\n\nStill another arrangement is called cellular manufacturing, in which the workcells are arranged flexibly around the factory floor in order to optimize factors such as workpiece transit time, parts delivery, or the mix of work orders.",
  "cpc": [
    "B25J 9/1666",
    "B25J 9/1676",
    "B25J 9/1697",
    "G01S 17/04",
    "G01S 17/87",
    "G01S 17/88",
    "G01S 17/89",
    "G01S 7/4808",
    "G01S 7/497",
    "G01V 8/20",
    "G05B 2219/40202",
    "G06T 17/05",
    "G06T 17/10",
    "Y10S 901/47",
    "Y10S 901/49"
  ],
  "ipc": [
    "B25J 9/16",
    "G01S 17/04",
    "G01S 17/87",
    "G01S 17/89",
    "G01S 7/48",
    "G01V 8/20",
    "G06T 17/05",
    "G06T 17/10"
  ],
  "assignees": [
    "Veo Robotics Inc"
  ],
  "inventors": [
    "Scott Denenberg",
    "Patrick Sobalvarro",
    "Clara Vu",
    "Alberto Moel",
    "Richard A. Kelsey"
  ],
  "filing_date": "2021-07-14",
  "publication_date": "2023-03-28",
  "grant_date": "2023-03-28",
  "priority_date": "2017-02-07",
  "application_number": "US-202117375447-A",
  "family_id": "71122464",
  "cited_by_count": 5,
  "citations": [
    "US5574637A",
    "US20050224479A1",
    "US9452531B2",
    "US20160354927A1",
    "US20180151012A1",
    "US10812778B1",
    "US20170302905A1",
    "US20180222052A1",
    "US20200206928A1",
    "US10099372B2",
    "US10899007B2",
    "US11097422B2",
    "JP2018207349A",
    "US10445944B2"
  ]
}