Systems and methods relating to document and fastener identification

{
  "publication_number": "US10542167B2",
  "country": "US",
  "kind": "B2",
  "title": "Systems and methods relating to document and fastener identification",
  "abstract": "Method and systems of automated document processing described herein include activating in sequence a plurality of illumination modules of an illumination source to illuminate a document, where the plurality of illumination modules are located at different positions relative to the document. The document can be imaged each time the document is illuminated by an illumination module to provide a plurality of images. A shadow profile of the document can be obtained based on the plurality of images. One or more of a boundary of the document and presence of a fastener attached to the document can be identified using the shadow profile. Any fasteners present may be removed using a robot arm.",
  "claims": [
    "1. A method of document processing, comprising: activating, in sequence, a plurality of illumination modules of an illumination source to illuminate a document, wherein the plurality of illumination modules are located at different positions relative to the document; imaging the document each time the document is illuminated by an illumination module of the plurality of illumination modules, to provide a plurality of images; obtaining a shadow profile of the document based on the plurality of images; identifying, with aid of at least one processor, at least one or more members selected from the group consisting of a boundary of the document and presence of a fastener attached to the document, using the shadow profile, wherein identifying the presence of the fastener comprises recognizing when the fastener is an inverted fastener; and when the fastener is the inverted fastener, applying a suction force upon a portion of the document using a suction applicator to lift the document from a document pedestal; positioning a first document platform under the lifted document; positioning a second document platform over the lifted document to place the document between and in contact with the first document platform and the second document platform; and rotating the first document platform and the second document platform around a horizontal axis to flip the document over.",
    "2. The method of claim 1, wherein identifying the boundary of the document comprises identifying an edge having a thickness of less than 0.1 millimeters (mm).",
    "3. A method of document processing, comprising: activating, in sequence, a plurality of illumination modules of an illumination source to illuminate a document, wherein the plurality of illumination modules are located at different positions relative to the document; imaging the document each time the document is illuminated by an illumination module of the plurality of illumination modules, to provide a plurality of images; obtaining a shadow profile of the document based on the plurality of images; identifying, with aid of at least one processor, at least one of a boundary of the document and presence of a fastener attached to the document using the shadow profile; and removing an identified fastener from the document, wherein the removing comprises: coupling a document transfer end effector to a robot arm; contacting a top surface of the document with the document transfer end effector coupled to the robot arm to apply a suction force upon the document to lift the document from a document pedestal on which the document is positioned to separate the document from other documents on the document pedestal; rotating a document pincher of the document transfer end effector to contact a bottom surface of the document to secure the document against the document transfer end effector; positioning a document platform under the lifted document to maintain separation of the document from the other documents; releasing the suction force applied upon the document by the document transfer end effector to place the document on the document platform; and releasing the document transfer end effector from the robot arm and coupling a fastener removal end effector to the robot arm.",
    "4. The method of claim 1 or claim 3, wherein the document further comprises a staple attached thereto, and wherein identifying the at least one of the boundary of the document and presence of the fastener comprises identifying presence of the staple.",
    "5. The method of claim 3, further comprising: contacting the top surface of the document with a document contacting portion of the fastener removal end effector to hold the document against the document platform; inserting a fastener removal tab of the fastener removal end effector between the fastener and the top surface of the document; gripping the fastener with a fastener gripper of the fastener removal end effector to position at least a portion of the fastener between the fastener insert tab and the fastener gripper; and moving the fastener gripper and the fastener removal tab away from the document to remove the fastener.",
    "6. A method of document processing, comprising: activating, in sequence, a plurality of illumination modules of an illumination source to illuminate a document, wherein the plurality of illumination modules are located at different position relative to the document; imaging the document each time the document is illuminated by an illumination module of the plurality of illumination modules, to provide a plurality of images; obtaining a shadow profile of the document based on the plurality of images; identifying, with aid of at least one processor, at least one of a boundary of the document and presence of a fastener attached to the document using the shadow profile; and identifying a corresponding fastener removal end effector for a robot arm based on type information of the fastener attached to the document."
  ],
  "description_excerpt": "Information can often be stored on physical document files. Such physical document files may be packed away in archives, warehouses and/or vaults. Access of information stored on physical document files may be costly and time consuming. Physical documents may be disorganized. Often the documents may include one or more fasteners attached thereto. Not only can the physical documents be difficult to locate and transport, but once the physical documents have been identified, extracting the information from the physical files can also be time consuming as well.\n\nAccording to some aspects, a method of document processing can comprise activating, in sequence, a plurality of illumination modules of an illumination source to illuminate a document, wherein the plurality of illumination modules are located at different positions relative to the document; imaging the document each time the document is illuminated by an illumination module of the plurality of illumination modules, to provide a plurality of images; obtaining a shadow profile of the document based on the plurality of images; and identifying, with aid of at least one processor, at least one of a boundary of the document and presence of a fastener attached to the document using the shadow profile.\n\nIn some embodiments, identifying the boundary of the document comprises identifying an edge having a thickness of less than 0.1 mm. In some embodiments, identifying the presence of the fastener comprises recognizing when the fastener is an inverted fastener.",
  "cpc": [
    "H04N 1/00755",
    "B25C 11/00",
    "B42F 1/02",
    "B65H 15/00",
    "B65H 2301/332",
    "B65H 2301/42212",
    "G03B 15/03",
    "G03B 15/07",
    "G03G 15/607",
    "G03G 2215/00308",
    "G07D 7/189",
    "H04N 1/00588",
    "H04N 1/00607",
    "H04N 1/00649",
    "H04N 1/0066",
    "H04N 1/125"
  ],
  "ipc": [
    "G03G 15/00",
    "H04N 1/00",
    "H04N 1/12",
    "G03B 15/03",
    "G03B 15/07",
    "G07D 7/189"
  ],
  "assignees": [
    "Ripcord Inc"
  ],
  "inventors": [
    "Alex Fielding",
    "Kevin Christopher HALL",
    "Kirk H. KNIGHT",
    "Jens Jordan HURLEY",
    "Jonathan Floyd GRUBB",
    "Joseph David Dills HARTNAGLE"
  ],
  "filing_date": "2018-06-13",
  "publication_date": "2020-01-21",
  "grant_date": "2020-01-21",
  "priority_date": "2015-12-19",
  "application_number": "US-201816007069-A",
  "family_id": "59057707",
  "cited_by_count": 5,
  "citations": [
    "US5087027A",
    "EP0731596A1",
    "US20040022563A1",
    "US6074334A",
    "US20030168308A1",
    "US20050087422A1",
    "US20110290851A1",
    "US20070091371A1",
    "US20100067071A1",
    "US20140341438A1",
    "CN201741888U",
    "CN103685840A",
    "WO2017106856A1"
  ]
}