Stereoscopic visualization camera and integrated robotics platform

{
  "publication_number": "US12219228B2",
  "country": "US",
  "kind": "B2",
  "title": "Stereoscopic visualization camera and integrated robotics platform",
  "abstract": "A method of calibrating a robotic imaging apparatus includes providing the robotic imaging apparatus, calibrating a stereoscopic camera portion of the apparatus to robot space, and registering the robot space to a patient space. An embodiment of the apparatus includes a base section defining the robot space, a robotic arm including a first end connected to the base section, a second end including a coupling interface, and a plurality of joints and links connecting the first end to the second end, each joint including a motor configured to rotate the joint around an axis and a joint sensor configured to transmit a position of the respective joint, a stereoscopic camera connected at the coupling interface, and a sensor positioned at the coupling interface and configured to detect force imparted on the stereoscopic camera by an operator and to transmit output data indicative of the detected force.",
  "claims": [
    "1. A method of calibrating a robotic imaging apparatus comprising: providing the robotic imaging apparatus, which comprises: a base section configured for connection to a secure structure or a cart and having a base section point defining a robot space; a robotic arm including a first end connected to the base section, a second end including a coupling interface, and a plurality of joints and links connecting the first end to the second end, each joint including a motor configured to rotate the joint around an axis and a joint sensor configured to transmit a position of the respective joint; a stereoscopic camera connected to the robotic arm at the coupling interface, the stereoscopic camera configured to record left and right images of a target surgical site for producing a stream of stereoscopic images of the target surgical site; and a sensor positioned at the coupling interface and configured to detect translational and/or rotational force imparted on the stereoscopic camera by an operator and to transmit output data that is indicative of the detected force; calibrating the stereoscopic camera to the robot space; and registering the robot space to a patient space, wherein the patient space is defined with respect to a patient point that is within or on a patient.",
    "2. The method of claim 1, further comprising: determining boundaries of the robotic arm and/or the stereoscopic camera relative to the patient space and/or the robot space.",
    "3. The method of claim 1, further comprising: registering at least one of a working distance, an object plane and a view vector of the stereoscopic camera to the robot space.",
    "4. The method of claim 1, wherein a spatial orientation between the robot space and the patient space is fixed.",
    "5. The method of claim 1, wherein a spatial orientation between the robot space and the patient space is variable, and wherein a position and orientation of the patient with respect to a reference frame or one or more physical objects is determined.",
    "6. The method of claim 1, wherein the calibrating step includes calibrating a view vector of the stereoscopic camera to the robot space, wherein the view vector is aligned coincidentally with an optical axis of the stereoscopic camera.",
    "7. The method of claim 1, further comprising: receiving view vector data, including one or more of a working distance between the stereoscopic camera and the target surgical site, a magnification level of the stereoscopic camera, optical axis information, and an interpupillary distance of the stereoscopic camera.",
    "8. The method of claim 1, further comprising: determining a spatial relationship between the stereoscopic camera and the robotic arm in robot space.",
    "9. The method of claim 8, wherein the determining step includes viewing, with the stereoscopic camera, one or more uniquely recognizable objects disposed at one or more known locations and/or in a known orientation with respect to the robot space.",
    "10. The method of claim 1, wherein the robotic imaging apparatus further comprises: a memory storing at least one algorithm defined by one or more instructions and/or data structures that specify a rotation direction, speed, and duration for each of the joints of the robotic arm based at least on a current position of the robotic arm and the detected force; and at least one processor communicatively coupled to the sensor and the robotic arm, the at least one processor configured to: receive the output data from the sensor that is indicative of the detected force, determine, using the at least one algorithm in the memory, a movement sequence for the robotic arm based on a current position of the robotic arm and the output data from the sensor, and cause at least one of the joints of the robotic arm to rotate based on the determined movement sequence via one or more motor control signals provided to the at least one joint.",
    "11. A method of calibrating a robotic imaging apparatus comprising: providing the robotic imaging apparatus, which comprises: a robotic arm including a first end for connection to a secure structure having a secure structure point which defines a robot space coordinate system, a second end including a coupling interface, and a plurality of joints and links connecting the first end to the second end, each joint including a motor configured to rotate the joint around an axis and a joint sensor configured to transmit a position of the respective joint; an imaging device connected to the robotic arm at the coupling interface, the imaging device configured to record images of a target surgical site; and a sensor positioned at the coupling interface and configured to detect force and/or torque imparted on the imaging device by an operator and to transmit output data that is indicative of the detected force and/or torque; calibrating the stereoscopic camera to the robot space coordinate system; and registering the robot space coordinate system to a patient space coordinate system, wherein the patient space coordinate system is defined with respect to a patient point that is within or on a patient.",
    "12. The method of claim 11, further comprising: determining boundaries of the robotic arm and/or the stereoscopic camera relative to the patient space coordinate system and/or the robot space coordinate system.",
    "13. The method of claim 11, further comprising: registering at least one of a working distance, an object plane and a view vector of the stereoscopic camera to the robot space coordinate system.",
    "14. The method of claim 11, wherein a spatial orientation between the robot space coordinate system and the patient space coordinate system is fixed.",
    "15. The method of claim 11, wherein a spatial orientation between the robot space coordinate system and the patient space coordinate system is variable, and wherein a position and orientation of the patient with respect to a reference frame or one or more fiducials is determined.",
    "16. The method of claim 11, wherein the calibrating step includes calibrating a view vector of the stereoscopic camera to the robot space coordinate system, wherein the view vector is aligned coincidentally with an optical axis of the stereoscopic camera.",
    "17. The method of claim 11, further comprising: receiving view vector data, including one or more of a working distance between the stereoscopic camera and the target surgical site, a magnification level of the stereoscopic camera, optical axis information, and an interpupillary distance of the stereoscopic camera.",
    "18. The method of claim 11, further comprising: determining a spatial relationship between the stereoscopic camera and the robotic arm in the robot space coordinate system by viewing, with the stereoscopic camera, one or more uniquely recognizable objects disposed at one or more known locations and/or in a known orientation with respect to the robot space coordinate system.",
    "19. The method of claim 11, wherein the robotic imaging apparatus further comprises: at least one processor communicatively coupled to the sensor and the robotic arm, the at least one processor configured to: receive the output data from the sensor, convert the output data into translational and rotational vectors, determine, using kinematics, a movement sequence for the robotic arm based on a current position of the robotic arm and the translational and rotational vectors, the movement sequence specifying a rotation direction, a speed, and a duration of movement for at least some of the joints of the robotic arm, and cause at least one of the joints of the robotic arm to rotate based on the determined movement sequence via one or more motor control signals provided to the at least one joint.",
    "20. A method of calibrating a robotic imaging apparatus comprising: providing the robotic imaging apparatus, which comprises: a robotic arm including a first end for connection to a secure structure having a secure structure point which defines a robot space, a second end including a coupling interface, and a plurality of joints and links connecting the first end to the second end, each joint including a motor configured to rotate the joint around an axis and a joint sensor configured to transmit a position of the respective joint; a stereoscopic camera connected to the robotic arm at the coupling interface, the stereoscopic camera configured to record left and right images of a target surgical site for producing a stream of stereoscopic images of the target surgical site; and a sensor positioned at the coupling interface and configured to detect translational and/or rotational force imparted on the stereoscopic camera by an operator and to transmit output data that is indicative of the detected force; calibrating a view vector of the stereoscopic camera to the robot space, wherein the view vector is aligned coincidentally with an optical axis of the stereoscopic camera; registering the robot space to a patient space, wherein the patient space is defined with respect to a patient point that is within or on a patient; and determining boundaries of the robotic arm and/or the stereoscopic camera relative to the patient space and/or the robot space."
  ],
  "description_excerpt": "Surgery is art. Accomplished artists create works of art that far exceed the capabilities of a normal person. Artists use a brush to turn canisters of paint into vivid images that provoke strong and unique emotions from viewers. Artists take ordinary words written on paper and turn them into dramatic and awe-inspiring performances. Artists grasp instruments causing them to emit beautiful music. Similarly, surgeons take seemingly ordinary scalpels, tweezers, and probes and produce life-altering biological miracles.\n\nLike artists, surgeons have their own methods and preferences. Aspiring artists are taught the fundamentals of their craft. Beginners often follow prescribed methods. As they gain experience, confidence, and knowledge, they develop their own unique artistry reflective of themselves and their personal environment. Similarly, medical students are taught the fundamentals of surgical procedures. They are rigorously tested on these methods. As the students progress through residency and professional practice, they develop derivations of the fundamentals (still within medical standards) based on how they believe the surgery should best be completed. For instance, consider the same medical procedure performed by different renowned surgeons. The order of events, pacing, placement of staff, placement of tools, and use of imaging equipment varies between each of the surgeons based on their preferences. Even incision sizes and shapes can be unique to the surgeon.\n\nThe artistic-like uniqueness and accomplishment of surgeons make them weary of surgical tools that change or alter their methods.",
  "cpc": [
    "H04N 23/51",
    "A61B 2090/064",
    "A61B 2090/365",
    "A61B 2090/371",
    "A61B 34/32",
    "A61B 34/77",
    "A61B 90/25",
    "B25J 9/0009",
    "B25J 9/04",
    "B25J 9/1607",
    "H04N 13/106",
    "H04N 13/204",
    "H04N 13/296",
    "H04N 23/57",
    "H04N 23/695"
  ],
  "ipc": [
    "A61B 34/00",
    "A61B 34/32",
    "B25J 9/00",
    "B25J 9/04",
    "B25J 9/16",
    "H04N 13/204",
    "H04N 23/51",
    "H04N 23/695"
  ],
  "assignees": [
    "Alcon Inc"
  ],
  "inventors": [
    "Maximiliano Ramirez Luna",
    "Michael Weissman",
    "Thomas Paul Riederer",
    "George Charles Polchin",
    "Ashok Burton Tripathi",
    "Patrick Terry"
  ],
  "filing_date": "2022-04-12",
  "publication_date": "2025-02-04",
  "grant_date": "2025-02-04",
  "priority_date": "2017-04-24",
  "application_number": "US-202217718735-A",
  "family_id": "68236605",
  "cited_by_count": 0,
  "citations": [
    "US7286287B1",
    "US20060146402A1",
    "US20120120486A1",
    "JP2009265221A",
    "CN202631839U",
    "WO2014139023A1",
    "CN104339349A",
    "CN105555222A",
    "US20170007342A1",
    "US20180129018A1",
    "US20170143442A1",
    "CN206170133U",
    "US10917543B2",
    "US11336804B2"
  ]
}