Procedural animation engine and editor for sample-based design of stylized walking gaits for bipedal robots

{
  "publication_number": "US12293445B2",
  "country": "US",
  "kind": "B2",
  "title": "Procedural animation engine and editor for sample-based design of stylized walking gaits for bipedal robots",
  "abstract": "A robot design and control system including a procedural animation engine and a graphical animation editor that enable animators to author stylized walking gaits achievable by physical robotic characters, including bipedal robots of varying design. The animation engine generates dynamically feasible reference trajectories for omnidirectional walking given a desired walking velocity that may be input from a joystick or an artificial intelligence (AI) planner. This allows a legged robot to walk along an arbitrary path while expressing a custom animation “style,” e.g., a happy walk, a sneaky walk, or other manner of walking. The stylized walking motion or gait is generalized by the animation engine from a small number of animation samples that are defined at key walking velocities and tracked using a whole-body controller. The set of samples that are used as input to define a walking style is authored by an animator using the animation editor.",
  "claims": [
    "1. A system for providing stylized walking gaits for a legged robot, comprising: memory storing, for each animation style of a set of animation styles, a walking gait sample complying with kinematic and dynamic constraints associated with the legged robot at each of a plurality of velocities; a processor communicatively linked to the memory; and an animation engine provided by the processor running software, wherein the animation engine receives as user input a desired velocity for the legged robot, a selection of one of the animation styles, and a robot state for either the legged robot or a simulation of the legged robot, and wherein the animation engine generates actuator commands to animate the legged robot or the simulation of the legged robot with a stylized walking gait modifying the walking gait sample based on the desired velocity, the selected one of the animation styles, and the robot state.",
    "2. The system of claim 1, wherein each of the walking gait samples comprises a gait parameter vector paired with an associated gait velocity.",
    "3. The system of claim 1, wherein each of the walking gait samples is defined by a set of parameters including time intervals, attributes, and function curves.",
    "4. The system of claim 3, wherein the attributes include stance splay, and stance width for the legged robot.",
    "5. The system of claim 3, wherein the function curves include left and right foot translation and rotation curves defined for each swing interval.",
    "6. The system of claim 3, wherein the function curves are selected from the group of function curves comprising left sole translation, left sole rotation, right sole translation, right sole rotation, pelvis z translation, pelvis rotation angles, left arm joint angles, right arm joint angles, spine joint angles, and neck joint angles.",
    "7. The system of claim 3, further comprising an animation editor provided by the processor running software, wherein the animation editor generates a graphical user interface (GUI), and wherein the GUI includes a three dimensional (3D) visualizer displaying a 3D model of the simulation of the legged robot animated based on the actuator commands.",
    "8. The system of claim 7, wherein the GUI is adapted to receive user input modifying at least one of the parameters, the desired velocity, or the selected one of the animation styles and wherein the animation engine, in response, generates additional actuator commands and the animation editor displays a revised 3D model of the simulation of the legged robot animated based on the additional actuator commands.",
    "9. The system of claim 8, wherein the user input includes modifying one or more waypoints in one of the function curves displayed in the GUI.",
    "10. The system of claim 3, wherein the animation engine comprises a sample blender providing sample interpolation of the walking gait samples for use in generating the actuator commands and wherein the sample interpolation comprises blending the set of parameters including one or more of the time intervals, the attributes, and the function curves.",
    "11. The system of claim 10, wherein the sample interpolation comprises use of gait phase maps to blend time-varying function curves from multiple walk cycles in a manner that preserves contact event-based timing constraints in blended curves.",
    "12. A system for providing stylized walking gaits for a target bipedal robot, comprising: memory storing, for each animation style of a set of animation styles, a walking gait sample complying with kinematic and dynamic constraints associated with the target bipedal robot at each of a plurality of velocities; a processor communicatively linked to the memory; and an animation editor provided by the processor running software, wherein the animation editor generates a graphical user interface (GUI), wherein the GUI includes a three dimensional (3D) visualizer displaying a 3D model of the target bipedal robot simulated with animation based on a set of actuator commands, and wherein the set of actuator commands animate the 3D model with a stylized walking gait modifying the walking gait sample based on a velocity and one of the animation styles selected by a user via interaction with the GUI.",
    "13. The system of claim 12, wherein each of the walking gait samples is defined by a set of parameters including time intervals, attributes, and function curves.",
    "14. The system of claim 13, wherein the GUI is adapted to receive user input modifying at least one of the parameters, the desired velocity, or the selected one of the animation styles and wherein the animation engine, in response, generates a second set of actuator commands differing from the set of actuator commands and the animation editor displays a revised 3D model of the simulation of the legged robot animated based on the second set of actuator commands.",
    "15. The system of claim 14, wherein the user input includes modifying one or more waypoints in one of the function curves displayed in the GUI.",
    "16. The system of claim 14, further comprising an animation engine provided by the processor running software, wherein the animation engine receives as input the velocity, the one of the animation styles, and a robot state for the simulation of the target bipedal robot, and wherein the animation engine generates the set of actuator commands.",
    "17. The system of claim 16, wherein the animation engine comprises a sample blender providing sample interpolation of the walking gait samples for use in generating the actuator commands and wherein the sample interpolation comprises blending a time-varying subset of the parameters that are constrained based on contact events.",
    "18. The system of claim 14, wherein the attributes include stance splay and stance width for the legged robot.",
    "19. The system of claim 14, wherein the function curves include left and right foot translation and rotation curves defined for each swing interval.",
    "20. A method of designing a stylized walking gait for a bipedal robot, comprising: storing in memory storing, for each animation style of a set of animation styles, a walking gait sample complying with kinematic and dynamic constraints associated with the bipedal robot at each of a plurality of velocities; via a GUI, receiving user input indicating a desired velocity for the bipedal robot and a selection of one of the animation styles, and generating a set of actuator commands to animate the bipedal robot or a simulation of the bipedal robot with a stylized walking gait modifying the walking gait sample based on the desired velocity and the selected one of the animation styles; generating and displaying a 3D model of the simulation of the bipedal robot animated based on the set of actuator commands; via the GUI, receiving user input modifying at least one of the parameters, the desired velocity, or the selected one of the animation styles; and modifying the set of actuator commands based on the user input and then repeating the step of generating and displaying the 3D model of the simulation of the bipedal robot animated based on the modified set of actuator commands.",
    "21. The method of claim 20, wherein each of the walking gait samples is defined by a set of parameters including time intervals, attributes, and function curves, wherein the user input modifies a value of at least one of the parameters, wherein the attributes include nominal stance parameters that determine procedural foothold placement, and wherein the function curves include left and right foot translation and rotation curves defined for each swing interval.",
    "22. The system of claim 1, wherein each of the walking gait samples that are used as input to define the stylized walking gait is authored by an animator using the animation editor."
  ],
  "description_excerpt": "The present description relates, in general, to robots (or “robotic systems” or “robotic characters”) and control systems and methods for such robots. More particularly, the description relates to methods and systems of generating control signals for a legged robotic system such as a bipedal robot (and to controllers implementing such a method and robotic systems or robots with such controllers) that facilitates procedural animation and sample-based design of stylized walking gaits with physical constraints.\n\nThe control of humanoid or bipedal robots to provide stylized walking remains a challenging problem. Ideally, it is desirable to design creative authoring tools and control software that enable robotic characters that walk with the same fluidness and expressivity as their computer animated or human counterparts. The walking control problem is complicated by two main challenges.\n\nFirst, walking gaits must satisfy the kinematic and dynamic constraints inherent to whole-body locomotion. For example, the rate of change of the character's momentum must be consistent with the forces imparted on each foot when in contact with the ground. Those forces must also satisfy certain friction constraints to prevent the foot from slipping during the support phase. While a walking animation for a virtual character can violate these constraints and still appear physically plausible to viewers, animation content that is authored for a robot must be governed by the laws of physics to be realized on hardware to achieve a useful walking gait for a particular legged robot.",
  "cpc": [
    "G06T 13/40",
    "B25J 13/06",
    "B25J 5/00",
    "B62D 57/032",
    "G06F 3/04815",
    "G06F 3/04847",
    "G06F 30/20",
    "G06T 2200/24"
  ],
  "ipc": [
    "B25J 5/00",
    "G06F 3/04815",
    "G06F 3/04847",
    "G06F 30/20",
    "G06T 13/40"
  ],
  "assignees": [
    "Disney Enterprises Inc"
  ],
  "inventors": [
    "Michael Anthony Hopkins",
    "Kyle Michael Cesare",
    "Georg Wiedebach",
    "Jared Edward Bishop"
  ],
  "filing_date": "2022-03-22",
  "publication_date": "2025-05-06",
  "grant_date": "2025-05-06",
  "priority_date": "2022-03-22",
  "application_number": "US-202217700658-A",
  "family_id": "88096220",
  "cited_by_count": 0,
  "citations": [
    "US20050075755A1",
    "US20060108958A1",
    "US20110301756A1",
    "US20160243699A1",
    "US20190204848A1",
    "US20200369333A1",
    "US20230334744A1"
  ]
}