Moving body control apparatus, moving body control method, and training method

{
  "publication_number": "US11449048B2",
  "country": "US",
  "kind": "B2",
  "title": "Moving body control apparatus, moving body control method, and training method",
  "abstract": "A moving body control apparatus for controlling a moving body includes: an acquisition device that acquires a control command for the moving body and an image of a view in the traveling direction of the moving body; and an information processing device that uses a machine learning model to output a control parameter for controlling the moving body, using the control command and the image acquired by the acquisition device as inputs.",
  "claims": [
    "1. A moving body control apparatus for controlling a moving body, the moving body control apparatus comprising: an acquisition device for acquiring one control command among N control commands for the moving body and an image of a view in a traveling direction of the moving body, where N is an integer greater than or equal to 2; and an information processing device that includes a feature amount extraction machine learning model and N control command machine learning models respectively corresponding to the N control commands, wherein the information processing device uses the feature amount extraction machine learning model to calculate, from the image acquired by the acquisition device, a feature amount associated with the one control command acquired by the acquisition device and indicating a feature of an object, selects one control command machine learning model corresponding to the one control command from among the N control command machine learning models, and uses the one control command machine learning model to output a control parameter for controlling the moving body, using the one control command acquired by the acquisition device and the feature amount as inputs, each of the N control commands is a signal indicating an intention to maneuver the moving body, the N control commands include a first control command which is one of turn left, turn right, stop, go forward, make a U-turn, and change lanes, and the N control command machine learning models include a first control command machine learning model which is one of a machine learning model for turning left, a machine learning model for turning right, a machine learning model for stopping, a machine learning model for going forward, a machine learning model for making a U-turn, and a machine learning model for changing lanes that corresponds to the first control command.",
    "2. The moving body control apparatus according to claim 1, wherein each of the N control command machine learning models is a neural network model.",
    "3. The moving body control apparatus according to claim 1, wherein the control parameter includes at least one of a speed and a steering angle.",
    "4. A training method for the moving body control apparatus according to claim 1, the training method comprising: a first step of training the feature amount extraction machine learning model and the N control command machine learning models using first training data that takes the N control commands for the moving body and a plurality of images of a view in a traveling direction of the moving body as inputs, and includes a plurality of control parameters for controlling the moving body as correct answers, the plurality of images respectively corresponding to the N control commands, and the plurality of control parameters respectively corresponding to the N control commands.",
    "5. The training method according to claim 4, further comprising: a second step of training the feature amount extraction machine learning model using second training data that takes the plurality of images as inputs and includes the N control commands as correct answers, wherein the second step is performed before the first step.",
    "6. The training method according to claim 5, further comprising: a third step of training the feature amount extraction machine learning model and the N control command machine learning models using a constraint condition that restricts a possible operating state of the moving body, wherein the third step is performed after the second step.",
    "7. The training method according to claim 5, further comprising: a fourth step of training the feature amount extraction machine learning model and the N control command machine learning models using a traffic rule to be obeyed by the moving body, wherein the fourth step is performed after the second step.",
    "8. A moving body control method for controlling a moving body, the moving body control method comprising: a first step of acquiring one control command among N control commands for the moving body and an image of a view in a traveling direction of the moving body, where N is an integer greater than or equal to 2; a second step of using a feature amount extraction machine learning model to calculate, from the image acquired in the first step, a feature amount associated with the one control command acquired in the first step and indicating a feature of an object; and a third step of selecting one control command machine learning model corresponding to the one control command from among N control command machine learning models respectively corresponding to the N control commands, and using the one control command machine learning model to output a control parameter for controlling the moving body, using the one control command acquired in the first step and the feature amount as inputs, wherein each of the N control commands is a signal indicating an intention to maneuver the moving body, the N control commands include a first control command which is one of turn left, turn right, stop, go forward, make a U-turn, and change lanes, and the N control command machine learning models include a first control command machine learning model which is one of a machine learning model for turning left, a machine learning model for turning right, a machine learning model for stopping, a machine learning model for going forward, a machine learning model for making a U-turn, and a machine learning model for changing lanes that corresponds to the first control command."
  ],
  "description_excerpt": "One or more exemplary embodiments disclosed herein relate generally to a moving body control apparatus for controlling a moving body\n\nConventionally, a technique for controlling a moving body, representative examples of which include a vehicle, a robot, etc., is known.\n\nFor example, Patent Literature (PTL) 1 discloses a technique for controlling autonomous driving of a vehicle based on a pre-prepared three-dimensional map. Moreover, for example, PTL 2 discloses a technique for controlling autonomous driving of a vehicle in accordance with a set driving route.\n\n[PTL 1] Japanese Unexamined Patent Application Publication No. 2009-199572\n\n[PTL 2] Japanese Unexamined Patent Application Publication No. H11-282530\n\nWith the technique disclosed in PTL 1, control of autonomous driving of a vehicle is limited in scope to a pre-prepared three-dimensional map. Moreover, with the technique disclosed in PTL 2, control of autonomous driving of a vehicle is limited in scope to a set driving route.\n\nIn view of this, one non-limiting and exemplary embodiment provides a moving body control apparatus, a moving body control method, and a training method capable of achieving autonomous driving of a moving body characterized by a higher degree of freedom than conventional art.",
  "cpc": [
    "G05D 1/0016",
    "B25J 9/16",
    "G05D 1/0022",
    "G05D 1/0033",
    "G05D 1/0038",
    "G06N 20/00",
    "G06N 3/0464",
    "G06N 3/08",
    "G06N 3/09",
    "G09G 5/00"
  ],
  "ipc": [
    "B25J 9/16",
    "G05D 1/00",
    "G06N 20/00",
    "G06N 3/08",
    "G09G 5/00"
  ],
  "assignees": [
    "Panasonic Intellectual Property Corp of America"
  ],
  "inventors": [
    "Karthikk HARIHARA SUBRAMANIAN",
    "Bin Zhou",
    "Sheng Mei Shen",
    "Sugiri Pranata Lim"
  ],
  "filing_date": "2019-03-25",
  "publication_date": "2022-09-20",
  "grant_date": "2022-09-20",
  "priority_date": "2017-06-28",
  "application_number": "US-201916362846-A",
  "family_id": "64742591",
  "cited_by_count": 6,
  "citations": [
    "JPH11282530A",
    "US7783391B2",
    "JP2009199572A",
    "JP2011238054A",
    "US20120166200A1",
    "JP2013113274A",
    "US20150032258A1",
    "US20150217449A1",
    "US10599155B1",
    "US20170213457A1",
    "US20160334229A1",
    "US20180032082A1",
    "US20170357257A1",
    "CN106080590A",
    "US20190294159A1",
    "US20190317502A1",
    "US20180247160A1",
    "US20190219998A1",
    "US20190034818A1",
    "US20190118394A1",
    "US20190188542A1",
    "US10922585B2",
    "US20190310648A1",
    "US20200242154A1",
    "US10576636B1",
    "US20210012193A1",
    "US20190391582A1"
  ]
}