Patent · US11571814B2 · B2 · US
Determining how to assemble a meal
- (11) Publication number
- US11571814B2
- (21) Application number
- 16/570,976
- (22) Filing date
- 2019-09-13
- (30) Priority date
- 2018-09-13
- (43) Publication date
- 2023-02-07
- (45) Date of grant
- 2023-02-07
- (51) IPC
- A47J 44/00; B25J 11/00; B25J 13/00; B25J 13/08; B25J 15/00; B25J 15/04; B25J 19/00; B25J 19/02; B25J 9/00; B25J 9/16; B65G 1/137; G05B 19/4061; G05D 1/02; G06N 3/08; G06Q 10/06; G06V 40/20; G10L 15/22; H04L 67/12
- (52) CPC
- B25J Manipulators; chambers provided with manipulation devices: 9/1666, 11/0045, 13/003, 13/085, 13/088, 15/0052, 15/0408, 19/0083, 19/023, 9/0009, 9/16, 9/161, 9/1633, 9/1653, 9/1664, 9/1674, 9/1676, 9/1682, 9/1687, 9/1697
- A47J Kitchen equipment; coffee mills; spice mills; apparatus for making beverages: 44/00
- B65G Transport or storage devices, e.g. conveyors for loading or tipping, shop conveyor systems or pneumatic tube conveyors: 1/137
- G05B Control or regulating systems in general; functional elements of such systems; monitoring or testing arrangements for such systems or elements: 19/4061, 2219/32335, 2219/39001, 2219/39091, 2219/39319, 2219/39342, 2219/39468, 2219/40201, 2219/40202, 2219/40411, 2219/40497, 2219/45111, 2219/49157, 2219/50391
- G05D Systems for controlling or regulating non-electric variables: 1/02
- G06N Computing arrangements based on specific computational models: 3/0464, 3/08
- G06Q Information and communication technology [ICT] specially adapted for administrative, commercial, financial, managerial or supervisory purposes; systems or methods specially adapted for administrative, commercial, financial, managerial or supervisory purposes, not otherwise provided for: 10/06316
- G06V Image or video recognition or understanding: 40/28
- G10L Speech analysis techniques or speech synthesis; speech recognition; speech or voice processing techniques; speech or audio coding or decoding: 15/22
- H04L Transmission of digital information, e.g. telegraphic communication: 67/12
- (73) Assignee
- Charles Stark Draper Laboratory Inc
- (72) Inventors
- David M. S. Johnson; Syler Wagner; Steven Lines; Mitchell Hebert
- (54) Title
- Determining how to assemble a meal
- (57) Abstract
In an embodiment, a method includes determining a given material to manipulate to achieve a goal state. The goal state can be one or more deformable or granular materials in a particular arrangement. The method further includes, for the given material, determining, a respective outcome for each of a plurality of candidate actions to manipulate the given material. The determining can be performed with a physics-based model, in one embodiment. The method further can include determining a given action of the candidate actions, where the outcome of the given action reaching the goal state is within at least one tolerance. The method further includes, based on a selected action of the given actions, generating a first motion plan for the selected action.
- Full text
- View on Google Patents
Claims (14)
- A method comprising: determining a given material to manipulate with an end effector of one or more autonomous robots to achieve a goal state, the goal state being one or more deformable or granular materials in a geometric distribution; for the given material, simulating a respective simulated geometric distribution for each of a plurality of candidate actions performed by the end effector of the one or more autonomous robots to manipulate the given material; selecting a given action of the candidate actions based on the respective simulated geometric distribution of the selected given action having a closest geometric distribution of the goal state; based on the selected given action, generating a first motion plan for the one or more autonomous robots to execute the selected given action; and executing the first motion plan using the one or more autonomous robots.
- The method of claim 1, further comprising: creating a new goal state having the given material manipulated; determining a new given material to manipulate to achieve a new goal state, the new goal state being one or more deformable or granular materials including the given material in a particular arrangement; for the new given material, simulating a respective simulated geometric distribution for each of a plurality of new candidate actions to manipulate the new given material; selecting a given new action of the new candidate actions based on the respective simulated geometric distribution of the new given action having a closest geometric distribution of the new goal state; and based on the new selected given action, generating a second motion plan for the new selected given action.
- The method of claim 2, further comprising: ordering, for execution, the first motion plan and second motion plan based on one or more rules, wherein the rules include one or more of material based rules, ease of assembly rules, and parallelization rules, said ordering resulting in an order.
- The method of claim 3, further comprising: executing the first motion plan and second motion plan in the order using one or more autonomous robots.
- The method of claim 4, wherein a first autonomous robot of the one or more autonomous robots executes the first motion plan and a second autonomous robot of the one or more autonomous robots executes the second motion plan.
- The method of claim 3, wherein ordering is based on at least one of: one or more instructions of ingredients or actions; a set of material rules; predicted based on a physics-based model, and a heuristic.
- The method of claim 1, wherein determining a given action of the candidate actions includes one or more of: adding the given material to the goal state; removing a given material from an existing state; and performing a process step to the given material and one or more other materials.
- A system comprising: a processor; and a memory with computer code instructions stored thereon, the processor and the memory, with the computer code instructions, being configured to cause the system to: determine a given material to manipulate with an end effector of one or more autonomous robots to achieve a goal state, the goal state being one or more deformable or granular materials in a geometric distribution; for the given material, simulate a respective simulated geometric distribution for each of a plurality of candidate actions performed by the end effector of the one or more autonomous robots to manipulate the given material; selecting a given action of the candidate actions based on the respective simulated geometric distribution of the selected given action having a closest geometric distribution of the goal state; based on the selected given action of the given one or more actions, generate a first motion plan for the one or more autonomous robots to execute selected action; and executing the first motion plan using one or more autonomous robots.
- The system of claim 8, wherein the instructions further cause the processor to: create a new goal state having the given material manipulated; determine a new given material to manipulate to achieve a new goal state, the new goal state being one or more deformable or granular materials including the given material in a particular arrangement; for the new given material, simulate, with a physics-based simulator, a respective simulated geometric distribution of new candidate actions to manipulate the new given material; select a given new action of the candidate actions based on the respective simulated geometric distribution of the given new action reaching the new goal state; based on a new selected action, generate a second motion plan for the new selected action.
- The system of claim 9, wherein the instructions further cause the processor to: order, for execution, the first motion plan and second motion plan based on one or more rules, wherein the rules include one or more of material based rules, ease of assembly rules, and parallelization rules, said ordering resulting in an order.
- The system of claim 10, wherein the instructions further cause the processor to: execute the first motion plan and second motion plan in the order using one or more autonomous robots.
- The system of claim 11, wherein a first autonomous of the one or more autonomous robot executes the first motion plan and a second autonomous robot of the one or more autonomous robots executes the second motion plan.
- The system of claim 10, wherein ordering is based on one or more instructions of ingredients or actions.
- The system of claim 8, wherein determining a given action of the candidate actions includes one or more of: adding the given material to the goal state; and performing a process step to the given material and one or more other materials.
Description
Traditionally, the food industry employs human labor to manipulate ingredients with the purpose of either assembling a meal such as a salad or a bowl, or packing a box of ingredients such as those used in grocery shopping, or preparing the raw ingredients. Robots have not yet been able to assemble complete meals from prepared ingredients in a food-service setting such as a restaurant, largely because the ingredients are arranged unpredictably and change shape in difficult-to-predict ways rendering traditional methods to move material ineffective without extensive modifications to existing kitchens. Additionally, traditional material handling methods are ill-suited to moving cooked foods without altering their texture and taste-profile. These difficulties arise because the friction, stiction, and viscosity of commonly consumed foods cause auger, conveyor, and suction mechanisms to become clogged and soiled, while these mechanisms simultaneously impart forces on the foodstuffs which alter their texture, consistency, and taste-profile in unappetizing ways.
In embodiments, the below disclosure solves problems in relation to employing robotics in the quick service fast food restaurant environment. In restaurants, orders are increasingly directly entered on a tablet or other digital device; if not directly entered digitally, they are often translated from paper into a central system at a kiosk. This digital order information encodes a specific set of recipes to be prepared for a customer, along with any modifications, changes, requested or allowed ingredient substitutions, and food allergies.
Citations (179)
- US4513709A
- US4624043A
- US4512709A
- EP0138461A2
- US4611377A
- US4676142A
- US4604787A
- US4896357A
- DE3723329A1
- US4875275A
- US5018266A
- DE3823102A1
- US4904514A
- US5136223A
- EP0474881A1
- US5044063A
- US5131706A
- US5360249A
- US5396346A
- US5774841A
- US5879277A
- WO1999003653A1
- US6223110B1
- US20050171643A1
- US6678572B1
- US6427995B1
- EP1145804A1
- US20020158599A1
- US20030060930A1
- US20020181773A1
- US20020144565A1
- US20020151848A1
- US20040039483A1
- US20040172380A1
- US6569070B1
- US8095237B2
- US20050004710A1
- US20060137164A1
- JP2004295620A
- US20060141200A1
- WO2005072917A1
- US20050193901A1
- US20050283475A1
- US20080161970A1
- US20060165953A1
- US20080201016A1
- US20070233321A1
- US20070276539A1
- US20070274812A1
- JP2008049462A
- US20080059178A1
- US20080177421A1
- US20080237921A1
- EP2011610A2
- US20100114371A1
- US20090075796A1
- WO2009045827A2
- US20100292707A1
- US20090292298A1
- US20110256995A1
- US20110060462A1
- US20110125504A1
- US20120016678A1
- US20110238212A1
- US20150114236A1
- US20130103198A1
- US20120255388A1
- US20120290134A1
- US20130079930A1
- US20130103918A1
- US9427876B2
- US20180214221A1
- US20140067121A1
- US20140163736A1
- US9259840B1
- US20160103202A1
- US20140316636A1
- US9346164B1
- US9186795B1
- US20150032260A1
- US20150051734A1
- US20150052703A1
- US20150117156A1
- US9189742B2
- US20150148953A1
- US20150149175A1
- US20150178953A1
- FR3015334A1
- US9659225B2
- US20180147718A1
- US20160372138A1
- US20150277430A1
- US20170080565A1
- US20150375402A1
- US20160016315A1
- US20160075023A1
- US20160073644A1
- US9547306B2
- US9801517B2
- US20160291571A1
- US20170004406A1
- US20190039241A1
- US20180200014A1
- US9744668B1
- US20180257221A1
- US11116593B2
- US20170087722A1
- US9621984B1
- US20170133009A1
- US20180354140A1
- US20170168488A1
- US20220066456A1
- US20180147723A1
- US20200298403A1
- US9615066B1
- US9800973B1
- US20170326728A1
- WO2017197170A1
- GB2550396A
- US20170334066A1
- US20170354294A1
- US20170361468A1
- US20170361461A1
- US20180043952A1
- US20180056520A1
- US20180070776A1
- US10131053B1
- CN106313068A
- US20180121994A1
- US20180144244A1
- US20180150661A1
- US20180200885A1
- US20180202819A1
- WO2018133861A1
- US20200030966A1
- US11351673B2
- US20180275632A1
- US20180338504A1
- CN107092209A
- US20180348783A1
- US20180345479A1
- US20190001489A1
- US10427306B1
- US20190291277A1
- US20190056751A1
- US20190049970A1
- US20190066680A1
- US20190212441A1
- US11016491B1
- US20190310611A1
- US20190321989A1
- US20190381617A1
- US20200023520A1
- US20200047349A1
- JP2020028957A
- US20200073367A1
- US20200073358A1
- US20200070355A1
- WO2020056362A1
- WO2020056380A1
- US20200086437A1
- WO2020056353A1
- WO2020056376A1
- WO2020056373A1
- US20200086482A1
- WO2020056301A1
- WO2020056374A1
- US20200086497A1
- WO2020056375A1
- US20200086509A1
- US20200086502A1
- WO2020056295A2
- US20200086503A1
- US20200087069A1
- US20200086487A1
- US20200086485A1
- US20200086498A1
- WO2020056279A1
- WO2020056377A1
Record as JSON
{
"publication_number": "US11571814B2",
"country": "US",
"kind": "B2",
"title": "Determining how to assemble a meal",
"abstract": "In an embodiment, a method includes determining a given material to manipulate to achieve a goal state. The goal state can be one or more deformable or granular materials in a particular arrangement. The method further includes, for the given material, determining, a respective outcome for each of a plurality of candidate actions to manipulate the given material. The determining can be performed with a physics-based model, in one embodiment. The method further can include determining a given action of the candidate actions, where the outcome of the given action reaching the goal state is within at least one tolerance. The method further includes, based on a selected action of the given actions, generating a first motion plan for the selected action.",
"claims": [
"1. A method comprising: determining a given material to manipulate with an end effector of one or more autonomous robots to achieve a goal state, the goal state being one or more deformable or granular materials in a geometric distribution; for the given material, simulating a respective simulated geometric distribution for each of a plurality of candidate actions performed by the end effector of the one or more autonomous robots to manipulate the given material; selecting a given action of the candidate actions based on the respective simulated geometric distribution of the selected given action having a closest geometric distribution of the goal state; based on the selected given action, generating a first motion plan for the one or more autonomous robots to execute the selected given action; and executing the first motion plan using the one or more autonomous robots.",
"2. The method of claim 1, further comprising: creating a new goal state having the given material manipulated; determining a new given material to manipulate to achieve a new goal state, the new goal state being one or more deformable or granular materials including the given material in a particular arrangement; for the new given material, simulating a respective simulated geometric distribution for each of a plurality of new candidate actions to manipulate the new given material; selecting a given new action of the new candidate actions based on the respective simulated geometric distribution of the new given action having a closest geometric distribution of the new goal state; and based on the new selected given action, generating a second motion plan for the new selected given action.",
"3. The method of claim 2, further comprising: ordering, for execution, the first motion plan and second motion plan based on one or more rules, wherein the rules include one or more of material based rules, ease of assembly rules, and parallelization rules, said ordering resulting in an order.",
"4. The method of claim 3, further comprising: executing the first motion plan and second motion plan in the order using one or more autonomous robots.",
"5. The method of claim 4, wherein a first autonomous robot of the one or more autonomous robots executes the first motion plan and a second autonomous robot of the one or more autonomous robots executes the second motion plan.",
"6. The method of claim 3, wherein ordering is based on at least one of: one or more instructions of ingredients or actions; a set of material rules; predicted based on a physics-based model, and a heuristic.",
"7. The method of claim 1, wherein determining a given action of the candidate actions includes one or more of: adding the given material to the goal state; removing a given material from an existing state; and performing a process step to the given material and one or more other materials.",
"8. A system comprising: a processor; and a memory with computer code instructions stored thereon, the processor and the memory, with the computer code instructions, being configured to cause the system to: determine a given material to manipulate with an end effector of one or more autonomous robots to achieve a goal state, the goal state being one or more deformable or granular materials in a geometric distribution; for the given material, simulate a respective simulated geometric distribution for each of a plurality of candidate actions performed by the end effector of the one or more autonomous robots to manipulate the given material; selecting a given action of the candidate actions based on the respective simulated geometric distribution of the selected given action having a closest geometric distribution of the goal state; based on the selected given action of the given one or more actions, generate a first motion plan for the one or more autonomous robots to execute selected action; and executing the first motion plan using one or more autonomous robots.",
"9. The system of claim 8, wherein the instructions further cause the processor to: create a new goal state having the given material manipulated; determine a new given material to manipulate to achieve a new goal state, the new goal state being one or more deformable or granular materials including the given material in a particular arrangement; for the new given material, simulate, with a physics-based simulator, a respective simulated geometric distribution of new candidate actions to manipulate the new given material; select a given new action of the candidate actions based on the respective simulated geometric distribution of the given new action reaching the new goal state; based on a new selected action, generate a second motion plan for the new selected action.",
"10. The system of claim 9, wherein the instructions further cause the processor to: order, for execution, the first motion plan and second motion plan based on one or more rules, wherein the rules include one or more of material based rules, ease of assembly rules, and parallelization rules, said ordering resulting in an order.",
"11. The system of claim 10, wherein the instructions further cause the processor to: execute the first motion plan and second motion plan in the order using one or more autonomous robots.",
"12. The system of claim 11, wherein a first autonomous of the one or more autonomous robot executes the first motion plan and a second autonomous robot of the one or more autonomous robots executes the second motion plan.",
"13. The system of claim 10, wherein ordering is based on one or more instructions of ingredients or actions.",
"14. The system of claim 8, wherein determining a given action of the candidate actions includes one or more of: adding the given material to the goal state; and performing a process step to the given material and one or more other materials."
],
"description_excerpt": "Traditionally, the food industry employs human labor to manipulate ingredients with the purpose of either assembling a meal such as a salad or a bowl, or packing a box of ingredients such as those used in grocery shopping, or preparing the raw ingredients. Robots have not yet been able to assemble complete meals from prepared ingredients in a food-service setting such as a restaurant, largely because the ingredients are arranged unpredictably and change shape in difficult-to-predict ways rendering traditional methods to move material ineffective without extensive modifications to existing kitchens. Additionally, traditional material handling methods are ill-suited to moving cooked foods without altering their texture and taste-profile. These difficulties arise because the friction, stiction, and viscosity of commonly consumed foods cause auger, conveyor, and suction mechanisms to become clogged and soiled, while these mechanisms simultaneously impart forces on the foodstuffs which alter their texture, consistency, and taste-profile in unappetizing ways.\n\nIn embodiments, the below disclosure solves problems in relation to employing robotics in the quick service fast food restaurant environment. In restaurants, orders are increasingly directly entered on a tablet or other digital device; if not directly entered digitally, they are often translated from paper into a central system at a kiosk. This digital order information encodes a specific set of recipes to be prepared for a customer, along with any modifications, changes, requested or allowed ingredient substitutions, and food allergies.",
"cpc": [
"B25J 9/1666",
"A47J 44/00",
"B25J 11/0045",
"B25J 13/003",
"B25J 13/085",
"B25J 13/088",
"B25J 15/0052",
"B25J 15/0408",
"B25J 19/0083",
"B25J 19/023",
"B25J 9/0009",
"B25J 9/16",
"B25J 9/161",
"B25J 9/1633",
"B25J 9/1653",
"B25J 9/1664",
"B25J 9/1674",
"B25J 9/1676",
"B25J 9/1682",
"B25J 9/1687",
"B25J 9/1697",
"B65G 1/137",
"G05B 19/4061",
"G05B 2219/32335",
"G05B 2219/39001",
"G05B 2219/39091",
"G05B 2219/39319",
"G05B 2219/39342",
"G05B 2219/39468",
"G05B 2219/40201",
"G05B 2219/40202",
"G05B 2219/40411",
"G05B 2219/40497",
"G05B 2219/45111",
"G05B 2219/49157",
"G05B 2219/50391",
"G05D 1/02",
"G06N 3/0464",
"G06N 3/08",
"G06Q 10/06316",
"G06V 40/28",
"G10L 15/22",
"H04L 67/12"
],
"ipc": [
"A47J 44/00",
"B25J 11/00",
"B25J 13/00",
"B25J 13/08",
"B25J 15/00",
"B25J 15/04",
"B25J 19/00",
"B25J 19/02",
"B25J 9/00",
"B25J 9/16",
"B65G 1/137",
"G05B 19/4061",
"G05D 1/02",
"G06N 3/08",
"G06Q 10/06",
"G06V 40/20",
"G10L 15/22",
"H04L 67/12"
],
"assignees": [
"Charles Stark Draper Laboratory Inc"
],
"inventors": [
"David M. S. Johnson",
"Syler Wagner",
"Steven Lines",
"Mitchell Hebert"
],
"filing_date": "2019-09-13",
"publication_date": "2023-02-07",
"grant_date": "2023-02-07",
"priority_date": "2018-09-13",
"application_number": "US-201916570976-A",
"family_id": "68069913",
"cited_by_count": 6,
"citations": [
"US4513709A",
"US4624043A",
"US4512709A",
"EP0138461A2",
"US4611377A",
"US4676142A",
"US4604787A",
"US4896357A",
"DE3723329A1",
"US4875275A",
"US5018266A",
"DE3823102A1",
"US4904514A",
"US5136223A",
"EP0474881A1",
"US5044063A",
"US5131706A",
"US5360249A",
"US5396346A",
"US5774841A",
"US5879277A",
"WO1999003653A1",
"US6223110B1",
"US20050171643A1",
"US6678572B1",
"US6427995B1",
"EP1145804A1",
"US20020158599A1",
"US20030060930A1",
"US20020181773A1",
"US20020144565A1",
"US20020151848A1",
"US20040039483A1",
"US20040172380A1",
"US6569070B1",
"US8095237B2",
"US20050004710A1",
"US20060137164A1",
"JP2004295620A",
"US20060141200A1",
"WO2005072917A1",
"US20050193901A1",
"US20050283475A1",
"US20080161970A1",
"US20060165953A1",
"US20080201016A1",
"US20070233321A1",
"US20070276539A1",
"US20070274812A1",
"JP2008049462A",
"US20080059178A1",
"US20080177421A1",
"US20080237921A1",
"EP2011610A2",
"US20100114371A1",
"US20090075796A1",
"WO2009045827A2",
"US20100292707A1",
"US20090292298A1",
"US20110256995A1",
"US20110060462A1",
"US20110125504A1",
"US20120016678A1",
"US20110238212A1",
"US20150114236A1",
"US20130103198A1",
"US20120255388A1",
"US20120290134A1",
"US20130079930A1",
"US20130103918A1",
"US9427876B2",
"US20180214221A1",
"US20140067121A1",
"US20140163736A1",
"US9259840B1",
"US20160103202A1",
"US20140316636A1",
"US9346164B1",
"US9186795B1",
"US20150032260A1",
"US20150051734A1",
"US20150052703A1",
"US20150117156A1",
"US9189742B2",
"US20150148953A1",
"US20150149175A1",
"US20150178953A1",
"FR3015334A1",
"US9659225B2",
"US20180147718A1",
"US20160372138A1",
"US20150277430A1",
"US20170080565A1",
"US20150375402A1",
"US20160016315A1",
"US20160075023A1",
"US20160073644A1",
"US9547306B2",
"US9801517B2",
"US20160291571A1",
"US20170004406A1",
"US20190039241A1",
"US20180200014A1",
"US9744668B1",
"US20180257221A1",
"US11116593B2",
"US20170087722A1",
"US9621984B1",
"US20170133009A1",
"US20180354140A1",
"US20170168488A1",
"US20220066456A1",
"US20180147723A1",
"US20200298403A1",
"US9615066B1",
"US9800973B1",
"US20170326728A1",
"WO2017197170A1",
"GB2550396A",
"US20170334066A1",
"US20170354294A1",
"US20170361468A1",
"US20170361461A1",
"US20180043952A1",
"US20180056520A1",
"US20180070776A1",
"US10131053B1",
"CN106313068A",
"US20180121994A1",
"US20180144244A1",
"US20180150661A1",
"US20180200885A1",
"US20180202819A1",
"WO2018133861A1",
"US20200030966A1",
"US11351673B2",
"US20180275632A1",
"US20180338504A1",
"CN107092209A",
"US20180348783A1",
"US20180345479A1",
"US20190001489A1",
"US10427306B1",
"US20190291277A1",
"US20190056751A1",
"US20190049970A1",
"US20190066680A1",
"US20190212441A1",
"US11016491B1",
"US20190310611A1",
"US20190321989A1",
"US20190381617A1",
"US20200023520A1",
"US20200047349A1",
"JP2020028957A",
"US20200073367A1",
"US20200073358A1",
"US20200070355A1",
"WO2020056362A1",
"WO2020056380A1",
"US20200086437A1",
"WO2020056353A1",
"WO2020056376A1",
"WO2020056373A1",
"US20200086482A1",
"WO2020056301A1",
"WO2020056374A1",
"US20200086497A1",
"WO2020056375A1",
"US20200086509A1",
"US20200086502A1",
"WO2020056295A2",
"US20200086503A1",
"US20200087069A1",
"US20200086487A1",
"US20200086485A1",
"US20200086498A1",
"WO2020056279A1",
"WO2020056377A1"
]
}
Record 858 of 8,000 in Patents full text (MLC-0201). Request the full dataset.