MLchartDataset catalogue

Patent · US12147962B2 · B2 · US

Article deduction apparatus, article deduction method, and program

(11) Publication number
US12147962B2
(21) Application number
18/231,605
(22) Filing date
2023-08-08
(30) Priority date
2019-03-01
(43) Publication date
2024-11-19
(45) Date of grant
2024-11-19
(51) IPC
A47F 10/00; G01G 19/52; G01S 17/08; G06Q 20/20; G06Q 30/06; G06T 7/20; G06T 7/50; G06T 7/70; G06V 20/52; G06V 40/10; G06V 40/20
(52) CPC
  • G06Q Information and communication technology [ICT] specially adapted for administrative, commercial, financial, managerial or supervisory purposes; systems or methods specially adapted for administrative, commercial, financial, managerial or supervisory purposes, not otherwise provided for: 20/208, 20/203, 30/06
  • A47F Special furniture, fittings, or accessories for shops, storehouses, bars, restaurants or the like; paying counters: 10/00, 10/02
  • B65G Transport or storage devices, e.g. conveyors for loading or tipping, shop conveyor systems or pneumatic tube conveyors: 1/137
  • G01G Weighing: 19/42, 19/52
  • G01S Radio direction-finding; radio navigation; determining distance or velocity by use of radio waves; locating or presence-detecting by use of the reflection or reradiation of radio waves; analogous arrangements using other waves: 17/08
  • G06T Image data processing or generation, in general: 2207/30196, 2207/30232, 7/20, 7/50, 7/70
  • G06V Image or video recognition or understanding: 20/52, 40/10, 40/16, 40/172, 40/20
  • G07G Registering the receipt of cash, valuables, or tokens: 1/0063, 1/0072, 1/14
(73) Assignee
NEC Corp
(72) Inventors
Motoyuki Nakamura; Hisayoshi MURA; Shinya Yamasaki
(54) Title
Article deduction apparatus, article deduction method, and program
(57) Abstract

An article information deduction apparatus (10) includes an acquisition unit (110) and an output unit (120). The acquisition unit (110) acquires data based on a change in a detected value of a weight sensor (30) (hereinafter described as weight change data). For example, the acquisition unit (110) determines data acquired by chronologically arranging data acquired from the weight sensor (30) as weight change data. Further, the acquisition unit (110) acquires data indicating a movement of a hand of a person positioned in a shelf-front space (hereinafter described as movement data). For example, the acquisition unit (110) acquires data acquired by chronologically arranging data output from a depth sensor (40) to the article information deduction apparatus (10) as movement data. The output unit (120) outputs article determination information of the article deduced to be taken out by the hand of the person positioned in the shelf-front space, by using the weight change data and the movement data.

Full text
View on Google Patents

Claims (20)

  1. An article deduction apparatus comprising: a memory storing instructions; and a processor configured to execute the instructions to: acquire weight change data being data based on a change in a detected value of a weight sensor provided on a shelf on which a plurality of articles can be placed and movement data indicating a movement of a hand of a person positioned in a shelf-front space being a space in front of the shelf; and output the article determination information of the article deduced to be taken out by the hand and associated with the partial area in which a change in a weight satisfies a criterion, out of a plurality of partial areas of aligned horizontally with the shelf viewed from a front, by using the weight change data and the movement data.
  2. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to: generate the movement data by using a detected value of a depth sensor a detection range of which includes the shelf-front space; and when a relation between a height of a shelf in which a change in a detected value of the plurality of weight sensors have occurred and a detected value of a depth sensor that can acquire the movement data satisfies a criterion, output the article determination information associated with the shelf.
  3. The article deduction apparatus according to claim 2, wherein a plurality of shelves with different heights exist, at least one piece of the article determination information is associated with each of the plurality of shelves, and the weight change data indicate a variation in a weight for each of the plurality of shelves.
  4. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to cause a storage to store the article determination information in association with person determination information for determining the person.
  5. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to acquire the person determination information from a person tracking apparatus tracking a movement of the person.
  6. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to: repeatedly acquire a first image being an image including at least part of the shelf-front space; and deduce the article taken out by the hand, by further using an image of the article included in the first image.
  7. The article deduction apparatus according to claim 2, wherein the processor is further configured to execute the instructions to: repeatedly acquire a second image being an image of the shelf captured from a front; and determine the article determination information of the article taken out by the hand, by further using a change in the second image.
  8. The article deduction apparatus according to claim 1, wherein a plurality of weight sensors are provided separately from one another on the shelf, and the processor is further configured to execute the instructions to generate the weight change data by using detected values of the plurality of weight sensors.
  9. The article deduction apparatus according to claim 1, wherein the shelf is installed in a store, the person is a customer, and the processor is further configured to execute the instructions to: perform checkout processing by using the article determination information; and output an electronic receipt based on the checkout processing.
  10. An article deduction method comprising, by a computer: acquiring weight change data being data based on a change in a detected value of a weight sensor provided on a shelf on which a plurality of articles can be placed and movement data indicating a movement of a hand of a person positioned in a shelf-front space being a space in front of the shelf; and outputting the article determination information of the article deduced to be taken out by the hand and associated with the partial area in which a change in a weight satisfies a criterion, out of a plurality of partial areas of aligned horizontally with the shelf viewed from a front, by using the weight change data and the movement data.
  11. The article deduction method according to claim 10, further comprising: generating the movement data by using a detected value of a depth sensor a detection range of which includes the shelf-front space; and when a relation between a height of a shelf in which a change in a detected value of the plurality of weight sensors have occurred and a detected value of a depth sensor that can acquire the movement data satisfies a criterion, outputting the article determination information associated with the shelf.
  12. The article deduction method according to claim 10, further comprising causing a storage to store the article determination information in association with person determination information for determining the person.
  13. The article deduction method according to claim 10, comprising acquiring the person determination information from a person tracking apparatus tracking a movement of the person.
  14. The article deduction method according to claim 10, further comprising: repeatedly acquiring a first image being an image including at least part of the shelf-front space; and deducing the article taken out by the hand, by further using an image of the article included in the first image.
  15. The article deduction method according to claim 10, wherein a plurality of weight sensors are provided separately from one another on the shelf, and the method further comprise generating the weight change data by using detected values of the plurality of weight sensors.
  16. A non-transitory computer-readable medium storing a program for causing a computer to perform operations, the operations comprising: acquiring weight change data being data based on a change in a detected value of a weight sensor provided on a shelf on which a plurality of articles can be placed and movement data indicating a movement of a hand of a person positioned in a shelf-front space being a space in front of the shelf; and outputting the article determination information of the article deduced to be taken out by the hand and associated with the partial area in which a change in a weight satisfies a criterion, out of a plurality of partial areas of aligned horizontally with the shelf viewed from a front, by using the weight change data and the movement data.
  17. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise: generating the movement data by using a detected value of a depth sensor a detection range of which includes the shelf-front space; and when a relation between a height of a shelf in which a change in a detected value of the plurality of weight sensors have occurred and a detected value of a depth sensor that can acquire the movement data satisfies a criterion, outputting the article determination information associated with the shelf.
  18. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise causing a storage to store the article determination information in association with person determination information for determining the person.
  19. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise acquiring the person determination information from a person tracking apparatus tracking a movement of the person.
  20. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise: repeatedly acquiring a first image being an image including at least part of the shelf-front space; and deducing the article taken out by the hand, by further using an image of the article included in the first image.

Description

The present invention relates to an article deduction apparatus, an article deduction method, and a program.

In recent years, technological development for reduction of labor at a store, a factory, and the like is under way. For example, PTL 1 describes, in work of boxing a plurality of types of articles taken out from an inventory shelf as one set, measuring the total weight of articles 5 stored in the inventory shelf and determining whether to issue a warning by using the measurement result.

Further, PTL 2 describes, in order to manage handling of articles, generating inventory data by using a result of processing a captured image of a bar code or a QR code (registered trademark) of an article. Inventory data are information indicating handling status of an article and, for example, are information correlating identification information of an article an image of which is captured, a date and time when checking of the article is performed, a location where the article is installed, and identification information of a user handling the article with one another.

In order to promote reduction of labor, it is preferable to enable automatic determination of an article taken out from a shelf. An object of the present invention is to improve determination precision of an article taken out from a shelf.

Citations (44)

  • US10430798B2
  • US20140179231A1
  • US20140249971A1
  • JP2014194732A
  • US20160203431A1
  • US20190147446A1
  • US20150012396A1
  • US20150039458A1
  • US20150169961A1
  • US20150213498A1
  • US20190007381A1
  • US20190281030A1
  • US10078003B2
  • US20190088096A1
  • US20170256148A1
  • US9911290B1
  • WO2017163909A1
  • US10198710B1
  • US10861086B2
  • JP2017210310A
  • JP2017218289A
  • US10339656B1
  • US20180107970A1
  • US20180139570A1
  • JP2018160107A
  • US20190149725A1
  • US20190087663A1
  • US10963704B2
  • US10516982B2
  • US11030442B1
  • US11284041B1
  • US10318569B1
  • US10885336B1
  • US20200410275A1
  • US20210004606A1
  • JP2018206372A
  • US20190392189A1
  • US20190392220A1
  • US11301984B1
  • US11468681B1
  • US10282852B1
  • US11085809B1
  • US11132658B2
  • US10805556B1
Record as JSON 
{
  "publication_number": "US12147962B2",
  "country": "US",
  "kind": "B2",
  "title": "Article deduction apparatus, article deduction method, and program",
  "abstract": "An article information deduction apparatus (10) includes an acquisition unit (110) and an output unit (120). The acquisition unit (110) acquires data based on a change in a detected value of a weight sensor (30) (hereinafter described as weight change data). For example, the acquisition unit (110) determines data acquired by chronologically arranging data acquired from the weight sensor (30) as weight change data. Further, the acquisition unit (110) acquires data indicating a movement of a hand of a person positioned in a shelf-front space (hereinafter described as movement data). For example, the acquisition unit (110) acquires data acquired by chronologically arranging data output from a depth sensor (40) to the article information deduction apparatus (10) as movement data. The output unit (120) outputs article determination information of the article deduced to be taken out by the hand of the person positioned in the shelf-front space, by using the weight change data and the movement data.",
  "claims": [
    "1. An article deduction apparatus comprising: a memory storing instructions; and a processor configured to execute the instructions to: acquire weight change data being data based on a change in a detected value of a weight sensor provided on a shelf on which a plurality of articles can be placed and movement data indicating a movement of a hand of a person positioned in a shelf-front space being a space in front of the shelf; and output the article determination information of the article deduced to be taken out by the hand and associated with the partial area in which a change in a weight satisfies a criterion, out of a plurality of partial areas of aligned horizontally with the shelf viewed from a front, by using the weight change data and the movement data.",
    "2. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to: generate the movement data by using a detected value of a depth sensor a detection range of which includes the shelf-front space; and when a relation between a height of a shelf in which a change in a detected value of the plurality of weight sensors have occurred and a detected value of a depth sensor that can acquire the movement data satisfies a criterion, output the article determination information associated with the shelf.",
    "3. The article deduction apparatus according to claim 2, wherein a plurality of shelves with different heights exist, at least one piece of the article determination information is associated with each of the plurality of shelves, and the weight change data indicate a variation in a weight for each of the plurality of shelves.",
    "4. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to cause a storage to store the article determination information in association with person determination information for determining the person.",
    "5. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to acquire the person determination information from a person tracking apparatus tracking a movement of the person.",
    "6. The article deduction apparatus according to claim 1, wherein the processor is further configured to execute the instructions to: repeatedly acquire a first image being an image including at least part of the shelf-front space; and deduce the article taken out by the hand, by further using an image of the article included in the first image.",
    "7. The article deduction apparatus according to claim 2, wherein the processor is further configured to execute the instructions to: repeatedly acquire a second image being an image of the shelf captured from a front; and determine the article determination information of the article taken out by the hand, by further using a change in the second image.",
    "8. The article deduction apparatus according to claim 1, wherein a plurality of weight sensors are provided separately from one another on the shelf, and the processor is further configured to execute the instructions to generate the weight change data by using detected values of the plurality of weight sensors.",
    "9. The article deduction apparatus according to claim 1, wherein the shelf is installed in a store, the person is a customer, and the processor is further configured to execute the instructions to: perform checkout processing by using the article determination information; and output an electronic receipt based on the checkout processing.",
    "10. An article deduction method comprising, by a computer: acquiring weight change data being data based on a change in a detected value of a weight sensor provided on a shelf on which a plurality of articles can be placed and movement data indicating a movement of a hand of a person positioned in a shelf-front space being a space in front of the shelf; and outputting the article determination information of the article deduced to be taken out by the hand and associated with the partial area in which a change in a weight satisfies a criterion, out of a plurality of partial areas of aligned horizontally with the shelf viewed from a front, by using the weight change data and the movement data.",
    "11. The article deduction method according to claim 10, further comprising: generating the movement data by using a detected value of a depth sensor a detection range of which includes the shelf-front space; and when a relation between a height of a shelf in which a change in a detected value of the plurality of weight sensors have occurred and a detected value of a depth sensor that can acquire the movement data satisfies a criterion, outputting the article determination information associated with the shelf.",
    "12. The article deduction method according to claim 10, further comprising causing a storage to store the article determination information in association with person determination information for determining the person.",
    "13. The article deduction method according to claim 10, comprising acquiring the person determination information from a person tracking apparatus tracking a movement of the person.",
    "14. The article deduction method according to claim 10, further comprising: repeatedly acquiring a first image being an image including at least part of the shelf-front space; and deducing the article taken out by the hand, by further using an image of the article included in the first image.",
    "15. The article deduction method according to claim 10, wherein a plurality of weight sensors are provided separately from one another on the shelf, and the method further comprise generating the weight change data by using detected values of the plurality of weight sensors.",
    "16. A non-transitory computer-readable medium storing a program for causing a computer to perform operations, the operations comprising: acquiring weight change data being data based on a change in a detected value of a weight sensor provided on a shelf on which a plurality of articles can be placed and movement data indicating a movement of a hand of a person positioned in a shelf-front space being a space in front of the shelf; and outputting the article determination information of the article deduced to be taken out by the hand and associated with the partial area in which a change in a weight satisfies a criterion, out of a plurality of partial areas of aligned horizontally with the shelf viewed from a front, by using the weight change data and the movement data.",
    "17. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise: generating the movement data by using a detected value of a depth sensor a detection range of which includes the shelf-front space; and when a relation between a height of a shelf in which a change in a detected value of the plurality of weight sensors have occurred and a detected value of a depth sensor that can acquire the movement data satisfies a criterion, outputting the article determination information associated with the shelf.",
    "18. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise causing a storage to store the article determination information in association with person determination information for determining the person.",
    "19. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise acquiring the person determination information from a person tracking apparatus tracking a movement of the person.",
    "20. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise: repeatedly acquiring a first image being an image including at least part of the shelf-front space; and deducing the article taken out by the hand, by further using an image of the article included in the first image."
  ],
  "description_excerpt": "The present invention relates to an article deduction apparatus, an article deduction method, and a program.\n\nIn recent years, technological development for reduction of labor at a store, a factory, and the like is under way. For example, PTL 1 describes, in work of boxing a plurality of types of articles taken out from an inventory shelf as one set, measuring the total weight of articles 5 stored in the inventory shelf and determining whether to issue a warning by using the measurement result.\n\nFurther, PTL 2 describes, in order to manage handling of articles, generating inventory data by using a result of processing a captured image of a bar code or a QR code (registered trademark) of an article. Inventory data are information indicating handling status of an article and, for example, are information correlating identification information of an article an image of which is captured, a date and time when checking of the article is performed, a location where the article is installed, and identification information of a user handling the article with one another.\n\nIn order to promote reduction of labor, it is preferable to enable automatic determination of an article taken out from a shelf. An object of the present invention is to improve determination precision of an article taken out from a shelf.",
  "cpc": [
    "G06Q 20/208",
    "A47F 10/00",
    "A47F 10/02",
    "B65G 1/137",
    "G01G 19/42",
    "G01G 19/52",
    "G01S 17/08",
    "G06Q 20/203",
    "G06Q 30/06",
    "G06T 2207/30196",
    "G06T 2207/30232",
    "G06T 7/20",
    "G06T 7/50",
    "G06T 7/70",
    "G06V 20/52",
    "G06V 40/10",
    "G06V 40/16",
    "G06V 40/172",
    "G06V 40/20",
    "G07G 1/0063",
    "G07G 1/0072",
    "G07G 1/14"
  ],
  "ipc": [
    "A47F 10/00",
    "G01G 19/52",
    "G01S 17/08",
    "G06Q 20/20",
    "G06Q 30/06",
    "G06T 7/20",
    "G06T 7/50",
    "G06T 7/70",
    "G06V 20/52",
    "G06V 40/10",
    "G06V 40/20"
  ],
  "assignees": [
    "NEC Corp"
  ],
  "inventors": [
    "Motoyuki Nakamura",
    "Hisayoshi MURA",
    "Shinya Yamasaki"
  ],
  "filing_date": "2023-08-08",
  "publication_date": "2024-11-19",
  "grant_date": "2024-11-19",
  "priority_date": "2019-03-01",
  "application_number": "US-202318231605-A",
  "family_id": "72337272",
  "cited_by_count": 0,
  "citations": [
    "US10430798B2",
    "US20140179231A1",
    "US20140249971A1",
    "JP2014194732A",
    "US20160203431A1",
    "US20190147446A1",
    "US20150012396A1",
    "US20150039458A1",
    "US20150169961A1",
    "US20150213498A1",
    "US20190007381A1",
    "US20190281030A1",
    "US10078003B2",
    "US20190088096A1",
    "US20170256148A1",
    "US9911290B1",
    "WO2017163909A1",
    "US10198710B1",
    "US10861086B2",
    "JP2017210310A",
    "JP2017218289A",
    "US10339656B1",
    "US20180107970A1",
    "US20180139570A1",
    "JP2018160107A",
    "US20190149725A1",
    "US20190087663A1",
    "US10963704B2",
    "US10516982B2",
    "US11030442B1",
    "US11284041B1",
    "US10318569B1",
    "US10885336B1",
    "US20200410275A1",
    "US20210004606A1",
    "JP2018206372A",
    "US20190392189A1",
    "US20190392220A1",
    "US11301984B1",
    "US11468681B1",
    "US10282852B1",
    "US11085809B1",
    "US11132658B2",
    "US10805556B1"
  ]
}

Record 311 of 8,000 in Patents full text (MLC-0201). Request the full dataset.