Multimedia device and method for controlling the same

{
  "publication_number": "US2021055801A1",
  "country": "US",
  "kind": "A1",
  "title": "Multimedia device and method for controlling the same",
  "abstract": "Disclosed are an extended reality (XR) device and a control method thereof, which are applicable to all of 5G communication technology field, a robot technology field, an autonomous driving technology field, and an AI technology field.",
  "claims": [
    "1. A method for operating a multimedia device to provide a virtual fitting service, the method comprising: capturing a user around the multimedia device using a camera; displaying the captured user or a corresponding virtual avatar thereto; recognizing a specific gesture of the user using the camera; determining a specific portion of a body of the user corresponding to the recognized specific gesture made by the user; and selecting at least one virtual clothes belonging to a specific category corresponding to the determined specific portion and displaying the selected at least one virtual clothes, wherein the virtual clothes are stored in a memory. 2. The method of claim 1, wherein the specific category changes based on the specific portion. 3. The method of claim 2, wherein the method further includes analyzing a gender and an age of the user using the camera. 4. The method of claim 3, wherein the specific category varies based on the gender and age of the user. 5. The method of claim 4, wherein the method further includes: calculating a time duration for which the specific gesture is recognized around the specific portion using the camera; determining whether the calculated time duration exceeds a predefined threshold value; upon determination that the calculated time duration exceeds the predefined threshold value, displaying the at least one virtual clothes belonging to the specific category. 6. A method of operating a multimedia device to provide a virtual fitting service, the method comprising: capturing a user around the multimedia device using a camera; displaying the captured user or a corresponding virtual avatar thereto; recognizing a first gesture and a face of the user using the camera; determining whether a plurality of virtual clothes-related specific categories corresponding to the recognized first gesture are present in a memory; displaying a graphic image for guiding a second gesture to select a single specific category when the plurality of virtual clothes-related specific categories corresponding to the recognized first gesture are present in the memory, wherein the graphic image is displayed for a predefined period P 1, wherein the predefined period P 1 is determined based on the recognized face of the user; selecting the single specific category among the plurality of virtual clothes-related specific categories based on the second gesture recognized using the camera; and selecting and displaying at least one virtual clothes belonging to the single specific category. 7. The method of claim 6, wherein displaying the graphic image includes displaying the graphic image only for the predefined period P 1 from a time T 1 at which the first gesture is recognized. 8. The method of claim 7, wherein the method further includes: calculating a difference between a time T 2 at which the second gesture is recognized and a time T 1 at which the first gesture is recognized; and mapping the recognized user's face to the difference and storing the mapping in a memory. 9. The method of claim 8, wherein the method further includes: re-recognizing a face of the user; and re-configuring the predefined period P 1 based on the user's face and the difference stored in the memory. 10. The method of claim 9, wherein the method further includes: connecting the device to a robot over short range communication or 5G communication network; receiving, from the robot, an image of the user taken by the robot. 11. A multimedia device for providing a virtual fitting service, the device comprising: a memory; a camera for capturing a user around the multimedia device; a display configured to display the captured user or a corresponding virtual avatar thereto; and a controller configured for: recognizing a specific gesture of the user using the camera; determining a specific portion of a body of the user corresponding to the recognized specific gesture made by the user; selecting at least one virtual clothes belonging to a specific category corresponding to the determined specific portion; and controlling the display to display the selected at least one virtual clothes thereon, wherein the virtual clothes are stored in the memory. 12. The device of claim 11, wherein the specific category changes based on the specific portion. 13. The device of claim 12, wherein the controller is further configured for analyzing a gender and an age of the user using the camera. 14. The device of claim 13, wherein the specific category varies based on the gender and age of the user. 15. The device of claim 14, wherein the controller is further configured for: calculating a time duration for which the specific gesture is recognized around the specific portion using the camera; determining whether the calculated time duration exceeds a predefined threshold value; upon determination that the calculated time duration exceeds the predefined threshold value, controlling the display for display the at least one virtual clothes belonging to the specific category. 16. A multimedia device for providing a virtual fitting service, the multimedia device comprising: a memory; a camera for capturing a user around the multimedia device; a display configured to display the captured user or a corresponding virtual avatar thereto; and a controller configured for: recognizing a first gesture and a face of the user using the camera; determining whether a plurality of virtual clothes-related specific categories corresponding to the recognized first gesture are present in a memory; controlling the display to display a graphic image for guiding a second gesture to select a single specific category when the plurality of virtual clothes-related specific categories corresponding to the recognized first gesture are present in the memory, wherein the graphic image is displayed for a predefined period P 1, wherein the predefined period P 1 is determined based on the recognized face of the user; selecting the single specific category among the plurality of virtual clothes-related specific categories based on the second gesture recognized using the camera; selecting at least one virtual clothes belonging to the single specific category; and controlling the display to display the selected at least one virtual clothes thereon. 17. The device of claim 16, wherein the controller is further configured for displaying the graphic image only for the predefined period P 1 from a time T 1 at which the first gesture is recognized. 18. The device of claim 17, wherein the controller is further configured for: calculating a difference between a time T 2 at which the second gesture is recognized and a time T 1 at which the first gesture is recognized; and mapping the recognized user's face to the difference and storing the mapping in the memory. 19. The device of claim 18, wherein the controller is further configured for: re-recognizing a face of the user; and re-configuring the predefined period P 1 based on the user's face and the difference stored in the memory. 20. The device of claim 19, wherein the controller is further configured for: connecting the device to a robot over short range communication or 5G communication network; receiving, from the robot, an image of the user taken by the robot."
  ],
  "cpc": [
    "G06F 3/017",
    "G06F 3/005",
    "G06F 3/011",
    "G06F 3/0304",
    "G06F 3/0482",
    "G06F 3/04845",
    "G06K 9/00228",
    "G06K 9/00355",
    "G06K 9/00362",
    "G06T 1/0007",
    "G06T 13/40",
    "G06T 19/003",
    "G06T 19/20",
    "G06V 10/764",
    "G06V 10/82",
    "G06V 40/10",
    "G06V 40/161",
    "G06V 40/178",
    "G06V 40/28"
  ],
  "assignees": [
    "LG ELECTRONICS INC"
  ],
  "filing_date": "2019-08-27",
  "publication_date": "2021-02-25",
  "priority_date": "2019-08-23",
  "application_number": "US-201916552793-A",
  "family_id": "74645711"
}