Electronic device and method for delivering message by to recipient based on emotion of sender

{
  "publication_number": "US11165728B2",
  "country": "US",
  "kind": "B2",
  "title": "Electronic device and method for delivering message by to recipient based on emotion of sender",
  "abstract": "Various embodiments of the present disclosure relate to an electronic device and a method for delivering a message by the same. The electronic device may comprise: a camera; at least one processor functionally connected to the camera; a driving circuitry configured to control moving of the electronic device; and a memory for storing at least one program configured to be executable by the at least one processor, wherein the program comprises instructions that are set to: in response to an input of a message from a sender, activate the camera and capture an image of the sender, analyze the image of the sender for an emotion of the sender, analyze the inputted message for a recipient, determine an output pattern based on the image, determine the recipient based on the inputted message, determine whether the determined recipient is located in a vicinity of the electronic device, and in response to the determined recipient not being located in the vicinity of the electronic device, identify a location of the determined recipient and control the driving circuitry to move the electronic device to the identified location.",
  "claims": [
    "1. An electronic device comprising: a camera; at least one processor functionally connected with the camera; a driving circuitry configured to control moving of the electronic device; and a memory configured to store at least one program configured to be executable by the at least one processor, wherein the at least one program comprises instructions that are set to: in response to an input of a message from a sender being detected, activate the camera and capture an image of the sender, analyze the image of the sender for extracting information related to an emotion of the sender, analyze the inputted message for extracting information related to a recipient, determine an output pattern based on a result of the analysis of the image of the sender, determine the recipient based on a result of the analysis of the inputted message, determine whether the determined recipient is located within a specified range of the electronic device, in response to the determined recipient being located in the specified range of the electronic device, deliver the inputted message to the determined recipient based on the determined output pattern, and in response to the determined recipient not being located in the specified range of the electronic device, identify a location of the determined recipient and control the driving circuitry to move the electronic device to the identified location of the determined recipient.",
    "2. The electronic device of claim 1, wherein the at least one program further comprises instructions that are set to: track a pupil of the determined recipient in response to the determined recipient being located in the specified range of the electronic device, and in response to a result of the tracking that the pupil of the determined recipient gazes at the electronic device, deliver the inputted message to the determined recipient.",
    "3. The electronic device of claim 1, wherein the at least one program further comprises instructions that are set to estimate the location of the determined recipient based on whether at least one other electronic device registered at a home network to which the electronic device belongs is used.",
    "4. The electronic device of claim 1, wherein the at least one program further comprises instructions that are set to determine whether the determined recipient is located in the specified range of the electronic device by identifying the determined recipient through speaker recognition in response to a sound of a predetermined level or higher being detected through a microphone, by identifying the determined recipient through face recognition in response to a subject being detected through the camera, or, in response to an external electronic device being connected through short-range wireless communication, by identifying whether the external electronic device connected through the short-range wireless communication is an electronic device of the determined recipient.",
    "5. The electronic device of claim 1, wherein the at least one program further comprises instructions that are set to analyze at least one of sender information, and an intensity of the emotion of the sender, an event related to the determined recipient, a preference of the determined recipient, and whether the inputted message is required to be kept secret.",
    "6. The electronic device of claim 5, wherein the at least one program further comprises instructions that are set to: in response to the inputted message being required to be kept secret, determine whether at least one other user is located proximate to the determined recipient, and in response to the at least one other user not being located proximate to the determined recipient, deliver the inputted message to the determined recipient.",
    "7. The electronic device of claim 1, further comprising a communication circuitry, wherein the at least one program further comprises instructions that are set to: identify whether there is an electronic device owned by the determined recipient, and in response to the electronic device being owned by the determined recipient, control the communication circuitry to deliver the output pattern to the electronic device owned by the determined recipient.",
    "8. The electronic device of claim 1, further comprising a communication circuitry, wherein the at least one program further comprises instructions that are set to: deliver the image of the sender and the inputted message to an emotion analysis server through the communication circuitry, request the emotion analysis server to analyze the emotion of the sender and an intensity of the emotion of the sender, and receive a result of the analysis.",
    "9. The electronic device of claim 1, wherein the output pattern comprises at least one of a face expression, a motion, an AUI, or a voice type.",
    "10. A method for delivering a message in an electronic device, the method comprising: in response to an input of a message from a sender being detected, capturing, using a camera of the electronic device, an image of the sender; analyzing the image of the sender for extracting information related to an emotion of the sender; analyzing the inputted message for extracting information related to a recipient; determining an output pattern based on a result of analyzing the image of the sender; determining the recipient based on a result of analyzing the inputted message; determining whether the determined recipient is located in a specified range of the electronic device; in response to the determined recipient being located in the specified range of the electronic device, delivering the inputted message to the determined recipient based on the determined output pattern; and in response to the determined recipient not being located in the specified range of the electronic device, identifying a location of the determined recipient, and moving, using a driving circuitry of the electronic device, the electronic device to the identified location of the determined recipient.",
    "11. The method of claim 10, wherein the determining of whether the determined recipient is located in the specified range of the electronic device comprises at least one of: identifying the determined recipient through speaker recognition in response to a sound of a predetermined level or higher being detected through a microphone; identifying the determined recipient through face recognition in response to a subject being detected through the camera; or in response to an external electronic device being connected through short-range wireless communication, identifying whether the external electronic device connected through the short-range wireless communication is an external electronic device owned by the determined recipient.",
    "12. The method of claim 10, wherein analyzing comprises at least one of: analyzing sender information; analyzing an intensity of the emotion of the sender; analyzing an event related to the determined recipient; analyzing a preference of the determined recipient; or analyzing whether the inputted message is required to be kept secret.",
    "13. The method of claim 12, wherein delivering the inputted message to the determined recipient comprises: in response to the inputted message being required to be kept secret, determining whether at least one other user is located proximate to the determined recipient; and in response to the at least one other user not being located proximate to the determined recipient, delivering the inputted message to the determined recipient.",
    "14. The method of claim 12, wherein analyzing the at least one of the emotion of the sender or the intensity of the emotion of the sender is performed by an emotion analysis engine in the electronic device or an emotion analysis server.",
    "15. The method of claim 10, wherein the delivering of inputted message comprises: recognizing a face of the determined recipient; tracking a pupil of the determined recipient; and in response to a result of the tracking that the pupil of the determined recipient gazes at the electronic device, delivering the inputted message to the determined recipient.",
    "16. The method of claim 10, wherein identifying the location of the determined recipient comprises: estimating the location of the determined recipient based on whether at least one other electronic device registered at a home network to which the electronic device belongs is used.",
    "17. The method of claim 10, further comprising: identifying, by the electronic device, whether there is an electronic device owned by the determined recipient (recipient-owned electronic device); in response to the electronic device being recipient-owned, delivering the output pattern to the recipient-owned electronic device; and delivering, by the recipient-owned electronic device, the inputted message to the determined recipient according to the output pattern.",
    "18. The method of claim 10, wherein determining the output pattern comprises: determining at least one of a face expression, a motion, a sound effect, an auditory user interface (AUI), or a voice type.",
    "19. The method of claim 17, wherein the delivering of the inputted message comprises causing a movement of the recipient-owned device to convey a gesture of the sender.",
    "20. The method of claim 10, where the specified range of the electronic device comprises within short-range wireless communication range."
  ],
  "description_excerpt": "Various embodiments of the disclosure relate to an electronic device and a method for delivering a message by the same.\n\nElectronic devices (for example, mobile terminals, smartphones, wearable electronic devices, or the like) may provide various functions. For example, a smartphone may provide an Internet access function, a music or video replay function, a capturing function, a navigation function, a messenger function, etc. in addition to a basic voice communication function. The messenger function enables messages in a text form to be exchanged with a specific user. However, a recent messenger function does not only exchange a simple text, but also transmits multimedia messages. For example, the electronic devices are able to transmit and receive emoticons, recorded files, photos and/or video files.\n\nTechnical Problem However, the electronic devices simply transmit a multimedia message including a recorded file, a photo, or a video file to an electronic device of a designated reception side. The electronic device of the reception side may display the received multimedia message on a screen regardless of whether there is a recipient in the vicinity of the electronic device. In addition, since the electronic device simply transmits the multimedia message, the electronic device may not appropriately convey an emotion of a sender to the recipient. To solve the above-described problems, various embodiments of the disclosure may receive a message that is to be delivered to a specific recipient from a sender, and may deliver the corresponding message to the recipient when the recipient is recognized.",
  "cpc": [
    "H04L 51/10",
    "B25J 5/007",
    "B25J 9/0003",
    "G06K 9/00288",
    "G06K 9/00302",
    "G06V 40/172",
    "G06V 40/174",
    "G06V 40/70",
    "G10L 25/63",
    "H04L 51/18",
    "H04L 51/222",
    "H04M 1/72415",
    "H04M 1/725"
  ],
  "ipc": [
    "B25J 5/00",
    "B25J 9/00",
    "G06V 40/70",
    "G10L 25/63",
    "H04L 12/58",
    "H04M 1/725"
  ],
  "assignees": [
    "SAMSUNG ELECTRONICS CO LTD"
  ],
  "inventors": [
    "LEE IM-SUNG",
    "KIM KI-WON",
    "OH SAE GEE",
    "GOO JA-MIN",
    "KWON KIYOUNG",
    "KIM EUN-YOUNG",
    "KIM JIMIN",
    "KIM CHULKWI",
    "KIM HYUNG-WOO",
    "NAMKUNG JOO",
    "PARK JI-HYUN",
    "YOU SEONG-HOON",
    "YU YONG-JU",
    "LEE DONGKYU",
    "LEE CHANWON",
    "JANG SI-HAK"
  ],
  "filing_date": "2017-12-21",
  "publication_date": "2021-11-02",
  "grant_date": "2021-11-02",
  "priority_date": "2016-12-27",
  "application_number": "US-201716474386-A",
  "family_id": "62710911",
  "citations": [
    "CN101686442A",
    "CN102780651A",
    "JP2001016514A",
    "KR20090105198A",
    "KR20140106891A",
    "KR20150059466A",
    "US10008196B2",
    "US10133918B1",
    "US10168859B2",
    "US10191920B1",
    "US10305850B1",
    "US10357881B2",
    "US10365788B2",
    "US10372293B2",
    "US10391636B2",
    "US10486312B2",
    "US10594638B2",
    "US10596708B2",
    "US10699104B2",
    "US10706271B2",
    "US10708203B2",
    "US10776584B2",
    "US2003097411A1",
    "US2005091684A1",
    "US2007135119A1",
    "US2008077277A1",
    "US2009110246A1",
    "US2010057875A1",
    "US2010086204A1",
    "US2012124456A1",
    "US2012233633A1",
    "US2012280951A1",
    "US2012316676A1",
    "US2013218339A1",
    "US2014157153A1",
    "US2015146925A1",
    "US2015206000A1",
    "US2015295867A1",
    "US2015314454A1",
    "US2015355468A1",
    "US2015381534A1",
    "US2016193732A1",
    "US2016199977A1",
    "US2016241500A1",
    "US2016328015A1",
    "US2017024087A1",
    "US2017364484A1",
    "US2018027307A1",
    "US2018077095A1",
    "US2018133900A1",
    "US2018229372A1",
    "US2018331989A1",
    "US2018331990A1",
    "US2019079922A1",
    "US2019126157A1",
    "US2019190865A1",
    "US2020014644A1",
    "US2020153772A1",
    "US7720572B2",
    "US7957837B2",
    "US8160549B2",
    "US8195333B2",
    "US8265793B2",
    "US8583282B2",
    "US8935006B2",
    "US9043196B1",
    "US9446510B2",
    "US9452525B2",
    "US9577963B2",
    "US9796078B2",
    "US9818228B2",
    "US9878445B2",
    "US9996217B2",
    "WO2015158876A1"
  ]
}