Emotion detection in voicemail

{
  "publication_number": "US9569424B2",
  "country": "US",
  "kind": "B2",
  "title": "Emotion detection in voicemail",
  "abstract": "Methods and apparatus for processing a voicemail message to generate a textual representation of at least a portion of the voicemail message. At least one emotion expressed in the voicemail message is determined by applying at least one emotion classifier to the voicemail message and/or the textual representation. An indication of the determined at least one emotion is provided in a manner associated with the textual representation of the at least a portion of the voicemail message.",
  "claims": [
    "1. A method for use with a voicemail transcription system that processes a voicemail message and generates a textual representation of at least a portion of the voicemail message, the method comprising: determining based, at least in part, on the textual representation of the at least a portion of the voicemail message, at least one emotion expressed in the voicemail message, wherein the determining comprises applying at least one emotion classifier to the textual representation of the at least a portion of the voicemail message; storing preference information for a user of a client device configured to receive voicemail transcriptions, wherein the preference information describes preferences for how the user of the client device wants emotion information associated with the received voicemail transcriptions to be conveyed to the user on the client device; and providing on the client device, in accordance with the stored preference information, an indication of the determined at least one emotion prior to displaying the textual representation of the at least a portion of the voicemail message on the client device, wherein providing an indication of the determined at least one emotion comprises displaying on the client device at least one graphical symbol representing the determined at least one emotion with a truncated version of the textual representation.",
    "2. The method of claim 1, wherein the determining the at least one emotion further comprises determining the at least one emotion based, at least in part, on an analysis of audio of the voicemail message.",
    "3. The method of claim 1, wherein the providing the indication of the determined at least one emotion comprises: selecting at least one audio output based, at least in part, on the determined at least one emotion; and associating the at least one audio output with the textual representation of the at least a portion of the voicemail message.",
    "4. The method of claim 1, wherein the at least one graphical symbol comprises an icon separated from but associated with the truncated version of the textual representation.",
    "5. The method of claim 4, wherein providing the indication of the determined at least one emotion comprises displaying the icon in a particular color, wherein the particular color is determined based, at least in part, on the determined at least one emotion.",
    "6. The method of claim 1, wherein providing the indication of the determined at least one emotion comprises outputting a particular ringtone from client device, selecting a particular agent persona to communicate with the user of the client device, or providing a particular vibration output from the client device.",
    "7. A non-transitory computer-readable storage medium encoded with a plurality of instructions that, when executed by at least one processor, perform a method for use with a voicemail transcription system that processes a voicemail message and generates a textual representation of at least a portion of the voicemail message, the method comprising: determining based, at least in part, on the textual representation of the at least a portion of the voicemail message, at least one emotion expressed in the voicemail message, wherein the determining comprises applying at least one emotion classifier to the textual representation of the at least a portion of the voicemail message; storing preference information for a user of a client device configured to receive voicemail transcriptions, wherein the preference information describes preferences for how the user of the client device wants emotion information associated with the received voicemail transcriptions to be conveyed to the user on the client device; and providing on the client device, in accordance with the stored preference information, an indication of the determined at least one emotion prior to displaying the textual representation of the at least a portion of the voicemail message on the client device, wherein providing an indication of the determined at least one emotion comprises displaying on the client device at least one graphical symbol representing the determined at least one emotion with a truncated version of the textual representation.",
    "8. The non-transitory computer-readable storage medium of claim 7, wherein the determining the at least one emotion further comprises determining the at least one emotion based, at least in part, on an analysis of audio of the voicemail message.",
    "9. The non-transitory computer-readable storage medium of claim 7, wherein determining based, at least in part, on the textual representation of the at least a portion of the voicemail message, at least one emotion expressed in the voicemail message comprises comparing at least one word in the textual representation to a collection of stored words and/or n-grams associated with one or more emotions.",
    "10. The non-transitory computer-readable storage medium of claim 7, wherein the at least one emotion classifier is selected from a group consisting of: support vector machines (SVMs), a logistic regression classifier, a Naïve Bayesian classifier, a Gaussian mixture model (GMM), and a statistical language model.",
    "11. The non-transitory computer-readable storage medium of claim 7, wherein the providing the indication of the determined at least one emotion comprises: selecting at least one audio output based, at least in part, on the determined at least one emotion; and associating the at least one audio output with the textual representation of the at least a portion of the voicemail message.",
    "12. The non-transitory computer-readable storage medium of claim 7, wherein the at least one graphical symbol comprises an icon separated from but associated with the truncated version of the textual representation.",
    "13. The non-transitory computer-readable storage medium of claim 12, wherein providing the indication of the determined at least one emotion comprises displaying the icon in a particular color, wherein the particular color is determined based, at least in part, on the determined at least one emotion.",
    "14. A computer system for use with a voicemail transcription system that processes a voicemail message and generates a textual representation of at least a portion of the voicemail message, the computer system comprising: at least one storage device configured to store: at least one emotion classifier for determining at least one emotion expressed in the voicemail message; and preference information for a user of a client device configured to receive voicemail transcriptions from the voicemail transcription system, wherein the preference information describes preferences for how the user of the client device wants emotion information associated with the received voicemail transcriptions to be conveyed to the user on the client device; and at least one processor programmed to: determine the at least one emotion, wherein the determining comprises applying the at least one emotion classifier to the textual representation of at least a portion of the voicemail message; and provide on the client device, in accordance with the stored preference information, an indication of the determined at least one emotion prior to displaying the textual representation of the at least a portion of the voicemail message on the client device, wherein providing an indication of the determined at least one emotion comprises displaying on the client device at least one graphical symbol representing the determined at least one emotion with a truncated version of the textual representation.",
    "15. The computer system of claim 14, wherein the determining the at least one emotion further comprises determining the at least one emotion based, at least in part, on an analysis of audio of the voicemail message.",
    "16. The computer system of claim 14, wherein the providing the indication of the determined at least one emotion comprises: selecting at least one audio output based, at least in part, on the determined at least one emotion; and associating the at least one audio output with the textual representation of the at least a portion of the voicemail message.",
    "17. The computer system of claim 14, wherein the at least one graphical symbol comprises an icon having a particular color determined based, at least in part, on the determined at least one emotion.",
    "18. The computer system of claim 14, wherein the at least one graphical symbol comprises an icon separated from but associated with the truncated version of the textual representation.",
    "19. The computer system of claim 18, wherein providing the indication of the determined at least one emotion comprises displaying the icon in a particular color, wherein the particular color is determined based, at least in part, on the determined at least one emotion."
  ],
  "cpc": [
    "G10L 25/63",
    "G06F 17/2785",
    "G06F 40/30",
    "G10L 15/187",
    "G10L 15/26",
    "H04M 2201/60",
    "H04M 3/5335"
  ],
  "assignees": [
    "NUANCE COMMUNICATIONS INC"
  ],
  "filing_date": "2013-02-21",
  "publication_date": "2017-02-14",
  "grant_date": "2017-02-14",
  "priority_date": "2013-02-21",
  "application_number": "US-201313773348-A",
  "family_id": "51351899",
  "citations": [
    "US2002194002A1",
    "US2003177008A1",
    "US2005108775A1",
    "US2005159939A1",
    "US2005244798A1",
    "US2006234680A1",
    "US2007037590A1",
    "US2007054678A1",
    "US2008027984A1",
    "US2008056470A1",
    "US2008059158A1",
    "US2010085416A1",
    "US2010158213A1",
    "US2010246799A1",
    "US2011010173A1",
    "US2011087483A1",
    "US2011200181A1",
    "US2011208522A1",
    "US2011295607A1",
    "US2012179751A1",
    "US2013060875A1",
    "US2013268611A1",
    "US2013297297A1",
    "US2014052441A1",
    "US2014112556A1",
    "US2014192229A1",
    "US2014212007A1",
    "US2014229175A1",
    "US8170872B2",
    "US9015046B2",
    "WO2012081889A1"
  ]
}