System and method for identifying the context of multimedia content elements

{
  "publication_number": "US11032017B2",
  "country": "US",
  "kind": "B2",
  "title": "System and method for identifying the context of multimedia content elements",
  "abstract": "A method and system for determining a current context of a multimedia content element are provided. The method includes receiving at least one multimedia content element from a user device; receiving at least one environmental variable related to the at least one multimedia content element; generating at least one signature for the multimedia content element; determining a context of the at least one multimedia content element based on the at least one contextual parameter; and determining the current context of the at least one multimedia content element based on at least one contextual parameter and the determined context.",
  "claims": [
    "1. A method for determining a current context of a multimedia content element, comprising: receiving at least one multimedia content element from a user device; receiving at least one environmental variable related to the at least one multimedia content element; generating signatures for the multimedia content element, wherein each signature is generated by a signature generator including a plurality of independent computational cores; wherein the plurality of independent computational cores comprise a plurality of neural networks, one neural network per independent computational core; and wherein each signature comprises a plurality of maps of firing neurons of the plurality neural network that were fed with the at least one multimedia content element; determining, based on comparing the received at least one environmental variable to a plurality of concepts, at least one contextual parameter; determining a first context of the at least one multimedia content element based on the generated signatures; and determining a current second context of the at least one multimedia content element based on the at least one contextual parameter and the determined first context, wherein the at least one contextual parameter is added as another layer to the determined first context to define the current second context.",
    "2. The method of claim 1, wherein determining a current second context defines a current relevancy of the first context with respect to the at least one environmental variable.",
    "3. The method of claim 1, wherein the at least one environmental variable is any one of: a time, a location of the user device, motion information related to the user device, and weather information.",
    "4. The method of claim 1, wherein each of the signatures represents a concept.",
    "5. The method of claim 4, further comprising: correlating concepts represented by the signatures to determine the first context of the multimedia content element, wherein each concept is a collection of signatures and metadata describing the concept; wherein the first context is represented as a matrix of co-occurrences of the concepts represented by the signatures.",
    "6. The method according to claim 5, wherein the correlating comprises calculating a correlation between signatures representing the concepts by utilizing probabilistic models to determine a probability that a signature appears in a same orientation and in a same ratio as another signature.",
    "7. The method of claim 4, comprising applying a threshold on a number of co-occurrences of concepts in the matrix to establish the context.",
    "8. The method of claim 1, wherein the at least one signature is robust to noise and distortion.",
    "9. The method of claim 1, wherein the multimedia content element is at least one of: an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, and portions thereof.",
    "10. The method of claim 1, wherein the at least one environmental variable is weather information.",
    "11. A non-transitory computer readable medium having stored thereon instructions for causing a processing system to execute the steps of: receiving at least one multimedia content element from a user device; receiving at least one environmental variable related to the at least one multimedia content element; generating signatures for the multimedia content element, wherein each signature is generated by a signature generator including a plurality of independent computational cores; wherein the plurality of independent computational cores comprise a plurality of neural networks, one neural network per independent computational core; and wherein each signature comprises a plurality of maps of firing neurons of the plurality neural network that were fed with the at least one multimedia content element; determining, based on comparing the received at least one environmental variable to a plurality of concepts, at least one contextual parameter; determining a first context of the at least one multimedia content element based on the generated signatures; and determining a current second context of the at least one multimedia content element based on the at least one contextual parameter and the determined first context, wherein the at least one contextual parameter is added as another layer to the determined first context to define the current second context.",
    "12. A system for determining a current context of a multimedia content element, comprising: a signature generator, wherein the signature generator includes a plurality of independent computational cores; wherein the plurality of independent computational cores comprise a plurality of neural networks, one neural network per independent computational core; and wherein each signature comprises a plurality of maps of firing neurons of the plurality neural network that were fed with the at least one multimedia content element; a processing system; a memory connected to the processing system, the memory comprising instructions that when executed by the processing system, configure the system to: generate signatures for the at least one multimedia content element, wherein each signature is generated by the signature generator; determine, based on the received at least one environmental variable, at least one contextual parameter; determine a first context of the at least one multimedia content element based on the signatures; and determine a current second context of the at least one multimedia content element based on the at least one contextual parameter and the determined first context, wherein the at least one contextual parameter is added as another layer to the determined first context to define the current second context.",
    "13. The system of claim 12, wherein the current second context defines a current relevancy of the first context with respect to the at least one environmental variable.",
    "14. The system of claim 12, wherein the at least one environmental variable is any one of: a time, a location of the user device, motion information related to the user device, and weather information.",
    "15. The system of claim 12, wherein each of the signatures represents a concept.",
    "16. The system of claim 15, wherein the system is further configured to: correlate concepts represented by the signatures to determine the first context of the multimedia content element; wherein the first context is represented as a matrix of co-occurrences of the concepts represented by the signatures.",
    "17. The system of claim 12, wherein the system is further configured to: applying a threshold on a number of co-occurrences of concepts in the matrix to establish the context.",
    "18. The system of claim 12, wherein the at least one signature is robust to noise and distortion.",
    "19. The system of claim 12, wherein the multimedia content element is at least one of: an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, and portions thereof."
  ],
  "description_excerpt": "The present invention relates generally to the analysis of multimedia content, and more specifically to a system for identifying a context of the multimedia content.\n\nIn the related art there are different techniques for identifying the context of multimedia content. One optional technique for determining a context of multimedia content (e.g., an image) is based on metadata data associated with the multimedia content. The metadata is typically associated with a multimedia content element and includes parameters such as the element's size, type, name, a short description, and so on. Attributes of the metadata may be mapped to various context categories, such as sports, friends, pets, and so on. For example, if a metadata attribute would include the name of the basketball player “Kobe Bryant” the respective image would be mapped to a sports context.\n\nHowever, the attributes of metadata associated with multimedia content of the element are typically provided by the creator of the multimedia content or by a person saving or placing the multimedia content in a local device and/or a website. Therefore, the metadata, in most cases, is not sufficiently descriptive of the multimedia element. For example, a user may save a picture featuring a cat under the file name of “weekend fun.” In such an example, the metadata would not be descriptive of the contents of the picture.\n\nFurthermore, techniques for determining the context of multimedia content based on their metadata or other textual information associated with the content cannot provide the current context of the multimedia content.",
  "cpc": [
    "H04H 60/40",
    "G06N 3/02",
    "H04H 2201/90",
    "H04H 60/37",
    "H04H 60/52",
    "H04H 60/56",
    "H04H 60/71",
    "H04N 21/25891",
    "H04N 21/2668",
    "H04N 21/466",
    "H04N 21/8106",
    "H04N 7/17318"
  ],
  "ipc": [
    "G06F 17/30",
    "G06N 3/02",
    "H04H 60/37",
    "H04H 60/40",
    "H04H 60/52",
    "H04H 60/56",
    "H04H 60/71",
    "H04N 21/258",
    "H04N 21/2668",
    "H04N 21/466",
    "H04N 21/81",
    "H04N 7/173"
  ],
  "assignees": [
    "Cortica Ltd"
  ],
  "inventors": [
    "Igal RAICHELGAUZ",
    "Karina ODINAEV",
    "Yehoshua Y. Zeevi"
  ],
  "filing_date": "2014-10-08",
  "publication_date": "2021-06-08",
  "grant_date": "2021-06-08",
  "priority_date": "2005-10-26",
  "application_number": "US-201414509552-A",
  "family_id": "52344453",
  "cited_by_count": 4,
  "citations": [
    "US3036893A",
    "US4733353A",
    "US4972363A",
    "US4932645A",
    "US5214746A",
    "US5436653A",
    "US5307451A",
    "US5638425A",
    "US5887193A",
    "US20010038876A1",
    "US5568181A",
    "US5835901A",
    "US5412564A",
    "US6052481A",
    "US5763069A",
    "US5745678A",
    "USRE44225E1",
    "US5978754A",
    "US6985172B1",
    "US6618711B1",
    "US6076088A",
    "US5852435A",
    "US5870754A",
    "US5991306A",
    "US5873080A",
    "US6243375B1",
    "US5806061A",
    "US6038560A",
    "US5940821A",
    "US5987454A",
    "US6523022B1",
    "US6137911A",
    "US6877134B1",
    "US6507672B1",
    "US6070167A",
    "US6594699B1",
    "US6122628A",
    "US20030184598A1",
    "US6329986B1",
    "US6144767A",
    "US6240423B1",
    "US6640015B1",
    "US6163510A",
    "US6681032B2",
    "US6243713B1",
    "US6275599B1",
    "US6526400B1",
    "US6493705B1",
    "US6493692B1",
    "US6363373B1",
    "US8332478B2",
    "US7158681B2",
    "US7313805B1",
    "US6524861B1",
    "US7199798B1",
    "US6611628B1",
    "US7013051B2",
    "US7302117B2",
    "US6643643B1",
    "US6819797B1",
    "US6381656B1",
    "US6601060B1",
    "US6774917B1",
    "US6643620B1",
    "US6557042B1",
    "US6732149B1",
    "US6128651A",
    "US6763519B1",
    "US20070195987A1",
    "US6754435B2",
    "US6314419B1",
    "US7340458B2",
    "US20020159640A1",
    "US6704725B1",
    "US6795818B1",
    "US6813395B1",
    "US7376722B1",
    "US6422617B1",
    "US6751363B1",
    "US6147636A",
    "US6751613B1",
    "US20030078766A1",
    "US6601026B2",
    "EP1085464A2",
    "US20030182567A1",
    "US20060248558A1",
    "US6665657B1",
    "US20010056427A1",
    "US20010019633A1",
    "US7047033B2",
    "US7920894B2",
    "US20100106857A1",
    "US6550018B1",
    "US20020161739A1",
    "US20070022374A1",
    "US6523046B2",
    "US20020032677A1",
    "US6804356B1",
    "US20020038299A1",
    "US6560597B1",
    "US20030093790A1",
    "US6901207B1",
    "US7260564B1",
    "US20020072935A1",
    "US6411229B2",
    "US7987217B2",
    "US6836776B2",
    "US6611837B2",
    "US20020019881A1",
    "US6763069B1",
    "US20020147637A1",
    "US7660737B1",
    "US20020010682A1",
    "US20020059580A1",
    "US6675159B1",
    "US20020157116A1",
    "US7464086B2",
    "US7392238B1",
    "US7904503B2",
    "US20020037010A1",
    "US20020042914A1",
    "WO2002031764A2",
    "US20020107827A1",
    "US20040133927A1",
    "US7043473B1",
    "US6845374B1",
    "US20030005432A1",
    "US20020099870A1",
    "US7006689B2",
    "US20040128511A1",
    "US20090089587A1",
    "US20020126872A1",
    "US20020152267A1",
    "US7277928B2",
    "US20020087530A1",
    "US20060173688A1",
    "US20040128142A1",
    "US20020123928A1",
    "US6742094B2",
    "US20020178410A1",
    "US20020113812A1",
    "US20020129296A1",
    "US20020143976A1",
    "US20020129140A1",
    "US7577656B2",
    "US7836054B2",
    "WO2003005242A1",
    "US6728706B2",
    "US20020163532A1",
    "US20030050815A1",
    "US20030037010A1",
    "US20040025180A1",
    "US7933407B2",
    "US20020184505A1",
    "US20020174095A1",
    "US6970881B1",
    "US6938025B1",
    "US20040230572A1",
    "US20020010715A1",
    "US20030028660A1",
    "US20030041047A1",
    "US7376672B2",
    "US20030089216A1",
    "US20040249779A1",
    "US20030086627A1",
    "US20030105739A1",
    "US20030126147A1",
    "US20030101150A1",
    "US20040215663A1",
    "US7353224B2",
    "US7020654B1",
    "US20030115191A1",
    "US6978264B2",
    "US20060120626A1",
    "US20050021394A1",
    "WO2003067467A1",
    "US20050144455A1",
    "US7215828B2",
    "US20040098671A1",
    "US20040095376A1",
    "US8635531B2",
    "US7023979B1",
    "US20040153426A1",
    "US20050172130A1",
    "US20030200217A1",
    "US20050177372A1",
    "US20030217335A1",
    "US20030229531A1",
    "US20040003394A1",
    "US20040091111A1",
    "US20050238238A1",
    "US20050281439A1",
    "US20030191764A1",
    "US20060129822A1",
    "WO2004019527A1",
    "US7296012B2",
    "US20040059736A1",
    "US20040068510A1",
    "US20060217818A1",
    "US20060217828A1",
    "US20060026203A1",
    "US20060013451A1",
    "US20060100987A1",
    "US20040107181A1",
    "US20040117638A1",
    "US20040111465A1",
    "US20040111432A1",
    "US20040117367A1",
    "US7124149B2",
    "US20060288002A1",
    "US7299261B1",
    "US7694318B2",
    "US20060184638A1",
    "US7406459B2",
    "US20120185445A1",
    "US20070033163A1",
    "US20040260688A1",
    "US20040267774A1",
    "US7689544B2",
    "US20060153296A1",
    "WO2005027457A1",
    "US20060143674A1",
    "US20110125727A1",
    "US20060033163A1",
    "US7346629B2",
    "US7574668B2",
    "US7562076B2",
    "US20070071330A1",
    "US20050114198A1",
    "US20050131884A1",
    "US20070168413A1",
    "US20090278934A1",
    "US20080228995A1",
    "US20050163375A1",
    "US7548910B1",
    "US20050193015A1",
    "US20080072256A1",
    "US20080204706A1",
    "US20060224529A1",
    "US20110035289A1",
    "US20050249398A1",
    "US20050238198A1",
    "US20050245241A1",
    "US7697791B1",
    "US20050256820A1",
    "US20080237359A1",
    "US20050262428A1",
    "US20080270373A1",
    "US20050289590A1",
    "US20090043637A1",
    "US20050289163A1",
    "US20060004745A1",
    "US7523102B2",
    "US20080201299A1",
    "US20060020860A1",
    "US20060020958A1",
    "US20060031216A1",
    "US20070298152A1",
    "US20060041596A1",
    "US20060218191A1",
    "US20060048191A1",
    "US20060064037A1",
    "US7526607B1",
    "US20090253583A1",
    "US20060112035A1",
    "US7542969B1",
    "US20090157575A1",
    "US20060204035A1",
    "US7340358B2",
    "US20060159442A1",
    "US20080166020A1",
    "US8316005B2",
    "US20130104251A1",
    "US7574436B2",
    "US7769221B1",
    "US20120221470A1",
    "US20060236343A1",
    "US20060242139A1",
    "US20060242130A1",
    "US20060242554A1",
    "US20060247983A1",
    "US20060253423A1",
    "US7657100B2",
    "US20080152231A1",
    "US20060251339A1",
    "US8345982B2",
    "US7660468B2",
    "US8315442B2",
    "US20070009159A1",
    "US20070011151A1",
    "US7433895B2",
    "US7788132B2",
    "US20070130112A1",
    "US20070019864A1",
    "US20070038614A1",
    "US20070038608A1",
    "US20070042757A1",
    "US7860895B1",
    "US20070067682A1",
    "US20080019614A1",
    "US7519238B2",
    "US20070156720A1",
    "US8677377B2",
    "US20070061302A1",
    "US20070067304A1",
    "US8023739B2",
    "US7450740B2",
    "US20070074147A1",
    "US7529659B2",
    "US7801893B2",
    "US20070083611A1",
    "US20080263579A1",
    "US20070091106A1",
    "US9235557B2",
    "US8326775B2",
    "US8112376B2",
    "US8818916B2",
    "US20170262437A1",
    "US8655801B2",
    "US20100042646A1",
    "US20170255620A1",
    "US9646006B2",
    "US20120331011A1",
    "US8880539B2",
    "US8880566B2",
    "US20140330830A1",
    "US8959037B2",
    "US9466068B2",
    "US8990125B2",
    "US9009086B2",
    "US20090216761A1",
    "US9031999B2",
    "US20150154189A1",
    "US20160239566A1",
    "US9087049B2",
    "US20090043818A1",
    "US8799196B2",
    "WO2007049282A2",
    "US9104747B2",
    "US20090282218A1",
    "US9384196B2",
    "US9191626B2",
    "US20130080433A1",
    "US8266185B2",
    "US9218606B2",
    "US9330189B2",
    "US8312031B2",
    "US8799195B2",
    "US9256668B2",
    "US8868619B2",
    "US20140188786A1",
    "US20070130159A1",
    "US20070174320A1",
    "US20070196013A1",
    "US20070220573A1",
    "US20070244902A1",
    "US20090204511A1",
    "US20070253594A1",
    "US20070255785A1",
    "US20070268309A1",
    "US7536417B2",
    "US20070282826A1",
    "US20070294295A1",
    "US8098934B2",
    "US20100023400A1",
    "US20080040277A1",
    "US20080046406A1",
    "US20080049789A1",
    "US20080049629A1",
    "US7536384B2",
    "US20080079729A1",
    "US20080091527A1",
    "US20140310825A1",
    "US20080109433A1",
    "US20100325581A1",
    "US20090034791A1",
    "US7555478B2",
    "US7555477B2",
    "US20080159622A1",
    "US20080165861A1",
    "US20080163288A1",
    "US20090245603A1",
    "US20090254572A1",
    "US20080172615A1",
    "US7788247B2",
    "US20110296315A1",
    "US20080201361A1",
    "US20080201314A1",
    "US20080253737A1",
    "US7921107B2",
    "US20090220138A1",
    "US20080270569A1",
    "US7974994B2",
    "US20080294278A1",
    "US20080307454A1",
    "US20080313140A1",
    "US20090013414A1",
    "US20090175538A1",
    "US20090022472A1",
    "US20100104184A1",
    "US20090024641A1",
    "US20090037408A1",
    "US8275764B2",
    "US20110145068A1",
    "US20090080759A1",
    "US20090119157A1",
    "US7987194B1",
    "US20090125544A1",
    "US20090125529A1",
    "US20090148045A1",
    "US9984369B2",
    "US20130066856A1",
    "US20160026707A1",
    "US20090172030A1",
    "US7991715B2",
    "US20100088321A1",
    "US20110164810A1",
    "US20090208106A1",
    "US8688446B2",
    "US20090216639A1",
    "US20090245573A1",
    "US8527978B1",
    "US20090254824A1",
    "US20090259687A1",
    "US20110038545A1",
    "US20120179642A1",
    "US20090277322A1",
    "US20090297048A1",
    "US9323754B2",
    "US20150254344A1",
    "US20110055585A1",
    "US20110313856A1",
    "US8898568B2",
    "US20100082684A1",
    "US20100111408A1",
    "US20100125569A1",
    "US8000655B2",
    "US20100162405A1",
    "US20100173269A1",
    "US20100191567A1",
    "US20100198626A1",
    "US8868861B2",
    "US20100268524A1",
    "US20100284604A1",
    "US20100306193A1",
    "US20100312736A1",
    "US20100318493A1",
    "US20100322522A1",
    "US20100325138A1",
    "US20110029620A1",
    "US20110052063A1",
    "US20130332951A1",
    "US20120191686A1",
    "US20110106782A1",
    "US20110164180A1",
    "US8706503B2",
    "US20110202848A1",
    "US20110208822A1",
    "US8682667B2",
    "US20110218946A1",
    "US20120239694A1",
    "US20110246566A1",
    "US20110251896A1",
    "US8655878B1",
    "US20110276680A1",
    "US20120082362A1",
    "US8781152B2",
    "US8990199B1",
    "US8634980B1",
    "US20160007083A1",
    "US20120131454A1",
    "US20120133497A1",
    "US20120150890A1",
    "US20120167133A1",
    "US20120179751A1",
    "US20120197857A1",
    "US20120227074A1",
    "US20120239690A1",
    "US20120301105A1",
    "US20140201330A1",
    "US20140111647A1",
    "US20120299961A1",
    "US8782077B1",
    "US20120330869A1",
    "US9807442B2",
    "US20130031489A1",
    "US20130067035A1",
    "US20130067364A1",
    "US9606992B2",
    "US20130086499A1",
    "US20130089248A1",
    "US20140300722A1",
    "US20130103814A1",
    "US20130159298A1",
    "US20130173635A1",
    "US8886648B1",
    "US20130212493A1",
    "US20130226820A1",
    "US20130226930A1",
    "US8457827B1",
    "US20130283401A1",
    "US20140019264A1",
    "US8548828B1",
    "US8775442B2",
    "US8495489B1",
    "US20130311924A1",
    "US20130325550A1",
    "US20140025692A1",
    "US20140059443A1",
    "US9165406B1",
    "US20140095425A1",
    "US20150289022A1",
    "WO2014076002A1",
    "US20150324356A1",
    "US20140147829A1",
    "US20140152698A1",
    "US20140169681A1",
    "US20140176604A1",
    "US20140193077A1",
    "US9197244B2",
    "US9438270B2",
    "US8922414B2",
    "US20180204111A1",
    "US20140250032A1",
    "US9298763B1",
    "WO2014137337A1",
    "US20140282655A1",
    "US20140341476A1",
    "US20140379477A1",
    "US20150033150A1",
    "US9679062B2",
    "US20150100562A1",
    "US20150117784A1",
    "US20150120627A1",
    "US20150134688A1",
    "US20160221592A1",
    "US9875445B2",
    "US20150286742A1",
    "US20150363644A1",
    "US20170109602A1",
    "WO2016040376A1",
    "US9440647B1",
    "US20170323568A1",
    "WO2016070193A1",
    "US9734533B1",
    "US20160210525A1",
    "US10133947B2",
    "US20160306798A1",
    "US20180081368A1",
    "US20160342683A1",
    "US20160357188A1",
    "US20170017638A1",
    "US20170032257A1",
    "US20170041254A1",
    "US20170154241A1",
    "US20190039627A1",
    "US20190043274A1",
    "US20180158323A1",
    "US10347122B2",
    "US20180101177A1",
    "US20180157916A1",
    "US20190364492A1",
    "US20200004248A1",
    "US20190005726A1",
    "US20190384312A1",
    "US20190056718A1",
    "US20190065951A1",
    "US20190188501A1",
    "US20190045244A1",
    "US20190220011A1",
    "US20200020212A1",
    "US20190317513A1",
    "US10491885B1",
    "US20190385460A1",
    "US20190384303A1",
    "US20190389459A1",
    "US20200004265A1",
    "US20200005631A1",
    "US20200004251A1",
    "US20200018606A1",
    "US20200018618A1",
    "US20200043326A1",
    "US20200133307A1",
    "US20200050973A1",
    "US20200073977A1",
    "US20200090484A1",
    "US20200097756A1"
  ]
}