Virtual Reality

{
  "publication_number": "US2019102928A1",
  "country": "US",
  "kind": "A1",
  "title": "Virtual Reality",
  "abstract": "A virtual reality apparatus is provided, which includes an image generator configured to generate images representing a user of a virtual environment. The image generator is responsive to selection of a predetermined input by the user. The apparatus also includes a video camera for use in image-based tracking of a user by the image generator. The predetermined input selected by the user is a performance of a predetermined physical gesture in view of the video camera. The image generator is arranged to generate a static or animated emote sticker responsive to the predetermined physical gesture of the user for display to other users.",
  "claims": [
    "1. A virtual reality apparatus comprising: an image generator configured to generate images representing a user of a virtual environment, the image generator being responsive to selection of a predetermined input by the user; and a video camera configured for use in image-based tracking of a user by the image generator; in which the predetermined input selected by the user is a performance of a predetermined physical gesture in view of the video camera; wherein the image generator is configured to generate a static or an animated emote sticker responsive to the predetermined physical gesture of the user for display to other users. 2. Apparatus according to claim 1, in which: the virtual environment includes an avatar representation of the user positioned within the virtual environment so that the user's viewpoint of the virtual environment substantially corresponds to a viewpoint of the avatar corresponding to that user; and the image generator is arranged to generate respective facial configurations of the avatar responsive to the predetermined physical gesture of the user for display to other users viewing the virtual environment. 3. Apparatus according to claim 1, in which the predetermined input further comprises one or more selected from the list consisting of: i. pressing one or more predetermined buttons; ii. adopting a predetermined pose while holding a control device; and iii. performing the predetermined physical gesture while holding the control device. 4. Apparatus according to claim 1, in which: the predetermined physical gesture comprises a plurality of predetermined physical gestures, and each of the plurality of predetermined physical gestures corresponds to a single predefined static or animated emote sticker. 5. Apparatus according to claim 1, in which: the predetermined physical gesture comprises a plurality of predetermined physical gestures, and two or more of the plurality of predetermined physical gestures correspond to a single predefined static or animated emote sticker. 6. Apparatus according to claim 1, in which the predetermined input is associated with selection of a predetermined static or animated emote sticker, and in which selection of an emotional expression in the sticker is in dependence upon one or more selected from the list consisting of: i. metadata associated with the virtual environment; ii. metadata associated with a virtual object in the virtual environment with which the user is interacting; iii. metadata associated with another avatar with which the user is interacting; and iv. time of day. 7. Apparatus according to claim 1, further comprising a tracking processor, the tracking processing being configured to track a skeletal model of the user as the user performs a given predetermined physical gesture, and is further configured to at least partially transpose the skeletal model of the user onto a skeletal model of the emote sticker. 8. Apparatus according to claim 7, in which a predetermined emotion is selected for the emote sticker in response to at least part of the performed predetermined physical gesture. 9. Apparatus according to claim 1, in which: an object in the virtual environment is associated with emotion preference metadata indicative of a preferred emotion; and the image generator is responsive to interaction by the user with the object to configure a face of the emote sticker in accordance with the emotion preference metadata. 10. Apparatus according to claim 1, in which the image generator is further configured to generate images representing avatars corresponding to other users interacting with the virtual environment. 11. Apparatus according to claim 10, in which the image generator is configured to generate an image representing a given avatar associated with a selected user, the given avatar having a facial configuration dependent upon user control operations by the selected user. 12. Apparatus according to claim 10, in which the image generator is configured to generate an image representing a particular avatar having an eye direction dependent upon a position, relative to the particular avatar, of an eye-look trigger item. 13. Apparatus according to claim 1, further comprising a head mountable display. 14. A method of generating images representing a user of a virtual environment, the method comprising the steps of; tracking a user in images captured by a video camera; detecting, by an image processor device, a predetermined physical gesture by the user in images captured by the video camera; and generating, by an image generator, a static or animated emote sticker responsive to the predetermined physical gesture of the user for display to other users. 15. (canceled) 16. A non-transitory, machine-readable storage medium on which computer-readable instructions are stored, the instructions, when executed by one or more processors, cause the processors to perform the method of claim 14. 17. The method of claim 14, in which the virtual environment includes an avatar representation of the user positioned within the virtual environment so that the user's viewpoint of the virtual environment substantially corresponds to a viewpoint of the avatar corresponding to that user, wherein the method further comprises: the image generator generating respective facial configurations of the avatar responsive to the predetermined physical gesture of the user for display to other users viewing the virtual environment. 18. The method of claim 14, in which the predetermined physical gesture comprises a plurality of predetermined physical gestures, and: each of the plurality of predetermined physical gestures corresponds to a single predefined static or animated emote sticker; or two or more of the plurality of predetermined physical gestures correspond to a single predefined static or animated emote sticker. 19. The method of claim 14, further comprising: tracking a skeletal model of the user as the user performs a given predetermined physical gesture, and at least partially transposing the skeletal model of the user onto a skeletal model of the emote sticker. 20. The method of claim 14, wherein: an object in the virtual environment is associated with emotion preference metadata indicative of a preferred emotion: and in response to interaction by the user with the object, the method further includes configuring a face of the emote sticker in accordance with the emotion preference metadata. 21. The method of claim 14, further comprising generating images representing avatars corresponding to other users interacting with the virtual environment."
  ],
  "cpc": [
    "G06T 13/40",
    "A63F 13/211",
    "A63F 13/213",
    "A63F 13/215",
    "A63F 13/26",
    "A63F 13/428",
    "A63F 13/56",
    "A63F 13/58",
    "A63F 13/87",
    "G02B 2027/014",
    "G02B 2027/0141",
    "G02B 2027/0187",
    "G02B 27/017",
    "G02B 27/0172",
    "G06F 3/011",
    "G06F 3/013",
    "G06F 3/017",
    "G06F 3/0304",
    "G06F 3/0346",
    "G06F 3/04815",
    "G06F 3/167",
    "G06K 9/00355",
    "G06T 11/60",
    "G06T 19/003",
    "G06T 19/006",
    "G06T 19/20",
    "G06V 40/28",
    "H04N 7/157"
  ],
  "assignees": [
    "SONY INTERACTIVE ENTERTAINMENT EUROPE LTD"
  ],
  "filing_date": "2017-03-10",
  "publication_date": "2019-04-04",
  "priority_date": "2016-03-11",
  "application_number": "US-201716082145-A",
  "family_id": "55952203",
  "citations": [
    "US2005261032A1",
    "US2008215975A1",
    "US2008309671A1",
    "US2012127164A1",
    "US2014364212A1",
    "US2016026253A1",
    "US2016033768A1",
    "US2016086365A1",
    "US2018081515A1"
  ]
}