User Detection For Projection-based Augmented Reality System
In one embodiment, a method includes sending, to an interaction device including a projector and a camera, a media content item and instructions causing the projector to project the media content item on a projectable surface and receiving, from the interaction device, one or more media objects captured by the camera, where one or more of the media objects include images of a user in proximity to the projectable surface. The method includes determining one or more movements of the user based on the one or more of the media objects and updating the media content item based on the determined movements. The method also includes sending, to the interaction device, the updated media content item and instructions causing the projector to project the updated media content item on the projectable surface.
Claim CLM-00001. 1. A method comprising:
by a computing device, sending, to an interaction device comprising a projector and a plurality of cameras, a media content item and instructions causing the projector to project the media content item on a projectable surface, wherein the interaction device performs a user detection using a combination of wide-angle camera and one or more detail-capturing cameras, and wherein the projectable surface is selected among a plurality of projectable surfaces based at least on a location of a user such that the projectable surface is in proximity to the user; by the computing device, receiving, from the interaction device, one or more media objects captured by the camera, wherein one or more of the media objects comprise images of the user in proximity to the projectable surface; by the computing device, detecting one or more movements of one or more body parts of the user based on the one or more of the media objects; by the computing device, extracting one or more features from the detected one or more movements of the one or more body parts of the user; by the computing device, determining an intended gesture of the user by comparing the extracted one or more features to a pre-defined model of gestures, wherein the determining is based on an identity of the user and personal preferences of the user that are allowed to be used by privacy settings of the user; by the computing device, updating the media content item based on the determined intended gesture of the user; and by the computing device, sending, to the interaction device, the updated media content item and instructions causing the projector to project the updated media content item on the projectable surface.
Claim CLM-00009. 9. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
send, to an interaction device comprising a projector and a plurality of cameras, a media content item and instructions causing the projector to project the media content item on a projectable surface, wherein the interaction device performs a user detection using a combination of wide-angle camera and one or more detail-capturing cameras, and wherein the projectable surface is selected among a plurality of projectable surfaces based at least on a location of a user such that the projectable surface is in proximity to the user; receive, from the interaction device, one or more media objects captured by the camera, wherein one or more of the media objects comprise images of the user in proximity to the projectable surface; detect one or more movements of one or more body parts of the user based on the one or more of the media objects; extract one or more features from the detected one or more movements of the one or more body parts of the user; determine an intended gesture of the user by comparing the extracted one or more features to a pre-defined model of gestures, wherein the determining is based on an identity of the user and personal preferences of the user that are allowed to be used by privacy settings of the user; update the media content item based on the determined intended gesture of the user; and send, to the interaction device, the updated media content item and instructions causing the projector to project the updated media content item on the projectable surface.
Claim CLM-00017. 17. A system comprising:
one or more processors; and one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
send, to an interaction device comprising a projector and a plurality of cameras, a media content item and instructions causing the projector to project the media content item on a projectable surface, wherein the interaction device performs a user detection using a combination of wide-angle camera and one or more detail-capturing cameras, and wherein the projectable surface is selected among a plurality of projectable surfaces based at least on a location of a user such that the projectable surface is in proximity to the user;receive, from the interaction device, one or more media objects captured by the camera, wherein one or more of the media objects comprise images of the user in proximity to the projectable surface;detect one or more movements of one or more body parts of the user based on the one or more of the media objects;extract one or more features from the detected one or more movements of the one or more body parts of the user;determine an intended gesture of the user by comparing the extracted one or more features to a pre-defined model of gestures, wherein the determining is based on an identity of the user and personal preferences of the user that are allowed to be used by privacy settings of the user;update the media content item based on the determined intended gesture of the user; andsend, to the interaction device, the updated media content item and instructions causing the projector to project the updated media content item on the projectable surface.
