Patent US11172189

User detection for projection-based augmented reality system
[+] Show Details

US11172189

1. A method comprising: (6) (7)

by a computing device, sending, to an interaction device comprising a projector and a plurality of cameras, a media content item and instructions causing the projector to project the media content item on a projectable surface, wherein the interaction device performs a user detection using a combination of wide-angle camera and one or more detail-capturing cameras, and wherein the projectable surface is selected among a plurality of projectable surfaces based at least on a location of a user such that the projectable surface is in proximity to the user;

by the computing device, receiving, from the interaction device, one or more media objects captured by the camera, wherein one or more of the media objects comprise images of the user in proximity to the projectable surface;

by the computing device, detecting one or more movements of one or more body parts of the user based on the one or more of the media objects;

by the computing device, extracting one or more features from the detected one or more movements of the one or more body parts of the user;

by the computing device, determining an intended gesture of the user by comparing the extracted one or more features to a pre-defined model of gestures, wherein the determining is based on an identity of the user and personal preferences of the user that are allowed to be used by privacy settings of the user;

by the computing device, updating the media content item based on the determined intended gesture of the user; and

by the computing device, sending, to the interaction device, the updated media content item and instructions causing the projector to project the updated media content item on the projectable surface.

2. The method of claim 1, wherein the intended gesture of the user comprises: (0) (2)

3. The method of claim 1, further comprising: (0) (3)

4. The method of claim 1, further comprising: (0) (1)

5. The method of claim 1, further comprising: (0) (1)

6. The method of claim 1, wherein the interaction device comprises an audio sensor, further comprising: (0) (3)

7. The method of claim 1, further comprising: (1) (1)

determining an identity of the user based at least in part on the one or more of the media objects.

8. The method of claim 7, wherein the determining the identity of the user comprises: (0) (2)

9. One or more computer-readable non-transitory storage media embodying software that is operable when executed to: (6) (7)

send, to an interaction device comprising a projector and a plurality of cameras, a media content item and instructions causing the projector to project the media content item on a projectable surface, wherein the interaction device performs a user detection using a combination of wide-angle camera and one or more detail-capturing cameras, and wherein the projectable surface is selected among a plurality of projectable surfaces based at least on a location of a user such that the projectable surface is in proximity to the user;

receive, from the interaction device, one or more media objects captured by the camera, wherein one or more of the media objects comprise images of the user in proximity to the projectable surface;

detect one or more movements of one or more body parts of the user based on the one or more of the media objects;

extract one or more features from the detected one or more movements of the one or more body parts of the user;

determine an intended gesture of the user by comparing the extracted one or more features to a pre-defined model of gestures, wherein the determining is based on an identity of the user and personal preferences of the user that are allowed to be used by privacy settings of the user;

update the media content item based on the determined intended gesture of the user; and

send, to the interaction device, the updated media content item and instructions causing the projector to project the updated media content item on the projectable surface.

10. The media of claim 9, wherein the intended gesture of the user comprises: (0) (2)

11. The media of claim 9, wherein the software is further operable when executed to: (0) (3)

12. The media of claim 9, wherein the software is further operable when executed to: (0) (1)

13. The media of claim 9, wherein the software is further operable when executed to: (0) (1)

14. The media of claim 9, wherein the interaction device comprises an audio sensor, and wherein the software is further operable when executed to: (0) (3)

15. The media of claim 9, wherein the software is further operable when executed to: (1) (1)

determine an identity of the user based at least in part on the one or more of the media objects.

16. The media of claim 15, wherein the software operable to determine the identity of the user comprises software that is operable when executed to: (0) (2)

17. A system comprising: (0) (2)

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to: send, to an interaction device comprising a projector and a plurality of cameras, a media content item and instructions causing the projector to project the media content item on a projectable surface, wherein the interaction device performs a user detection using a combination of wide-angle camera and one or more detail-capturing cameras, and wherein the projectable surface is selected among a plurality of projectable surfaces based at least on a location of a user such that the projectable surface is in proximity to the user;

receive, from the interaction device, one or more media objects captured by the camera, wherein one or more of the media objects comprise images of the user in proximity to the projectable surface;

detect one or more movements of one or more body parts of the user based on the one or more of the media objects;

extract one or more features from the detected one or more movements of the one or more body parts of the user;

update the media content item based on the determined intended gesture of the user; and

send, to the interaction device, the updated media content item and instructions causing the projector to project the updated media content item on the projectable surface.

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Patent US11172189

User detection for projection-based augmented reality system

Patent Matrix® Search