Patent US10825197

Three dimensional position estimation mechanism
[+] Show Details

US10825197

1. An apparatus to facilitate three dimensional (3D) position estimation, comprising: (4) (2)

one or more processors to receive a plurality of two dimensional (2D) images captured by a camera array during a live event, locate key-points of human joints of a plurality of event participants included in the images, associate key-points of each participant across the images and recover a 3D body position of each of the plurality of participants based on the associated key-points, wherein locating the key-points of the human joints comprises detecting major joints of each of the plurality of participants associated with a bounding box and labeling the key-points at the major joints, and

wherein the one or more processors are further to perform triangulation of a position of each of the major joints in a 3D space for a participant having the human joints, wherein the triangulation is performed based on camera projection matrices and pixel-locations of the major joints across the images.

2. The apparatus of claim 1, wherein the one or more processors further generates an animated model including animated motion for one or more of the plurality of participants. (0)

3. The apparatus of claim 1, wherein associating the key-points of each participant across the images comprises determining a geometric pairwise correspondence between the images and determine one or more groups of images associated with a same participant from a plurality of camera angles. (1) (0)

5. The apparatus of claim 1, wherein the major joints are detected via a convolution neural network (CNN). (0)

6. The apparatus of claim 1, wherein the one or more processors further perform kinematic model fitting to capture a position and orientation of one or more major segments of each of the plurality of participants to generate a model corresponding to each of the plurality of participants. (1) (0)

8. A method to facilitate three dimensional (3D) position estimation, comprising: (2) (5)

receiving a plurality of two dimensional (2D) images captured by a camera array during a live event;

locating key-points of human joints of a plurality of event participants included in the images, wherein locating the key-points of the human joints comprises detecting major joints of each of the plurality of participants associated with a bounding box and labeling the key-points at the major joints;

associating key-points of each participant across the images;

recovering a 3D body position of each of the plurality of participants based on the associated key-points; and

performing triangulation of a position of each of the major joints in a 3D space for a participant associated with the human joints, wherein the triangulation is performed based on camera projection matrices and pixel-locations of the major joints across the images.

9. The method of claim 8, further comprising generating an animated model including animated motion for one or more of the plurality of participants. (1) (0)

10. The method of claim 9, wherein associating the key-points of each participant across the images comprises: (1) (2)

determining a geometric pairwise correspondence between the images; and

determining one or more groups of images associated with a same participant from a plurality of camera angles.

11. The method of claim 10, further comprising: (0) (3)

12. The method of claim 8, further comprising: (0) (2)

13. At least one non-transitory computer readable medium having instructions stored thereon, which when executed by one or more processors, cause the processors to: (2) (5)

receive a plurality of two dimensional (2D) images captured by a camera array during a live event;

locate key-points of human joints of a plurality of event participants included in the images, wherein locating the key-points of the human joints comprises detecting major joints of each of the plurality of participants associated with a bounding box and labeling the key-points at the major joints;

associate key-points of each participant across the images;

recover a three-dimensional (3D) body position of each of the plurality of participants based on the associated key-points; and

14. The non-transitory computer readable medium of claim 13, wherein associating the key-points of each participant across the images comprises: (1) (2)

determining a geometric pairwise correspondence between the images; and

determining one or more groups of images associated with a same participant from a plurality of camera angles.

15. The non-transitory computer readable medium of claim 14, having instructions stored thereon, which when executed by one or more processors, further cause the processors to: (0) (3)

16. The non-transitory computer readable medium of claim 13, having instructions stored thereon, which when executed by one or more processors, further cause the processors to: (0) (2)

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Patent US10825197

Three dimensional position estimation mechanism

Patent Matrix® Search