Patent US9314685

Action detection and activity classification
[+] Show Details

US9314685

1. A computer-implemented method comprising: (8) (11)

receiving, by an activity processing system, a type of athletic activity to be performed;

identifying, by the activity processing system, a sensor subscription associated with the type of athletic activity to be performed;

transmitting, from the activity processing system and to a sensor system having one or more sensors, the identified sensor subscription;

receiving, from one or more sensors and by the sensor system, raw sensor data;

identifying, by the sensor system and from the raw sensor data, a plurality of events detected from one or more sensor signal streams during performance of the athletic activity by a user;

responsive to identifying the plurality of events, transmitting from the sensor system to the activity processing system, the sensor data, transmitting the sensor data including controlling, by the sensor system, at least one of an amount of data and a type of data based on the sensor subscription;

analyzing, by the activity processing system, the received sensor data to evaluate the plurality of events against one or more constraints of a first action template to determine whether the user performed a first type of action corresponding to the first action template;

determining, by the activity processing system, whether the plurality of events matches the first action template based on the evaluation of the plurality of events against the one or more constraints of the first action template;

in response to determining that the plurality of events matches the one or more constraints of the first action template, registering, by the activity processing system, user performance of the first type of action corresponding to the first action template;

transmitting, by the activity processing system, the registration of the user performance of the first type of action corresponding to the first action template to one or more user devices different from the activity processing system; and

displaying, on a display of the one or more user devices, the user performance of the first type of action.

2. The computer-implemented method of claim 1, further comprising: (0) (2)

3. The computer-implemented method of claim 1, wherein comparing the plurality of events to a first action template includes: (1) (2)

selecting a first state from the first action template to evaluate; and

determining whether a first candidate event of the plurality of events meets one or more non-relative constraints of the first state.

4. The computer-implemented method of claim 3, wherein comparing the plurality of events to the first action template further includes: (3) (1)

determining whether the first candidate event meets one or more relative constraints of the first state, wherein the one or more relative constraints defines a required relationship between the first candidate event and a second candidate event.

5. The computer-implemented method of claim 4, wherein the one or more relative constraints further define at least one of: a required relative timing between a current template matching analysis and one or more past template matching analyses of one or more other events, and a required relationship with statistics regarding prior occurrences of events of a type corresponding to the first state. (0)

6. The computer-implemented method of claim 4, wherein determining whether the first candidate event meets the one or more relative constraints of the first state includes: (0) (2)

7. The computer-implemented method of claim 4, wherein upon determining that the first candidate event meets the non-relative and relative constraints of the first state, analyzing each of one or more remaining template states in a sequential order such that the analysis for each of the one or more remaining template state includes: (0) (3)

determining whether a candidate event from the plurality of events meets the one or more non-relative constraints of the template state being analyzed;

determining whether the candidate event from the plurality of events that satisfied the non-relative constraints of the template state being analyzed also satisfies one or more relative constraints of the template state being analyzed, wherein the one or more relative constraints depend on one or more template states already analyzed; and

in response to determining that the candidate event meets the one or more non-relative and relative constraints of the template state being analyzed, proceeding to analyze another remaining template state.

8. The computer-implemented method of claim 1, further comprising: (0) (1)

9. The computer-implemented method of claim 1, further comprising: (0) (1)

10. The computer-implemented method of claim 1, further comprising non-dimensionalizing one or more characteristics of the plurality of events. (0)

11. The computer-implemented method of claim 1, further comprising: (0) (2)

12. The computer-implemented method of claim 1, further comprising: (1) (3)

determining that the plurality of events matches the first action template and a second action template;

selecting one of the first and second templates; and

defining the first type of action based on the selected one of the first and second templates.

13. The computer-implemented method of claim 12, wherein selecting the one of the first and second action templates includes: (0) (3)

14. The computer-implemented method of claim 1, wherein the first action template includes a match tolerance. (0)

15. A system comprising: (0) (3)

an activity processing system, including: a first processor; and

first memory storing computer readable instructions that, when executed, cause the first processor to: receive a type of athletic activity to be performed;

identify a sensor subscription associated with the type of athletic activity to be performed;

transmit, to a sensor system, the identified sensor subscription; and

the sensor system including one or more sensors, the sensor system further including: a second processor; and

second memory storing computer readable instructions that, when executed, cause the second processor to: receive, from one or more sensors, raw sensor data;

identify, from the raw sensor data, a plurality of events detected from one or more sensor signal streams during performance of the athletic activity by a user;

responsive to identifying a plurality of events, transmit to the activity processing system, the sensor data, transmitting the sensor data including controlling, by the sensor system, at least one of an amount of data and a type of data based on the sensor subscription;

the activity processing system further including instructions that, when executed, cause the first processor to: receive the transmitted, controlled amount or type of sensor data;

analyze the received, controlled amount or type of sensor data to evaluate the plurality of events against one or more constraints of a first action template to determine whether the user performed a first type of action corresponding to the first action template;

determine whether the plurality of events matches the first action template based on the evaluation of the plurality of events against the one or more constraints of the first action template; and

in response to determining that the plurality of events matches the one or more constraints of the first action template, register user performance of the first type of action corresponding to the first action template;

transmit, by the activity processing system, the registration of the user performance of the first type of action corresponding to the first action template to one or more user devices different from the activity processing system; and

display, on a display of the one or more user devices, the user performance of the first type of action.

16. A computer-implemented method comprising: (3) (12)

receiving, by an activity processing system, a type of athletic activity to be performed;

identifying, by the activity processing system, a sensor subscription associated with the type of athletic activity to be performed;

transmitting, from the activity processing system and to a sensor system having one or more sensors, the identified sensor subscription;

identifying, by the activity processing system, a plurality of action templates associated with the type of physical activity;

receiving, from the one or more sensors and by the sensor system, raw sensor data;

identifying, by the sensor system and from the raw sensor data, a plurality of events detected from one or more sensor signal streams during performance of the identified type of athletic activity by a user, the one or more signal streams including at least one stream from an image-capturing device;

analyzing, by activity processing system, the sensor data received from the sensor system to evaluate the plurality of events against one or more constraints of the plurality of action templates to determine whether the user performed a first type of action corresponding to the first action template;

determining, by the activity processing system, whether the plurality of events matches at least one action template of the plurality of action templates based on the evaluation of the plurality of events against the one or more constraints of the plurality of action templates;

in response to determining that the plurality of events matches the one or more constraints of the at least one action template of the plurality of action templates, registering, by the activity processing system user performance of a first type of action associated with the identified physical activity and corresponding to the at least one action template of the plurality of action templates;

displaying, on a display of the one or more user devices, the user performance of the first type of action.

17. The method of claim 16, further including determining at least one metric associated with performance of the first type of action. (0)

18. The method of claim 16, further including: (1) (2)

20. The method of claim 16, wherein the at least one action template includes a match tolerance. (0)

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Patent US9314685

Action detection and activity classification

Patent Matrix® Search