Application US20190138908

ARTIFICIAL INTELLIGENCE INFERENCE ARCHITECTURE WITH HARDWARE ACCELERATION
[+] Show Details

US20190138908

1. A computing device adapted for artificial intelligence (AI) model processing, the computing device comprising: (8) (2)

communication circuitry to receive a request for an AI operation using an AI model; and

processing circuitry configured to: process the request for the AI operation;

identify, based on the request, an AI hardware platform for execution of an instance of the AI model; and

cause execution of the AI model instance using the AI hardware platform.

2. The computing device of claim 1, wherein the request includes input data to be analyzed with the execution of the AI model instance, and wherein the execution of the AI model instance performs an inference operation with the AI model on the input data. (0)

3. The computing device of claim 1, wherein the request for the AI operation indicates service level agreement (SLA) information and cost information for execution of the instance of the AI model. (0)

4. The computing device of claim 1, wherein the request for the AI operation includes an identifier of the AI model, wherein the processing circuitry is further configured to obtain a binary for the AI model instance based on the identifier. (1) (0)

6. The computing device of claim 1, wherein the request for the AI operation includes a description of the AI model, wherein the description of the AI model specifies a type of neural network, a type of structures used in the neural network, and weights applied in the neural network. (0)

7. The computing device of claim 1, wherein the request for the AI operation includes binary data for the AI model instance, and wherein the AI model instance is executed using the binary data. (0)

8. The computing device of claim 1, further comprising: (0) (1)

9. The computing device of claim 1, wherein the request for the AI operation includes an indication of an accelerator type, and wherein the accelerator type corresponds to a type of AI hardware platform from among a plurality of platform types. (0)

10. The computing device of claim 1, wherein the computing device is implemented as an edge gateway or edge switch within an edge computing platform, and wherein the AI hardware platform comprises an accelerator operable as one of a plurality of hardware accelerators within the edge computing platform. (2) (0)

13. A method for artificial intelligence (AI) model processing with an AI hardware platform, the method comprising a plurality of operations executed with at least one processor and memory of a computing device, and the operations comprising: (8) (3)

obtaining a request for an AI operation using an AI model;

identifying, based on the request, an AI hardware platform for execution of an instance of the AI model; and

causing execution of the AI model instance using the AI hardware platform.

14. The method of claim 13, wherein the request includes input data to be analyzed with the execution of the AI model instance, and wherein the execution of the AI model instance performs an inference operation with the AI model on the input data. (0)

15. The method of claim 13, wherein the request for the AI operation indicates service level agreement (SLA) information and cost information for execution of the instance of the AI model. (0)

16. The method of claim 13, wherein the request for the AI operation includes an identifier of the AI model, the operations further comprising obtaining a binary for the AI model instance based on the identifier. (1) (0)

18. The method of claim 13, wherein the request for the AI operation includes a description of the AI model, wherein the description of the AI model specifies a type of neural network, a type of structures used in the neural network, and weights applied in the neural network. (0)

19. The method of claim 13, wherein the request for the AI operation includes binary data for the AI model instance, and wherein the AI model instance is executed using the binary data. (0)

20. The method of claim 13, further comprising: (0) (1)

21. The method of claim 13, wherein the request for the AI operation includes an indication of an accelerator type, and wherein the accelerator type corresponds to a type of AI hardware platform from among a plurality of platform types. (0)

22. The method of claim 13, wherein the computing device is implemented as an edge gateway or edge switch within an edge computing platform, (1) (2)

wherein the AI hardware platform comprises an accelerator operable as one of a plurality of hardware accelerators within the edge computing platform, and

wherein the plurality of hardware accelerators comprises hardware designated to perform AI operations, the hardware selected from among: field programmable gate array (FPGA) units, neural processing units, application-specific integrated circuit (ASIC) units, a neural compute stick, a vision processing unit, a graphics processing unit (GPU) array, or neuromorphic hardware units.

23. The method of claim 22, wherein the request for the AI operation is received from an edge device communicatively coupled to the edge computing platform, the operations further comprising: (0) (1)

24. At least one non-transitory machine-readable storage medium, comprising a plurality of instructions adapted for artificial intelligence (AI) model processing with an AI hardware platform, wherein the instructions, responsive to being executed with processor circuitry of a computing machine, cause the processor circuitry to perform operations comprising: (1) (4)

obtaining a request for an AI operation;

identifying, based on the request, an AI hardware platform for execution of an instance of the AI model; and

causing execution of the instance of the AI model using the AI hardware platform;

wherein the computing device is implemented as an edge gateway or edge switch within an edge computing platform, and wherein the AI hardware platform comprises an accelerator operable as one of a plurality of hardware accelerators within the edge computing platform.

25. The machine-readable storage medium of claim 24, (0) (1)

View Abstract and Specification Size

PDF with Images and Document Face >

Full Text Publication >

Application US20190138908

ARTIFICIAL INTELLIGENCE INFERENCE ARCHITECTURE WITH HARDWARE ACCELERATION

Patent Matrix® Search