Cloud Native + Open Source Virtual Summit China 2020

The emergence of 5G technology has made it increasingly important to reduce costs by deploying model inference on edge. Baidu internally deploys recommendation model inference on distributed CDN nodes by leveraging Knative and EDL, reduced core network load by 25% and network communication latency by 40%, thus the accuracy of recommendation model inference service got improved.

This talk will share some experiences & lessons Baidu learnt from deploying and optimizing recommendation model inference on edge by leveraging Knative and EDL, including inference auto-scaling and ingress traffic optimization based on geographic location, cold start optimization on CDN node with model pre-loading and legacy services interaction with Cloud Events.