Public listing
腾讯云-MaaS平台推理流量调度高级工程师(北/杭/深)
Tencent
Shenzhen, China
- Experience> 3 years of working experience
- ListedPosting date unavailable
- SourceOwn
Role overview
About the role
Only a short excerpt is available here.
The full responsibilities and requirements may be missing. Sign in to open the original listing and confirm the details before applying.
1.负责推理流量的全地域、跨集群调度与分发，结合模型特性、实时负载、算力容量及地域约束设计调度策略，提高 GPU 有效利用率和负载均衡能力；
2.建设租户级流控与配额管理体系，完善并发及 Token 级限流、优先级排队、熔断降级和过载保护机制，保障服务稳定性、资源公平性和租户隔离；
3.建设潮汐调度、HPA等弹性编排能力，结合流量预测、排队情况与推理负载，完善动态扩缩容、模型预热、请求排空及跨资源池容量调配，平衡资源成本与服务质量；
4.开展推理链路性能调优，建设缓存感知路由，协同优化前缀缓存复用、请求排队与批处理，提升缓存命中率和吞吐，降低首 Token时延、Token间时延及尾延迟。
Experience
> 3 years of working experience
Job alerts
Get notified about new openings
Set your preferences and JobBeeper will send matching roles as it finds them.