1.
Overview
2.
Build
2.1.
Telemetry(alpha)
3.
Architecture
4.
Structure
5.
Components
5.1.
Router
5.1.1.
Disaggregation
5.1.2.
Exec_Blitz
5.1.3.
Infer
5.1.4.
Migration_queue
5.1.5.
Planner(TODO)
5.1.6.
Colocation(TODO)
5.2.
Client
5.3.
Server
6.
Scripts
6.1.
Args
6.2.
Usage
Example
7.
Example1
Q&A
8.
Questions
9.
Acknowledgement
Light
Rust
Coal
Navy
Ayu
blitzscale-doc
Components
组件可基本分为三个
router: rust实现，进行blitzscale核心zigzag、扩缩容的实现，接受client的推理请求，并将batch request发送给server进行推理。兼容openai接口
server: c++实现，负责拉取参数，进行算子执行
client: python实现的request-sim，用于测试时模拟客户请求