Signature
Parameters
int
required
id of endpoint group to update
Optional[float]
minimum floor load in perf units/s (token/s for LLms)
Optional[float]
minimum floor load in perf units/s (token/s for LLms), but allow handling with cold workers
Optional[str]
active, suspended, or stopped
Optional[float]
target capacity utilization (fraction, max 1.0, default 0.9)
Optional[float]
cold/stopped instance capacity target as multiple of hot capacity target (default 2.5)
Optional[int]
min number of workers to keep ‘cold’ when you have no load (default 5)
Optional[int]
max number of workers your endpoint group can have (default 20)
Optional[str]
deployment endpoint name (allows multiple workergroups to share same deployment endpoint)
Optional[float]
maximum seconds requests may be queued on each worker (default 30.0)
Optional[float]
target seconds for the queue to be cleared (default 10.0)
Optional[int]
seconds of no traffic before the endpoint can scale to zero active workers
Returns
dict