-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathopenapi.yaml
More file actions
593 lines (563 loc) · 21.2 KB
/
Copy pathopenapi.yaml
File metadata and controls
593 lines (563 loc) · 21.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
openapi: 3.1.0
# The daemon's HTTP control API — what `spinloop daemon` always exposes, and what
# `spinloop serve --api` exposes alongside a foreground engine.
#
# This file is checked against the implementation by
# internal/daemon/openapi_test.go: the routes here must match the ones the
# handler registers, and each schema's properties must match the JSON tags of
# the Go struct it describes. The test compares names, not types — where the two
# could still disagree, the Go code is the source of truth.
#
# Prose on the behaviour a schema cannot express lives in docs/http-api.md.
info:
title: spinloop daemon control API
summary: Supervise and observe a local inference engine.
description: |
The control API drives one supervised inference engine: start it, stop it,
ask what it is doing, and set what it serves next.
It listens on `:4242` by default (`--api-addr` overrides; on the daemon,
`--loopback` binds `127.0.0.1:4242`). A non-loopback
listen with no bearer token configured refuses to start, since the API can
start and stop processes; a loopback listen may go tokenless.
Under `spinloop daemon` nothing runs until a start request asks, and stopping
the engine never ends the daemon — the API keeps answering. Under
`spinloop serve --api` the engine is foreground-managed, so start always
fails as already-running and stopping the engine ends serve itself.
license:
name: MIT
identifier: MIT
version: "1"
servers:
- url: http://127.0.0.1:4242
description: The daemon's default loopback address.
security:
- bearerAuth: []
tags:
- name: engine
description: Driving and observing the supervised engine.
- name: config
description: What the engine serves.
paths:
/v1/status:
get:
tags: [engine]
operationId: getStatus
summary: Report the supervised engine's state and idle time.
description: |
`lastActiveAt` and `idleSeconds` are the daemon's own answer to whether
the engine is busy, derived from token counters it samples every 15
seconds. Both are absent until an engine has run.
responses:
"200":
description: The engine's current state.
content:
application/json:
schema:
$ref: "#/components/schemas/StatusResponse"
"401":
$ref: "#/components/responses/Unauthorized"
/v1/start:
post:
tags: [engine]
operationId: startEngine
summary: Start the engine.
description: |
The body may carry a deploy config naming what to run, and the key the
engine is gated with — validated and persisted exactly as a push, then
started. With no body, the stored deploy config is served, gated with
the stored key; with nothing stored, the start fails saying so.
A body sent with a rejected start is not stored, neither its config nor
its key.
requestBody:
required: false
content:
application/json:
schema:
$ref: "#/components/schemas/StartRequest"
responses:
"200":
description: The engine started; the reply is its new state.
content:
application/json:
schema:
$ref: "#/components/schemas/StatusResponse"
"400":
description: |
The config is invalid, the engine failed to start, there is nothing
to serve, or a key was supplied for an engine that cannot be gated
by one.
content:
application/json:
schema:
$ref: "#/components/schemas/Error"
"401":
$ref: "#/components/responses/Unauthorized"
"409":
description: An engine is already running. Nothing changed, and a carried config was not stored.
content:
application/json:
schema:
$ref: "#/components/schemas/Error"
/v1/stop:
post:
tags: [engine]
operationId: stopEngine
summary: Stop the engine.
description: |
Idempotent: stopping when nothing runs succeeds. Under `spinloop daemon`
this never ends the daemon itself.
responses:
"200":
description: The engine is stopped; the reply is its new state.
content:
application/json:
schema:
$ref: "#/components/schemas/StatusResponse"
"401":
$ref: "#/components/responses/Unauthorized"
"500":
description: The engine could not be stopped.
content:
application/json:
schema:
$ref: "#/components/schemas/Error"
/v1/metrics:
get:
tags: [engine]
operationId: getMetrics
summary: Return collected engine and host metrics.
description: |
Engine token counters scraped from the engine's own Prometheus endpoint,
plus host GPU, CPU and memory figures. Every stat is optional: a host
with no source for one simply omits it, so a machine without
`nvidia-smi` reports engine stats and no GPU figures.
responses:
"200":
description: The collected stats.
content:
application/json:
schema:
$ref: "#/components/schemas/Stats"
"401":
$ref: "#/components/responses/Unauthorized"
/v1/logs:
get:
tags: [engine]
operationId: getLogs
summary: Return a slice of the engine's captured output.
description: |
The supervised engine's stdout and stderr, as captured to the file
`/v1/status` reports as `logPath`. Read-only: it never touches the
engine, so it answers whether the engine is running, stopped or
crashed — the last of which is when it is wanted most.
Reads are always bounded. Omit `offset` to read the **end** of the log;
pass the `nextOffset` from a previous reply to receive only what has
been appended since, which makes following exact — no overlap window
and no de-duplication. Nothing rotates this file, so it grows for the
daemon's lifetime and a full read is never offered.
parameters:
- name: offset
in: query
required: false
description: |
Byte position to read from, normally the `nextOffset` of a previous
reply. Omitted, the end of the log is returned.
schema:
type: integer
format: int64
minimum: 0
- name: limit
in: query
required: false
description: |
Maximum bytes to return. Capped by the daemon regardless of what is
asked for; omitted, a default slice is returned.
schema:
type: integer
format: int64
minimum: 1
responses:
"200":
description: The requested slice of the log.
content:
application/json:
schema:
$ref: "#/components/schemas/LogsResponse"
"400":
description: A query parameter was not a whole number, or was negative.
content:
application/json:
schema:
$ref: "#/components/schemas/Error"
"401":
$ref: "#/components/responses/Unauthorized"
/v1/deploy-config:
put:
tags: [config]
operationId: putDeployConfig
summary: Set what the next start serves.
description: |
A running engine is deliberately untouched — the config takes effect on
the next start, which the reply says.
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/DeployConfig"
responses:
"200":
description: The config was stored.
content:
application/json:
schema:
$ref: "#/components/schemas/Message"
"400":
description: The config is invalid or names a runner this host cannot serve.
content:
application/json:
schema:
$ref: "#/components/schemas/Error"
"401":
$ref: "#/components/responses/Unauthorized"
components:
securitySchemes:
bearerAuth:
type: http
scheme: bearer
description: |
The token comes from the `SPINLOOP_API_TOKEN` environment variable —
reachable through the Spinloop's adjacent `.env` — never from a flag, so
the secret stays out of the process table. When no token is configured
the API is unauthenticated, which `Listen` permits only on loopback.
responses:
Unauthorized:
description: The bearer token is missing or wrong.
content:
application/json:
schema:
$ref: "#/components/schemas/Error"
schemas:
StartRequest:
type: object
description: |
The body of a start request: what to run, and the key the engine is
gated with. Mirrors Go's daemon.StartRequest, which embeds a
DeployConfig.
allOf:
- $ref: '#/components/schemas/DeployConfig'
- type: object
properties:
engineApiKey:
type: string
description: |
The API key the engine is started with. Supplied by the caller
— a node sources no key of its own — and travelling with the
config it accompanies: a start carrying a config and no key
opens the engine, while a start carrying neither reuses what
was stored.
It is never returned. No reply, error, or log line produced by
this API contains it, and the daemon passes it to the engine
as a file path rather than an argument, so it does not appear
in the node's process list either.
StatusResponse:
type: object
description: The supervised engine's state. Mirrors Go's daemon.StatusResponse.
required: [state]
properties:
state:
type: string
enum: [idle, running, stopped, crashed]
description: |
`idle` means nothing has been started yet; `crashed` means the
engine exited unprompted, and is reported rather than restarted.
runner:
type: string
description: The engine being served, when known.
examples: [llamacpp, omlx, vllm, mtplx]
model:
type: string
description: What it is serving, when known.
uptimeSeconds:
type: integer
description: How long the engine has been running. Zero unless running.
logPath:
type: string
description: The engine's log file, when running under the daemon.
lastActiveAt:
type: string
format: date-time
description: |
When the engine last did any work, RFC 3339. Moved forward by a
sample showing requests in flight or a moved counter, and by an
engine start. Absent until an engine has run.
idleSeconds:
type: integer
description: Seconds since `lastActiveAt`. Absent when that is.
engine:
allOf:
- $ref: '#/components/schemas/EngineEndpoint'
description: |
Where the running engine answers inference requests. Absent unless
an engine is running.
ready:
type: string
enum: [ready, not-ready]
description: |
Whether the running engine last answered its own health check,
distinct from `state` reaching `running` — the process can be
alive while still loading weights. Absent, not `not-ready`, when
it does not apply: no engine is running, the running engine's
runner has no known health-check convention, or this daemon
predates the check. Mirrored on Stats' `ready`, from the same
record.
version:
type: string
description: |
The spinloop binary's build-time version string, from
`main.version`. Set to "dev" when not overridden at build time.
EngineEndpoint:
type: object
description: |
Where the supervised engine serves. Parts rather than a URL on
purpose: the daemon knows its engine binds `127.0.0.1:8080`, which is
useless to anyone else, and it cannot know the name a client reaches
this host by — a LAN name, a tailscale name, a published container
port. The caller composes these against the host it already has.
required: [port]
properties:
port:
type: integer
description: |
The port the engine listens on — the engine's, never the control
API's. One is not derivable from the other.
examples: [8080, 8000]
path:
type: string
description: |
The OpenAI-compatible path prefix, when it is not the usual `/v1`.
Absent means the default.
loopbackOnly:
type: boolean
description: |
The engine is bound to loopback, so it answers only on that
machine. Lets a remote caller explain a refused connection rather
than merely suffer it.
requiresKey:
type: boolean
description: |
The engine was started with an API key, so a caller needs one. The
key itself is never reported under any endpoint: authorisation to
drive a node is not authorisation to be handed its engine's
credential.
Stats:
type: object
description: Collected engine and host metrics. Mirrors Go's metrics.Stats.
required: [state]
properties:
state:
type: string
enum: [idle, running, stopped, crashed]
runner:
type: string
modelId:
type: string
uptimeSeconds:
type: integer
tokens:
$ref: "#/components/schemas/TokenStats"
gpus:
type: array
items:
$ref: "#/components/schemas/GpuStat"
cpu:
$ref: "#/components/schemas/CpuStat"
memory:
$ref: "#/components/schemas/MemoryStat"
errors:
type: array
description: Collection failures. An absent source is omitted rather than reported here.
items:
type: string
lastActiveAt:
type: string
format: date-time
description: |
When the engine last did any work, RFC 3339. The same value
`/v1/status` reports, from the same record. Reported whatever the
engine's state — a stopped engine still says when it last worked,
even though the figures above are absent. Absent until an engine
has run.
idleSeconds:
type: integer
description: Seconds since `lastActiveAt`. Absent when that is, and absent at zero.
ready:
type: string
enum: [ready, not-ready]
description: |
Whether the running engine last answered its own health check.
The same value `/v1/status` reports, from the same record.
Absent, not `not-ready`, when it does not apply: no engine is
running, its runner has no known health-check convention, or
this daemon predates the check.
TokenStats:
type: object
description: Per-engine token and request counters, read from its Prometheus endpoint.
properties:
running:
type: integer
description: Requests in flight (processing plus waiting/deferred).
counter:
type: integer
description: The sum of the engine's cumulative counters — the activity signal.
promptTokens:
type: integer
generationTokens:
type: integer
requests:
type: integer
GpuStat:
type: object
description: One GPU's figures, from nvidia-smi.
properties:
index:
type: integer
name:
type: string
examples: ["NVIDIA L40S"]
utilization:
type: integer
description: Percent.
memoryUsed:
type: integer
description: Bytes.
memoryTotal:
type: integer
description: Bytes.
temperature:
type: integer
description: Degrees Celsius.
CpuStat:
type: object
description: Whole-host CPU utilisation.
properties:
utilization:
type: number
description: Percent.
MemoryStat:
type: object
description: System memory, in bytes.
properties:
total:
type: integer
used:
type: integer
DeployConfig:
type: object
description: |
What to serve, in the same shape `spinloop remote deploy` derives from an
Spinloop. Mirrors Go's remote.DeployConfig. There is no default runner: an
unset or invalid one fails loudly rather than guessing.
required: [runner]
properties:
runner:
type: string
# The engines a deploy config can name. mtplx appears because a fleet
# node can be woken with it; it is not a cloud runner (no machine
# image). omlx is not a wakeable runner yet.
enum: [llamacpp, vllm, mtplx]
modelId:
type: string
description: The weights to serve — a Hugging Face repo, or a local path on the instance.
quant:
type: string
description: The quantisation to select, where the runner takes one.
contextSize:
type: integer
parallel:
type: integer
description: |
The number of concurrent request slots the engine should run
with. Zero/absent means unset: no parallelism flag is added, and
contextSize is used unscaled. Translated into each runner's own
flag at start time — llamacpp's ctx-size is scaled by this value,
since llama.cpp divides that budget across its parallel slots;
vllm's context is left unscaled, since its concurrency is bounded
independently via max-num-seqs.
servedModelName:
type: string
description: The name the endpoint advertises the model under.
serveArgs:
type: array
description: Runner-specific flags, pre-tokenised.
items:
type: string
companions:
type: object
description: |
Companion weights loaded beside the main weights, keyed by role.
Each value is a bare filename within the model's own Hugging Face
repo, never a path — the deployment decides where it lands on disk
and names it there itself. Omitted when there are none.
additionalProperties:
type: string
propertyNames:
enum: [draft, mmproj]
spinloopVersion:
type: string
description: |
The spinloop release the environment's instances install at boot.
Empty or absent means the boot installs the latest published
release. A pin is normalised before it is sent — the leading v
of a release tag is not part of the version (1.26.1, not
v1.26.1). The environment's control plane reads it when it
renders the boot script; the daemon does not act on it.
Message:
type: object
description: A plain acknowledgement, where there is nothing to report but acceptance.
required: [message]
properties:
message:
type: string
LogsResponse:
type: object
description: |
A bounded slice of the engine's captured output, with the cursor needed
to read on from it.
required: [content, nextOffset, size]
properties:
content:
type: string
description: The slice of the log that was read.
nextOffset:
type: integer
format: int64
description: |
The position immediately after `content`. Pass it back as `offset`
to receive only what has been appended since.
size:
type: integer
format: int64
description: The log's current length, so a caller can see how far behind it is.
path:
type: string
description: The log file being read, matching `logPath` from status.
missing:
type: boolean
description: |
There is no log file: no engine has ever run here, or the daemon
forwards engine output to its own stdio. Distinct from a log that
exists and is empty.
staleOffset:
type: boolean
description: |
The requested `offset` is past the end of the log — it was truncated
or replaced — so resume from `nextOffset` rather than waiting for a
position that will never arrive.
Error:
type: object
description: The failure reply. Every non-2xx status carries one.
required: [error]
properties:
error:
type: string