-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathSpinloop
More file actions
35 lines (35 loc) · 1.86 KB
/
Copy pathSpinloop
File metadata and controls
35 lines (35 loc) · 1.86 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# Remote Qwen endpoint, run as an environment on the shared layer.
#
# This file is hand-maintained — nothing generates or rewrites it. The shared
# infrastructure is deployed once per account with `spinloop remote bootstrap`;
# then, from this directory:
# spinloop remote deploy # create the environment REMOTE names + set what it
# # serves (this file + preset); registers its
# # remote.json under ~/.config/spinloop/remotes/
# spinloop remote start # boot (or re-wake a stopped) the instance; prints
# # the OPENAI_API_KEY export
# spinloop apply # point your coding agent at the endpoint
# spinloop remote pause # stop now (kept for a re-wake) instead of waiting
# # for the idle timer
# spinloop remote stop # terminate now instead of waiting for the idle timer
#
# PROVIDER names the inference engine, so this file is what chooses llama.cpp
# over vLLM: `spinloop remote deploy` asks the cloud for that engine, and the same
# file runs it locally under `spinloop serve`. To cut back to vLLM, set PROVIDER
# to vllm, drop PRESET, and set MODEL to the FP8 repo (Qwen/Qwen3.6-27B-FP8).
# Either way the deploy Lambda seeds the weights into S3 if they are not there.
#
# ALIAS is both the name your coding agent asks for and the name the server is
# started under (llama-server's --alias), so the two cannot drift apart.
#
# REMOTE names the environment: its EIP, API key and config are its own, so
# several Spinloops can run several instances side by side in one account.
#
# There is deliberately no BASEURL: the environment's address belongs to the
# deployment, so `spinloop apply` takes it from the environment's remote.json.
# Add a BASEURL here only to override that.
PROVIDER llamacpp
ALIAS qwen3.6-27b
CONTEXT 131072
PRESET ./preset.ini
REMOTE qwen3.6-27b