Running as a Service

A model server can be handed to the operating system to supervise, so it comes up at boot and stays up. No manager process is involved, and there is no wrapper to install.

Why The Runner And Not The Manager

The thing worth keeping alive is the model server. It holds the model on the GPU and answers requests, so it is what belongs under the OS supervisor. The manager is an editor and launcher for endpoints plus the Studio UI, which is something you start when you want it.

This works because an endpoint's config file is the whole configuration. A service is then nothing more than "run the binary with this config file at boot", which is why no wrapper such as NSSM or WinSW is needed on Windows and no hand-written unit is needed on Linux.

Installing

Point the verb at a config file. paddock serve writes one per endpoint at ~/paddock/servers/<port>.toml, and a hand-written file works just as well.

# Windows, from an elevated terminal
paddock-runner service install --config servers\11540.toml

# Linux
paddock-runner service install --config ~/paddock/servers/11540.toml

# remove it again
paddock-runner service uninstall --config ~/paddock/servers/11540.toml

The service is named after the port it serves, paddock-runner-11540, so one port is one server is one service and two endpoints can never collide. --name overrides it, and uninstall then takes the same name.

The config path is resolved to an absolute path at install time. Services start with an arbitrary working directory, so a relative path would break the moment the system launched it rather than when you typed it.

Windows

Installing registers a native auto-start Windows service running as LocalSystem. It needs an elevated terminal; without one the service manager cannot be opened, and the error says so rather than failing obscurely.

Stopping the service from the service manager is a clean stop, not a kill: the runner drains its in-flight requests through its own admin surface and then exits, and the service manager reads that exit as stopped. A request that was mid-generation finishes instead of vanishing.

Linux

The same verb writes a systemd unit and enables it. Which kind depends on who you are:

A user unit starts at login rather than at boot. Installing one prints the note that says so, along with the fix, because "starts at boot" and "starts when you log in" are different promises and the difference only shows up after a reboot nobody is watching:

loginctl enable-linger $USER

The generated unit is ordinary and readable. It runs the binary against your config file, restarts on failure after three seconds, and waits for the network:

[Unit]
Description=Paddock model server (paddock-runner-11540)
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/paddock-runner --config /home/you/paddock/servers/11540.toml
Restart=on-failure
RestartSec=3

[Install]
WantedBy=default.target

One Boot Owner Per Model

A manager started next to service-managed runners adopts them over the admin pipe: it sees them, lists them in paddock ps and in the Studio, and never spawns a second server on a port that already has one.

What it will not do is decide which of you owns starting it. Pick one boot owner per model, either the service unit or a manager start-on-boot election, and not both. Two owners for one port is how you get a fight over a GPU at boot.

Running Without A Manager At All

Nothing about a service needs the manager, and this is the deployment shape for a headless box: install the units, and the machine serves models after a reboot with no UI process running. The manager remains available for when you want to change something, and adopts whatever it finds.