Logs and health checks
When something is wrong, the log is the first place to look, and the health address is the quickest way to know whether the site is up. This page is for whoever runs the server.
The health check
GET /api/health
It needs no login and answers with a small JSON document:
{"status":"ok","platform":"node","version":"1.6.12"}
| Field | Meaning |
|---|---|
status | Always ok when the site answers at all. |
platform | node or cloudflare. |
version | The version of the code that is running. After an update, this is how you confirm which version is live. |
Point an uptime monitor or a load balancer at it and expect HTTP 200. The Docker image uses it as its health check, every 30 seconds (wget -qO- http://localhost:3001/api/health), and the launcher uses it to decide whether an update came up.
The health check proves the process is running and answering. It does not test the database, the file store or email. A site with a broken database connection can still answer ok. To check those, log in to the admin panel.
What the log is
The server writes to the standard output and standard error of its process, and nowhere else. There is no log file, no log rotation and no log setting. LOG_LEVEL is read but changes nothing, and there is no debug level. Keeping, rotating and shipping the logs is the job of whatever runs the process (systemd, Docker, your platform).
Most lines start with a prefix in square brackets.
| Prefix | Who writes it | What it tells you |
|---|---|---|
[launcher] | The program that starts the server | Which version it is starting, and whether a downloaded release came up or was rolled back. |
[config] | Start-up | A problem with the settings, such as [config] LICENSE_KEY is not set: .... These are warnings, not errors; the site starts. The same list is printed by npm run config:check. |
[site.config.json] | Start-up | A value in your defaults file that the settings schema rejected, as [site.config.json] identity: .... |
[secrets] | Start-up | [secrets] 12 value(s) loaded from vault, or could not read NAME_FILE=.... |
[db] | Start-up | [db] schema is up to date (sqlite): the schema was applied (unless DB_AUTO_MIGRATE=false). |
[server] | The server | version 1.6.12 listening on http://localhost:3001 (production) when ready, shutting down when stopped, dist/ not found: serving the API only if the built site is missing, and failed to start: followed by the reason when it could not. |
[updates] | The updater | version X is unpacked and verified; restarting into it. |
[email] | The log email provider | Only with EMAIL_PROVIDER=log: the message that would have been sent, including login links. |
[sessions] | The Redis session store | redis error: ... when Redis cannot be reached. |
Lines without a prefix are errors from the code that did the work. The ones you are most likely to need:
| Line starts with | Meaning |
|---|---|
Update check failed: / Update download failed: / Update unpack failed: / Database backup failed: | An update step failed. See When an update fails. |
Resend rejected the email: (also SendGrid, Postmark, Mailgun, SES) | The provider refused a message, with its reason. See Nobody can log in, or emails do not arrive. |
SMTP send failed: | The mail server refused or could not be reached. |
Email sending failed: | Any other failure sending. |
Email not sent: EMAIL_PROVIDER is none / EMAIL_FROM is not set | Mail is switched off or incomplete. |
Member login link for ... not sent: email is off / Admin login link ... | A login link was made but could not be sent. |
turnstile verification failed: (or hcaptcha, recaptcha) | The bot check refused a token, with the provider's error codes. |
Contact form error:, Upload error:, Error installing update: | An unexpected error in that action. |
The log can contain email addresses (for example in login-link messages) and, when email is set to log, the text of the messages, including login links. Treat the logs as private.
![A terminal showing the start-up log with the launcher line, two [config] warnings, the [db] line and the [server] listening line](/img/shots/terminal-start-config-warnings.png)
Read the log on each platform
| Platform | Follow the log live | Look back |
|---|---|---|
| systemd | journalctl -u choirmaster -f | journalctl -u choirmaster --since "1 hour ago" |
| Docker Compose | docker compose logs -f app | docker compose logs --since 1h app |
| Node in a terminal | The terminal itself | Redirect it: node server/launcher.js >> choirmaster.log 2>&1 |
| Cloudflare Workers | npx wrangler tail | Cloudflare's dashboard, under the Worker's Logs |
Use your own service and Compose names. npx wrangler tail shows requests and log lines as they happen, and it can filter: --status error shows only failed requests, and --search "login" matches log text. The [config] warnings on Cloudflare are printed once each time the Worker starts, not on every request.
Start, stop and restart
| systemd (Node) | Docker Compose | Cloudflare Workers | |
|---|---|---|---|
| Start | sudo systemctl start choirmaster | docker compose up -d | Always running once deployed |
| Stop | sudo systemctl stop choirmaster | docker compose stop app | Remove the route, or npx wrangler delete to remove the Worker |
| Restart | sudo systemctl restart choirmaster | docker compose restart app | npx wrangler deploy again (new settings only take effect on a deploy) |
| Apply new settings | Restart | docker compose up -d (recreates the container if the file changed) | Change wrangler.toml or wrangler secret put, then deploy |
| Go back a version | See Update by hand | See Update a Docker site | npx wrangler rollback |
For the Docker SQLite stack add -f docker-compose.sqlite.yml to each docker compose command.
A Node site needs a restart for any change to its settings, because settings are read once at start-up. Settings changed in Site Settings in the admin panel (names, colours, wording) are in the database and take effect without a restart.
How the site stops
On SIGTERM or SIGINT (what systemctl stop and docker stop send), the server stops taking requests, closes the database, and exits. It gives up and exits after five seconds in any case. The launcher passes the signal on and exits when the server has.
If the server crashes, the launcher exits with its code. Restarting it is the job of whatever supervises it: set Restart=on-failure in a systemd unit and restart: unless-stopped in Compose (the product's compose files already do). A restart the server asks for itself, after an update, is handled by the launcher.