Skip to content
XeroML XeroML

Self-Hosting Troubleshooting & FAQ

Application Won’t Start

Check required environment variables

The most common cause is a missing required variable. Check the logs:

Terminal window
docker compose logs web
# or
kubectl logs deployment/xeroml-web -n xeroml

Look for errors mentioning DATABASE_URL, NEXTAUTH_SECRET, or SALT.

Check database connectivity

XeroML requires a reachable PostgreSQL instance. Test the connection:

Terminal window
docker compose exec web npx prisma db ping

If it fails, verify the DATABASE_URL is correct and the database is reachable from the container’s network.

Run migrations

If the database exists but the schema is missing, run migrations manually:

Terminal window
docker compose exec web npx prisma migrate deploy

Traces Not Appearing

Verify the base URL

Your SDK’s XEROML_BASE_URL must point to your self-hosted instance, not XeroML Cloud:

Terminal window
XEROML_BASE_URL="https://xeroml.yourdomain.com"

Check ingestion endpoint

Test that the ingestion endpoint is reachable:

Terminal window
curl -u "pk-xm-...:sk-xm-..." https://xeroml.yourdomain.com/api/health

Check the worker

Traces are processed by the worker service. If the worker is down, traces may queue up or be dropped:

Terminal window
docker compose logs worker

High Memory Usage

Increase container memory limits

The web and worker containers may need more memory under load. In Docker Compose:

services:
  web:
    mem_limit: 2g
  worker:
    mem_limit: 1g

Enable ClickHouse for large datasets

PostgreSQL is not optimized for analytics queries over millions of traces. If you have high trace volumes, configure ClickHouse:

Terminal window
CLICKHOUSE_URL=http://clickhouse:8123

Timezone Display Issues

If timestamps in the UI appear in the wrong timezone, verify your server timezone:

Terminal window
# In Docker Compose
services:
  web:
    environment:
      TZ: UTC

XeroML stores all timestamps in UTC and converts to the user’s browser timezone in the UI. Server timezone does not affect stored data.

Upgrade Issues

Background migrations

Some upgrades include long-running database migrations that run in the background. Check migration status:

Terminal window
docker compose logs worker | grep -i migration

Do not roll back while a background migration is running.

Version compatibility

Check the changelog before upgrading to ensure your SDK versions are compatible with the new platform version.

FAQ

Can I use XeroML without ClickHouse?

Yes. For smaller deployments, XeroML stores traces in PostgreSQL. ClickHouse is recommended for production deployments with >100K traces/day.

How do I reset the admin password?

Use the headless initialization API:

Terminal window
curl -X POST https://xeroml.yourdomain.com/api/admin/reset-password \
  -H "X-Admin-Secret: <XEROML_ADMIN_SECRET>" \
  -d '{"email": "admin@example.com", "newPassword": "new-password"}'

What ports does XeroML use?

  • 3000 — Web application (HTTP)
  • 5432 — PostgreSQL
  • 6379 — Redis
  • 8123 — ClickHouse HTTP
  • 9000 — ClickHouse TCP (internal)

Is XeroML stateless?

The web application is stateless and can be scaled horizontally. The worker processes background jobs and should run as a single instance per project (or use distributed locking if scaling).

Getting Help