Skip to content
XeroML XeroML

LLM Playground

The XeroML Playground is an interactive environment for testing prompts against real LLM models directly in the UI. Use it to validate prompt changes before updating the production label.

What You Can Do

  • Test prompt variables — fill in variable values and see the compiled prompt
  • Switch models — run the same prompt against different models for comparison
  • Adjust model parameters — change temperature, max tokens, and other settings
  • Compare versions — run the current and previous versions side by side
  • Save results — traces from Playground runs appear in your project’s trace list

Using the Playground

  1. Open a prompt in the XeroML UI
  2. Click Open in Playground
  3. Fill in any variable values
  4. Select a model (requires an LLM Connection configured in your project)
  5. Click Run

The response appears in the right panel. The Playground trace is automatically linked to the prompt version being tested.

Comparing Versions

To compare two versions:

  1. Open the Playground for a prompt
  2. Click Compare to open a split-panel view
  3. Select the versions to compare on left and right
  4. Run both — results appear side by side

Use this before deploying a new version to verify it produces better output on your representative test cases.

LLM Connections

The Playground uses LLM Connections configured in your project settings to make actual API calls. You can add connections for:

  • OpenAI
  • Anthropic
  • Azure OpenAI
  • Google Vertex AI
  • Ollama (for local models)
  • Any OpenAI-compatible endpoint

Go to Project Settings → LLM Connections to configure.