Graphsignal: Inference Profiler

Graphsignal is a production-scale inference profiling platform that helps engineers optimize AI performance across models, engines, GPUs, and other accelerators. It provides essential visibility across the inference stack, including:

• Continuous, high-resolution profiling timelines exposing operation durations and resource utilization across inference workloads.
• LLM generation tracing with per-step timing, token throughput, and latency breakdowns for major inference frameworks.
• System-level metrics for inference engines and hardware (CPU, GPU, accelerators).
• Error monitoring for device-level failures and inference errors.
• Inference telemetry for AI agents to identify bottlenecks and drive targeted improvements across the inference stack.

Learn more at graphsignal.com.

Install

UV_TOOL_BIN_DIR=/usr/local/bin uv tool install 'graphsignal[cu12]'   # CUDA 12.x
# or
UV_TOOL_BIN_DIR=/usr/local/bin uv tool install 'graphsignal[cu13]'   # CUDA 13.x

Alternative: install into your workload environment

If you prefer a single environment, or you use the graphsignal.watch() Python API (which requires graphsignal importable by your application), install it directly into your workload's environment instead:

pip install 'graphsignal[cu12]'   # CUDA 12.x
# or
pip install 'graphsignal[cu13]'   # CUDA 13.x

Profile

Wrap your launch command with graphsignal-run:

export GRAPHSIGNAL_API_KEY=<my-api-key>
graphsignal-run vllm serve <model> --port 8001

Environment variables read by the profiler:

Sign up for a free account at graphsignal.com; you'll find the API key in Settings / API Keys.

See the Profiler CLI reference for the full set of options.

Applications that bootstrap themselves can call graphsignal.watch() from Python instead — see the Profiler API reference.

See integration documentation for libraries and inference engines:

• PyTorch
• vLLM
• SGLang

Optimize

Log in to Graphsignal to monitor and analyze your application.

Optimize with AI

Install the Graphsignal skill to let your AI coding agent (Claude Code, Codex, or Gemini) fetch and analyze signal context directly from your agent. See AI Optimization for setup instructions.

Overhead

The profiler has minimal impact on production performance. CUDA kernel activity is collected via CUPTI with low-overhead APIs, and analysis and upload happen in the sidecar process.

Security and Privacy

The profiler only establishes outbound connections to api.graphsignal.com to send data; inbound connections or commands are not possible.

Content and sensitive information, such as prompts and completions, are not recorded.

Troubleshooting

If something doesn't look right, report it to our support team via your account.

In case of connection issues, please make sure outgoing connections to https://api.graphsignal.com are allowed.