The AcruxCore Team

Most tool comparisons are written from docs and marketing pages. We didn't do that here. We ran nine LLM-ops platforms ourselves — LangSmith, Langfuse, PromptLayer, Arize Phoenix, Opik, MLflow, Helicone, and Laminar, plus our own AcruxCore as the baseline — and did the same thing on each one: create a prompt, version it, run it live with a real model key, inspect the resulting trace, and try to build an eval. Then we wrote a small script against each platform's own SDK and ran that too, and timed every one of them against a direct call to the provider.

Tracing and monitoring — what most people mean by "LLM observability tools" — is only one of nine angles below; prompt management, evals, guardrails, and tool-calling get the same hands-on treatment.

In the previous post we wrote the same tool-calling agent against a raw OpenAI-compatible client on both platforms and hand-rolled the loop. AcruxCore needed 105 lines to LangSmith's 63 — we lost that one, and said so.

This post is the other half: what happens when you use each platform's own abstraction instead of writing the loop yourself. LangChain's create_agent against the AcruxCore SDK's run_tool_loop. Same tool, same question, same model (openai/gpt-4o-mini via OpenRouter), both run live.

A tool call is where an agent stops being a text generator and starts touching the world. It is also where tracing earns its keep: when an agent gives a wrong answer, the question is almost never "what did the model say" — it is "what did the tool return, and how long did it take".

So we built the same tool twice and ran it for real. Same tool, same question, same model (openai/gpt-4o-mini through OpenRouter on both sides, so nothing hides behind a model difference). One run traced by LangSmith, one by AcruxCore. Then we did it again with streaming, which is where we found a bug in our own product.

Every number and every output below comes from a run we actually executed, and all three scripts are at the end of the post in full.

This is the AcruxCore leg of a hands-on comparison series. We ran the exact same steps — create a prompt, version it, generate a trace, try to build a dataset — against the hosted product ourselves, so the LangSmith, Langfuse, and PromptLayer write-ups have a fair baseline to compare against. No marketing framing here — just what the dashboard actually does. The main comparison post pulls the findings from all four platforms together.

Langfuse is one of the best-known open-source LLM engineering platforms — it covers prompt management, tracing (recording what an LLM app actually did, step by step), evaluation, and datasets. This is the Langfuse leg of a hands-on comparison series: we logged into Langfuse's hosted EU cloud with a real account and clicked through the whole loop ourselves — open a prompt, version it, inspect a real trace, and set up a dataset for testing. This post is a plain, factual account of what we found — the good and the rough edges. The main comparison post pulls the findings from Langfuse, LangSmith, PromptLayer, and AcruxCore together.

We logged into the real, hosted version of LangSmith (US region) and worked through the same four steps we're using across every platform in this comparison: create a prompt, run it and inspect the trace, and build a small eval dataset. This post is just the LangSmith leg — screenshots, actual UI, no marketing copy. The main comparison post pulls the findings from LangSmith, Langfuse, PromptLayer, and AcruxCore together.

This is a hands-on look at PromptLayer, one of the products we compare ourselves against. Instead of reading their docs, we logged into a real hosted PromptLayer account and did the same four things we do on every platform in this series: create a prompt, version it, run it and inspect the trace, and see what evaluation looks like. This post is just the PromptLayer leg — the main comparison post pulls the findings from all platforms together.

Putting a gateway in front of your model providers buys you a lot: one endpoint for every provider, cost accounting, caching, virtual keys, and budgets. But it raises an obvious worry — am I paying for that with latency? Every request now takes an extra hop, and for a user-facing app, milliseconds matter.

So we measured five ways to reach the same model — through our gateway, around it with your own key, and raw — to see exactly where the milliseconds go.

If you're building on LLMs, you eventually need four things: prompts you can change without redeploying, a reliable path to model providers, visibility into what each call did, and a way to tell whether a change helped. LangSmith and Acrux Core both address this space from different starting points. This is an honest look at where they overlap and where they differ.