What does the OpenInference code donation mean for AI tracing?

@arizeai’s @mikeldking and OTel contributor @ne_skazu discuss open source collaboration and GenAI semantic conventions on @Dynatrace’s Observation Deck.

Watch the episode: https://t.co/0hta6QY11d

TL;DR: Arize’s donation of OpenInference to OpenTelemetry marks a key step in GenAI observability’s shift from vendor-specific semantic conventions toward vendor-neutral standards.

OpenInference Donated to OpenTelemetry: What Will Change for AI Tracing?

This is a special conversation in Dynatrace’s live series, “Observation Deck.” Hosts Adriana Villela (from Toronto, Canada) and Katharina Sick (in Linz, Austria, today) invited two guests representing the donor and recipient to unpack the story behind the donation from both perspectives.

The guests are Mickey King from Arize, the product lead at the company behind OpenInference, and Ludmila Molkova from the OpenTelemetry side, who has been deeply involved in receiving the donation. As Katharina put it, when you hear the word “donation,” it sounds simple at a high level, but there must be plenty of stories behind it.

Updates from the Open Source Community

Adriana had just returned from Prague, which hosted the first Observability Summit Europe—an exciting event. Many OTel maintainers and practitioners were there, giving her a chance to meet the people she usually works with face to face and catch up at social events. These opportunities usually only arise at KubeCon, but anyone who has attended KubeCon knows how busy it is: nobody has time to chat. That same week, she also attended Open Source Summit Europe in Prague, which had a very different atmosphere, bringing together the broader open source community. Next, she would fly to Edinburgh to begin a speaking tour spanning three countries and multiple talks.

Katharina attended a local Cloud Native Days event and met quite a few new people. She is currently at home with her baby and does not travel much. She has been working on cloud native content and some Advent of Code-related material, putting together a small collection of cloud native projects. Adriana noted that some of the CNCF projects involved are less well known, making this a good opportunity to learn something new about cloud native technologies. Katharina’s takeaway: hands-on practice is the most effective way to learn.

Introducing the Guests

Mickey King is an experienced product professional who has launched many products, built communities, and persuaded community members to join and take an interest in Arize. He focuses on AI and platform work and was also a founding team member of a well-known project. Away from the screen, he is a mountain biker—so both personally and professionally, he really does operate “at altitude.” The hosts also joked that, since he is a climber, they would have to talk about bouldering.

Ludmila Molkova knows OpenTelemetry inside out: she is a member of the OpenTelemetry Technical Committee and a maintainer of its semantic conventions. As the hosts put it, if you have ever wondered who decides what your telemetry data should look like, she is one of the people behind it. Previously, she worked on observability for Azure client libraries at Microsoft, helped build distributed tracing in the .NET ecosystem, and wrote a book about it: “Modern Distributed Tracing in .NET.” Today, she is a senior software engineer at Google and is deeply involved in OpenTelemetry’s GenAI work.

What the OTel GenAI Instrumentation SIG Does

Ludmila explained that this is currently a relatively large initiative within OpenTelemetry. Its first goal is to define semantic conventions and provide the community with vendor-neutral instrumentation. The instrumentation is viewed more as a reference implementation, while the central focus is establishing consistent standards across the industry.

This field has historically been difficult to unify: there might have been three, four, five, or even 25 different ways to emit telemetry data. Now, that is converging toward roughly two or three approaches, with the OpenTelemetry format becoming increasingly popular. The SIG’s approach is to identify what can be unified and standardized, incorporate it into the formal specification, help people implement it, and then measure how closely they follow it.

To summarize in her words: essentially, we are here to make sense of this chaotic AI world from a telemetry perspective.

How Quickly Semantic Conventions Are Converging

Adriana interjected that she had given a talk in Prague the previous week about the convergence of GenAI semantic conventions. When she and her co-presenter first started preparing, semantic conventions for GenAI instrumentation were still highly divergent—they were looking at OpenLLMetry and OpenInference at the time. By the time they began using the GenAI normalizer, which helps convert other frameworks’ semantic conventions into GenAI semantic conventions, OpenLLMetry was already using many of those conventions. Then, the previous week, they discovered that OpenInference had a switch that could be enabled to comply with the GenAI specification. The pace of change in this field is remarkable.

Ludmila explained that one reason for the divergence is that startups and enterprises alike are moving extremely quickly in AI. If you work at a startup, or anywhere that needs to deliver a product to users, you cannot wait for consensus; you have to ship first and converge later. So the initial 25 versions may eventually shrink to just a few, and the hope is that OpenTelemetry will be the most widely adopted among them—which is also why she was there. Katharina agreed.

What OpenInference Is and Why It Has That Name

Mickey King started from the beginning: Arize was a startup trying to figure out how to make AI observable, and GenAI semantic conventions did not yet exist. He began working on open source around early 2022, and about two months later, they experienced the GPT-3 moment.

Tracing immediately became one of the most important priorities because, although large language models are extremely powerful, they are very unreliable when it comes to what goes into their context windows. The team recognized that LLMs have many different failure modes, involving not only the LLM calls themselves but also the retrieval steps that take place before content is fed into the context window.

Initially, they did not choose OTel because it did not seem like the right fit—it felt more like a DevOps tool. But about two months later, they looked ahead and realized that AI agents would be distributed, crossing network boundaries, infrastructure, and programming languages. They began talking to people who used Ruby and wanted to use the Arize platform, so building tracing capabilities around large language models became a natural decision. OTel had been a tremendous help when Mickey was at Apple, and it was equally helpful as Arize, a startup, tried to bring observability to its applications.

He acknowledged that, to some extent, their efforts overlapped with Ludmila’s work. They invested so heavily in OpenInference because AI was evolving so quickly. OpenInference was their way of exploring what things would look like when “reasoning models arrive” or “OpenAI launches the Responses API,” allowing Arize to deliver value to users and customers very quickly. Their efforts therefore focused more on instrumentation than semantics. Mickey sees OpenInference as one of the projects that helped make convergence around OTel less vague in the early stages, and helped make OTel a good vehicle for observing agents.

Has the AI World Slowed Down Enough for Everyone to Converge?

Mickey found this an interesting question. Agents are now running as services, making it more important to obtain these signals from vendors. He gave considerable credit to LiteLLM, LangChain, and others, as well as to the convergence taking place, seeing it as strong evidence that foundation models are moving in that direction. Distillation is, of course, a particularly tricky aspect, so he acknowledged that he might not have as complete a picture as he would like. But at least from his perspective, signs of convergence around OTel are indeed emerging.

Why Arize Decided to Make the Donation

Asked what prompted Arize to make the donation, especially given an earlier failed attempt to donate OpenLLMetry to OTel, Mickey offered a two-part answer.

His team’s mission is to make agents and large language models observable and evaluable, enabling people to use AI in the right way. OpenInference’s overarching goal is exactly the same as OTel’s: to make these systems observable and evaluable. From that perspective, Ludmila’s goals and his are 100% aligned at a higher level.

OpenInference was already an open source project. When discussing the donation, Mickey essentially told Ludmila outright: “Please just take the code. You can have it.” But the CNCF requires a formal donation and license-removal process. There was no hesitation on the donor’s side because their goals were aligned—this really was about helping users. For him, as for Ludmila, they had spent years discussing how to accelerate GenAI’s progress. The OpenInference donation fully expresses Arize’s commitment to advancing agent telemetry faster.

Considerations on the OpenTelemetry Side

Ludmila confirmed that, from OpenTelemetry’s perspective, this was a clear choice: OTel wants instrumentation that it can ensure is vendor-neutral. Customers of companies involved in OpenTelemetry may be concerned about depending on something that is not vendor-neutral, and they want a say in how it evolves.

But OTel needs something to build on, and having OpenInference’s code is extremely helpful: “We know where the gaps are, and we know how to address them. We already have something that someone else has actually driven forward.”

She also mentioned that she was very curious about Arize’s development process: how the team quickly dives into new APIs, figures them out, and makes decisions without getting stuck on questions like “How good is it?” or “How long will it last?”—while still making very good decisions. At the end of her remarks, Ludmila also emphasized that she should not be seen as the main driving force behind OpenTelemetry’s GenAI semantic conventions. She represents the community, and many people have poured tremendous effort into the work.

Source: https://www.youtube.com/watch?v=FSrw6Fnmj1Q