OTP and Gleam

**TL;DR:** Speaker Louie (creator of Gleam) explains in depth what OTP is, how Gleam works with OTP, and how to achieve type-safe inter-process communication, ultimately proving that Gleam can fully leverage the magic of OTP while retaining the benefits of static types. ## Introduction Hi everyone, thank you very much. Today is an exciting day — we have a full day of Gleam talks, the whole Gleam track. This talk is called "OTP and Gleam," and I want to show how these two important things work together, explain whether they can work together, what problems and challenges exist, and dig into the internals to understand why certain decisions were made, because that context helps you better understand the system. I'm Louie, I created Gleam, so they asked me to give this talk. ## What is OTP? The name OTP (Open Telecom Platform) doesn't really mean much on its own, and when you first start with BEAM, it's hard to accurately understand what it really is. I think the best way to understand it is to step back a bit and look at BEAM as a whole. BEAM gives us (and likes to boast about) features: massive scalability, unmatched reliability, production observability, reliable low latency, and distributed computing. But if you think about it, BEAM doesn't directly give us these features — it gives us **messages**, it gives us **error isolation**. You can see the relationship between these things, but just having a primitive doesn't necessarily mean you have the other feature. Our job is to take these primitives, these useful tools, and turn them into a system that has the features we as BEAM developers like to boast about. If you start building from these small building blocks, it's easy to make mistakes, especially when dealing with things like concurrency. So, I think this is the real meaning of OTP: OTP is a bunch of code that ships with your Erlang distribution, which leverages those BEAM primitives to implement a set of very useful design patterns, providing us with a series of building blocks we can use in our programs: supervisors, generic servers, etc. These implementations are built by brilliant people and have been thoroughly tested and used for nearly 30 years, so we know we can trust them. The last part of OTP is a set of protocols, a set of interfaces. Things that implement these interfaces can plug into the larger OTP system, such as special processes that respond to system messages, modules that implement callbacks for a certain module behavior, etc. This is my understanding of OTP. **OTP is the key to the magic of BEAM** — when we like to boast about BEAM's features, we might actually be boasting about OTP. This is true whether you write Erlang or Elixir. ## Problem: Can Gleam use OTP? Someone might say on forums: "Oh, you can't use OTP in Gleam because of types." And types are somehow inherently bad for things like actors? That can't possibly work. But the fact is, **it works perfectly**. We need to make it very clear to people that these things work together excellently. We won't do it exactly the way Erlang and Elixir do, because it's a different language, but it's there, and we can use the same framework. ## How OTP is implemented in Gleam OTP is implemented in Gleam as two packages; it's not part of the language (similar to Erlang and Elixir). There's an additional reason for the separation in Gleam: we also need to run on JavaScript, and JavaScript doesn't have those BEAM primitives to build OTP on. The goals of these two packages: - **Production-ready and efficient**: We want people to build real things, not academic projects. - **Fully compatible with Erlang OTP**: We don't make a standalone Gleam OTP; it's a window from Gleam into existing OTP. Gleam programs can leverage all the good things created by previous BEAM developers, and conversely, things built in Gleam can be used by Elixir friends. - **Provide core building blocks**: supervisors, generic servers, etc. - **Static type safety**: The type system magic shouldn't disappear when dealing with concurrency and actors. - **Expressive and capable**: Programs writable in Erlang should also be writable in Gleam. Not something strictly weaker, but equivalent. - **Long-term sustainable**: OTP is nearly 30 years old, and Gleam should also last forever like the VM. ## How to achieve type safety? Two approaches ### Approach 1: Reuse existing API (example: supervisor) Take the static child process creation supervisor as an example. The Gleam API works very similarly to Erlang, just omitting the child spec list and using pipe syntax. The semantics are exactly the same, and we can implement one using the other: import the `start_link` function (private), then put a nice wrapper that converts the data structures Gleam expects into the shapes Erlang expects. **Erlang is already good, we don't have to reinvent the wheel** — use it if it works. ### Approach 2: Redesign the API (example: generic server) Sometimes the API in some ways doesn't fit Gleam and can't be easily masked with small functions. For example, the `handle_call` callback: in Erlang, a case expression can return an integer or a boolean, but Gleam doesn't allow case branches to return values of different types, nor does it have union types for integers and booleans. So we must design a different API. We have to implement Gleam OTP in Gleam on top of the same primitives as Erlang. The core primitives are six: `spawn`, `link`, `trap_exit`, `monitor`, `send`, and `receive`. The first four are easy to import or wrap. **The biggest problem is `send` and `receive`**: if we directly import Erlang's design, there is no relationship between the receiver and the message type; any process can send any type of message to any process, which is not the type safety we want. We want to only send messages to processes that accept them. A common attempt is parameterized PID: add a message type parameter (like `PID(Int)`), but this is not flexible enough — the same PID needs to be able to receive different types of messages (like temperature and a reply about whether it's raining), but the reply types differ, and the PID cannot encode these semantics. In reality, Erlang has already solved this problem. Looking inside the `gen` module reveals the `from` data structure, which attaches semantic identifying information to individual messages in the process's inbox, used for matching "this is a reply to what." We can adopt the same design, faithfully translate it to Gleam, add a type parameter (message type), and rename `from` to `subject`, `reply` to `send` — the concept already exists. This gives us a way to send typed messages. For receiving from multiple different subjects simultaneously, we can use a `selector`, e.g., receive a string message and an integer message, then map the integer to a string. All principles are there. Is this enough to implement gen_server? gen_server does a few things: synchronous initialization (succeed or fail and pass value back to parent), hold and update state, handle system messages (for observer debugging), handle synchronous and asynchronous messages (main business logic). **We have the same six primitives as Erlang, and can directly copy Erlang's implementation** — open the Erlang source code, translate it directly, only making necessary type adaptations. ## Conclusion Gleam can fully use the magic of OTP while maintaining static type safety. Through clever design (like subject/selector), we gain the benefits of the type system without sacrificing expressiveness. Gleam OTP is fully compatible with Erlang OTP and can continue to evolve on the same timescale. **Source:** https://www.youtube.com/watch?v=DJwxAJoUUOU