I really like the focus on “data and pure functions” from the beginning of the post.
I’ve read a lot on HN about how BEAM execution model is perfect for AI. I think a crucial part that’s usually missing in LLM-focused libraries is the robustness story in the face of node failures, rolling deployments, etc. There’s a misconception about Elixir (demonstrated in one of the claw comments below) that it provides location transparency - it ain’t so. You can have the most robust OTP node, but if you commit to an agent inside a long running process, it will go down when the node does.
Having clear, pure agent state between every API call step goes a long way towards solving that - put it in Mnesia or Redis, pick up on another node when the original is decommissioned. Checkpointing is the solution
I’ve read a lot on HN about how BEAM execution model is perfect for AI. I think a crucial part that’s usually missing in LLM-focused libraries is the robustness story in the face of node failures, rolling deployments, etc. There’s a misconception about Elixir (demonstrated in one of the claw comments below) that it provides location transparency - it ain’t so. You can have the most robust OTP node, but if you commit to an agent inside a long running process, it will go down when the node does.
Having clear, pure agent state between every API call step goes a long way towards solving that - put it in Mnesia or Redis, pick up on another node when the original is decommissioned. Checkpointing is the solution