Comment by simonw

Claude Code helped me figure out this recipe (inspired by a similar workaround in the CI scripts):

  git clone --recursive https://github.com/SharpAI/SwiftLM.git
  cd SwiftLM

  swift build -c release

  # Trick to copy in that missing mlx.metallib file
  uv run --with mlx-metal python -c "
  import importlib.metadata, pathlib, shutil
  d = importlib.metadata.distribution('mlx-metal')
  metallib = pathlib.Path(d._path).parent / 'mlx/lib/mlx.metallib'
  shutil.copy(metallib, '.build/release/')
  print(f'Copied {metallib} -> .build/release/mlx.metallib')

  # Now start the server (downloads 69GB Qwen model)
  .build/release/SwiftLM \
    --model mlx-community/Qwen3.5-122B-A10B-4bit \
    --stream-experts \
    --port 5413

But the server crashed when I tried to run a prompt through it:

  freed pointer was not the last allocation