Comment

Comment by simonw

I couldn't get the downloadable binary to work, or the binary I compiled myself:

  ./SwiftLM \
    --model mlx-community/Qwen3.5-122B-A10B-4bit \
    --stream-experts \
    --port 5413

Error:

  [SwiftLM] Loading model: mlx-community/Qwen3.5-122B-A10B-4bit
  [SwiftLM] Enabled Async SSD Streaming on directory: e9c67b08899964be5fdd069bb1b4bc8907fe68f5
  [SwiftLM]  Memory strategy: FULL GPU (69.6GB model, 133.4GB available)
  [SwiftLM] Download: [===================>] 100% ⠋ (66395.4 MB / 66395.4 MB) | Speed: 0.0 MB/s      
  MLX error: Failed to load the default metallib. library not found library not found library not found library not found  at /Users/runner/work/SwiftLM/SwiftLM/LocalPackages/mlx-swift/Source/Cmlx/mlx-c/mlx/c/stream.cpp:115

parent simonw 7 months ago

Comment by simonw

Discussion