Show HN: Deploy Hugging Face Models to AWS Lambda

I've been working on Scaffoldly since 2020 to simplify AWS Lambda deployments. Recently discovered you can run Hugging Face models efficiently using EFS for caching. Here's what's interesting:

   - Uses EFS for model file persistence
   - Pre-downloads models after deployment for faster cold starts
   - Cold start: ~20s (model loading), warm requests: 5-20s (CPU inference)
   - Fully automated container builds and deployment
   - Works with private/gated models via HF_TOKEN
Example deployment:

  npx scaffoldly create app --template python-huggingface
  cd python-huggingface && npx scaffoldly deploy
Scaffoldly is Open Source and I'm excited for all feedback and contributions from the community!

https://github.com/scaffoldly/scaffoldly

https://github.com/scaffoldly/scaffoldly-examples/tree/pytho...

1 points cnuss 1 year ago 0 comments

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton