juliensalinas

I use KWatch.io (https://kwatch.io) for social listening and it works very well for HN monitoring in my case. They also support other platforms (Reddit, Linkedin, Twitter..). But they don't propose advanced features like dashboards, analytics...

Comment · 2 years ago

Many are trying to install and deploy their own LLaMA 3 model, so here is a tutorial I just made showing how to deploy LLaMA 3 on an AWS EC2 instance: https://nlpcloud.com/how-to-install-and-deploy-llama-3-into-...

Deploying LLaMA 3 8B is fairly easy but LLaMA 3 70B is another beast. Given the amount of VRAM needed you might want to provision more than one GPU and use a dedicated inference server like vLLM in order to split your model on several GPUs.

LLaMA 3 8B requires around 16GB of disk space and 20GB of VRAM (GPU memory) in FP16. As for LLaMA 3 70B, it requires around 140GB of disk space and 160GB of VRAM in FP16.

I hope it is useful, and if you have questions please don't hesitate to ask!

Julien

Comment · 2 years ago

How does this solution compare to platforms like https://kwatch.io or brand24 for hacker news monitoring?

Does it monitor hacker news in real time?

Comment · 2 years ago

We tried hard to move some of our inference workloads to TPUs at NLP Cloud, but finally gave up (at least for the moment) basically for the reasons you mention. We now only perform our fine-tunings on TPUs using JAX (see https://nlpcloud.com/how-to-fine-tune-llama-openllama-xgen-w...) and we are happy like that.

It seems to me that Google does not really want to sell TPUs but only showcase their AI work and maybe get some early adopters feedback. It must be quite a challenge for them to create a dynamic community around JAX and TPUs if TPUs stay a vendor locked-in product...