backend-dev-33
Joined 11 years ago
Karma
8
Submissions
25
Member since
Jan 18
Recent submissions
Comment · 1 year ago
Comment · 2 years ago
> additional challenge of offering it over a network socket across multiple requests and doing so in a performant manner.
@kkielhofner thanks a lot! now I realize it. I see, there is even GRPC support in Triton, so it make sense.
Comment · 2 years ago
I was just told about this thing:
https://aws.amazon.com/ec2/instance-types/g4/
one NVIDIA T4 GPU, 16 GB RAM, and, this is an EC2 instance, it means "install anything" all this for $0.526 /Hour
do you see any hidden gotchas?
Comment · 2 years ago
UPDATE: how to do the same classification task using some hosting provider with GPU?
Let us discuss it here -> https://news.ycombinator.com/item?id=41768088
Comment · 2 years ago
Thanks @Terretta
Well, the categories I use - do not overlap at all with the list of 1092 categories in Google Content Categories.
> it handles other classifications as well
hm... I highly doubt that. First of all - I do not see API to upload list of MY categories. Second: Can somebody with Google Cloud account try it? I have no account and when creating it - it asks for credit card...
Comment · 2 years ago
thanks for Text Embedding Inference (never heard about it before)
> you can always run a zero shot pipeline in HF with a simple Flask/FastAPI application.
Yeah, sometimes things that are right in front of your nose, you don't see them. you mean this? https://huggingface.co/docs/api-inference/index
Comment · 2 years ago
Well, looking at Triton Inference Server + OpenVINO backend [1]...uff... as you said: "significant amount of development effort". Not easy to handle when you do it first time.
Is ONMX runtime + OpenVINO [2] a good idea ? Seems easier to install and to use: Pre-built Docker image and Python package... Not sure about performance (the hardware-related performance improvements - they are in OpenVINO anyway, right?).
[1] https://github.com/triton-inference-server/openvino_backend
[2] https://onnxruntime.ai/docs/execution-providers/OpenVINO-Exe...
Comment · 3 years ago
is it possible to use SetFit through some command-line interface of some API?
Comment · 7 years ago
And here Part1 as link:
https://news.ycombinator.com/item?id=20256681
P.S. I am not the author: just found the link in the Llama 4 thread.