kristoo
Joined 6 years ago
Karma
64
Submissions
17
Member since
Feb 13
Recent submissions
Comment · 2 years ago
The benchmark result is for 100 concurrent users
Comment · 5 years ago
Thanks! Great question.
The seconds vary a lot on a task, model and input basis.
For example, if you do something quick like text vectorisation on a couple of sentences then it is less than 100ms per call. That would be 10,000, 50,000 and 200,000 calls (0.0005, 0.0002, 0.0001 per additional call) respectively.
On the other end, if using GPT-2 Large to generate around 40 words then that takes around 2500ms per call. Giving 400, 2000 and 8000 calls (0.0125, 0.005, 0.0025 per additional call) respectively.
Comment · 5 years ago
This is a PyTorch based library my team and I have been working on for the last few months, with the goal of making finetuning and using models as easy as possible for devs, even without extensive ML experience.
We've currently got support for text and image-based tasks (classification, generation, q&a, etc.), with wrappers around models like Google's T5, OpenAI's CLIP, GPT-2, Facebook's BART, and others.
We've got some features that make deployment easy, but for full transparency, it is through a paid platform [0] we've developed that is by no means necessary to use the library.
We're happy with the progress we've made, but we're curious to hear what people think so we can keep improving.
Comment · 5 years ago
Nvidia GPU support on Linux is great. I meant the lack of the support on Mac. Apps that are lacking and I frequently use off the top of my head is just Teams tbh.
But, as we know, Mac has great app support. The same level of polish and availability would be great. Just to have the options whenever the need arises. Currently finding good webapps is the best alternative.
Comment · 5 years ago
I have not. I use a MacBook Pro as my laptop.
Comment · 5 years ago
That would be a dream come true (if it had Mac level app support and additionally supported Nvidia GPUs). Probably next to impossible, unfortunately.
Comment · 5 years ago
Nice! Out of curiosity, did you build it to learn/practice the low level concepts of deep learning?
If so, was it worth it?
Comment · 5 years ago
I use PopOS as my OS on a high end PC and it is pretty good.
There are some things which I find annoying, such as Bluetooth headphones going to a headset mode at the start of calls. Just tiny things. Would I pay to have a smoother experience? Probably,
- I would like the OS to be as smooth as possible.
Although, I don't think that's the main problem with a Linux distro. Most of the friction still comes from third party apps, such as Teams (which I use for calls multiple times a week and it absolutely sucks).
Comment · 5 years ago
You might get some really promising results with finetuning.
If anything, you could build writing assistance that almost automates responses.
I've been co-authoring a library that lets you finetune such models in a single line of code.
https://github.com/backprop-ai/backprop
In specific the text generation finetuning example should be what you are looking for: https://github.com/backprop-ai/backprop/blob/main/examples/F...
Hope this helps, happy to chat more about it. Pretty curious about the results.
Comment · 5 years ago
We have exposed the large GPT 2 model as an API.
Anyone can access it and our pricing is usage based. If you use less than 1000 seconds of inference a month then it's completely free.