realPtolemy
Joined 3 years ago
Karma
46
Submissions
29
Member since
May 20
Recent submissions
Comment · 16 hours ago
Comment · 19 hours ago
Do you think we'll return to the concept of time-sharing?
I personally believe that it'll be normalized that mid-to-large size companies will build their own stack of compute for local ai model inference (some already have).
As we'll see more companies stacking hardware that isn't utilized to 100% at all times, do you believe that we'll see cross-company agreements to share this compute with other companies, for a given fee (i.e. the old school concept of time-sharing.)?
Comment · 8 months ago
I recall that my old German teacher taught us that listening to Music in a particular language, and watching TV where they speak a language, were the two best ways to learn a languages.
Her reason for why: Context and various slang words are grasped much quicker compared to the cumbersome process of repeating of words and phrases (She did not omit the need of the latter though).
She was great, 60 years old at the time and had us repeat the lyrics of Rammstein songs in class, her favorite band.
Comment · 1 year ago
I love CSV
Comment · 2 years ago
Or Elixir! Quite easy to grasp as well.
Comment · 2 years ago
Thanks, I must have read through the document to hastily.
Comment · 2 years ago
Could there also be a “legal hedging” reason for why you would release a paper like this?
By reaffirming that “we don’t know how this works, nobody does” it’s easier to avoid being charged with copyright infringement from various actors/data sources that have sued them.
Comment · 2 years ago
Indeed, and the very last section about how they’ve now “open sourced” this research is also a bit vague. They’ve shared their research methodology and findings… But isn’t that obligatory when writing a public paper?
Comment · 2 years ago
I’m curious to know why they don’t support Elixir and Haskell to a greater extent.
Comment · 2 years ago
How is it compared to 7B LLaMA quantized to run on a raspberry pi?
Comment · 2 years ago
They can already do that with some simple googling.
Comment · 2 years ago
I’m running Manjaro on a late 2012 MacBook Pro, it’s running flawlessly.
The only initial issue I had was that I had to configurate the broadcom WiFi driver manually. Super easy after some googling for the more technically interested person… But nothing I would expect the general person to solve without frustration.
8gb RAM, 125gb SSD. 1xBattery replacement.
It does not run IDE’s like ST32 (based on Eclipse) and OneNote via Firefox smoothly simultaneously. But everything else so far has worked flawlessly. I use it every day and have done so since the beginning of this fall.
Edit: Call me a script kid all you want, for this particular laptop it’s just more convenient for me to run Manjaro rather than Arch right of the bat. But I guess I could eventually set up a shared partition for storage and an individual partition for each OS.
Comment · 2 years ago
I use it multiple times per week, more than I use Instagram and most other apps.
VPS well, yes - but also no, you don't rent your self in on Costcos servers. The "cloud era" arose because the cost of VPS became so much cheaper than having all servers stacked in-house, and the fact that they provide easier maintenance and off-loads various data security risks that a regular company would have difficulties to comply with. I.e. higher up-time, lower latency for customers, less data security risks, less maintenance, lower cost.
But what we see with GPUs and AI driven compute, it can effectively be done locally, at a cheaper cost depending on the quantity of compute, requires little maintenance, the output can be offloaded on a cloud hosted VPS, the customer interface can still be routed through a cloud hosted VPS, but from a data security perspective you may deem it to be less risky to run local models on local compute.
I guess you could see XAI's (XSI's?) deal with Anthropic for compute as a sort of time-sharing, but I expect us to see these kind of deals being brokered between "non-tech" companies - unless token costs fall dramatically and unless it'll be truly ensured that all token processing is conducted in fully encrypted and closed of environments that are not used for training at all.
We're currently seeing a massive compute bottleneck - GPUs are expensive to buy and people/corps want to push as many tokens as possible into various models. Hence - Timesharing: A Solution to Computer Bottlenecks https://www.youtube.com/watch?v=Q07PhW5sCEk
Perhaps OpenAI's biggest moat in the future will not be that they provide an AI model, but rather that they facilitate a time-share structure for its clients. They will still be able to provide the most cutting edge model, but companies themselves will be able to run local inference on their own compute.