brucethemoose2

Some phones have limiters to keep the battery at 60%-80%.

I believe most can do this with the right software.

It's not a fix, but it should extend the life considerably.

Another benefit: modern smartphones have large GPUs, large media blocks, and fast RAM.

With the right software, they can be a surprisingly powerful AI host or transcoding server.

Real world GPU performance is hugely influenced by hand optimization of the CUDA kernels.

Power/Weight is extremely high. A tiny wankel will do the job, and weight is everything on cars.

It does prefer a narrow RPM band, which is fine.

Reliability is the biggest concern TBH, but maybe that's not a huge bummer if its more of a backup/assistant engine.

Yeah, its an unspoken but rampant thing in the llm community. Basically no one respects licenses for training data.

I'd say the majority of instruct tunes, for instance, use OpenAI output (which is against their TOS).

But its all just research! So who cares! Or at least, that seems to be the mood.

Yeah I know, hence its odd I found it kind of dumb for personal use. Moreso with the smaller models, which lost an objective benchmark I have to some Mistral finetunes.

And I don't think I was using it wrong. I know, for instance, the Chinese language models are funny about sampling since I run Yi all the time.

I would note the actual leading models right now (IMO) are:

- Miqu 70B (General Chat)

- Deepseed 33B (Coding)

- Yi 34B (for chat over 32K context)

And of course, there are finetunes of all these.

And there are some others in the 34B-70B range I have not tried (and some I have tried, like Qwen, which I was not impressed with).

Point being that Llama 70B, Mixtral and Grok as seen in the charts are not what I would call SOTA (though mixtral is excellent for the batch size 1 speed)

The conspiracy theorist in me says thats a low priority due to perverse incentives (namely selling more storage at a huge markup).

Another rationale is that the Apple ecosystems tends to not use JPEG by default anyway, right? It uses HEIC or something.

Being a "hero" open source dev for a project like that can require a lot of neuroticism.

Sometimes it works, but sometimes the project is just too big, I think.

It's not either or, you can use different vendors for different tasks.

tinygrad isn't in the realm of production ready though, AFAIK.

The MI300 is the best accelerator you can buy, for many current workloads.

It's technically way more advanced. Not as outrageously priced as an H100 either.

I think you are preaching to the choir, and AMD is not listening.

AMD would be selling 48GB 7900s or AI-only W7900s if they really wanted a consumer card ramp.

They don't. Not because they can't (they literally prevent OEMs from doing so, who would double up VRAM in a heartbeat without AMD lifting a finger), but because AMD doesn't want that.

I never followed Hotz, so perhaps I missed something cool. But I never understood the hype myself.

Well, personally, SDXL just blows 1.5 out of the water for me. I haven't had a reason to even touch 1.5 in months.

But note that SDXL is really awful in automatic1111 or vanilla HF diffusers for me. You have to use something with proper augmentations (like ComfyUI or Fooocus(which runs on ComfyUI)).