gbickford

Small models don't "know" as much so they hallucinate more. They are better suited for generations that are based in a ground truth, like in a RAG setup.

A better comparison might be Flash 2.0 vs 4o-mini. Even then, the models aren't meant to have vast world knowledge, so benchmarking them on that isn't a great indicator of how they would be used in real-world cases.

It's for visualizing datasets where fine-grained cluster details and broader relationships matter. There are example renderings in the paper.

From the README:

> PaCMAP (Pairwise Controlled Manifold Approximation) is a dimensionality reduction method that can be used for visualization, preserving both local and global structure of the data in original space. PaCMAP optimizes the low dimensional embedding using three kinds of pairs of points: neighbor pairs (pair_neighbors), mid-near pair (pair_MN), and further pairs (pair_FP).

> Previous dimensionality reduction techniques focus on either local structure (e.g. t-SNE, LargeVis and UMAP) or global structure (e.g. TriMAP), but not both, although with carefully tuning the parameter in their algorithms that controls the balance between global and local structure, which mainly adjusts the number of considered neighbors. Instead of considering more neighbors to attract for preserving glocal structure, PaCMAP dynamically uses a special group of pairs -- mid-near pairs, to first capture global structure and then refine local structure, which both preserve global and local structure. For a thorough background and discussion on this work, please read our paper.

> Create Hello World application:

> sudo mgrg -i -u $(whoami) helloworld

Why does it need sudo to compile?

This article is from 2017

It's always disappointing when people publish things to GitHub without the intention of collaborating or sharing.

I couldn't find any training code in the MXL examples.

There's gotta be somewhere in the middle. Vercel's movements feel a lot like the "Embrace, extend, and extinguish" playbook.

Maybe there is a class of developer out there that doesn't get spooked by that but it definitely has created an adversarial place for Vercel in my mind. I feel like I need to be careful when touching anything Vercel have touched so that I don't fall into a trap.

Llama.cpp is an inference engine. The author of llama.cpp designed gguf. Funcionary is a model that does function calling. You can download functionary weights in the gguf format and then run it using llama.cpp on low-end machines using CPU or GPU or a mix of both.

Have you tried generating two sets of qapairs, one with bad answers, and using DPO?