newhouseb
Joined 17 years ago
Karma
2,321
Submissions
475
Member since
Feb 16
Recent submissions
Comment · 2 years ago
A more extreme version of the ADC/DAC projects mentioned here is my (old) project that built a Bluetooth transceiver using just an FPGA and an antenna (and a low pass filter if you don't want the FCC coming after you): https://github.com/newhouseb/onebitbt
Tl;dr: if you blast in/out the right 5GHz binary digital signal from a SERDES block on an FPGA connected to a short wire you can talk to your phone!
Comment · 3 years ago
The short but unfulfilling answer is that this is because the DOCSIS standard historically has allocated a much broader frequency range for DL than UL. Unlike other forms of communication (like cell) that can use similar frequency ranges for transmit and receive, DOCSIS tends to slice something like < 70Mhz for UL and >70Mhz to 1Ghz for DL (I'm probably remembering the details incorrectly, trust Google over me!). Switching the frequency ranges often requires different circuitry and therefore different hardware.
I would guess that unlike fiber -- rarely saturated in a consumer context -- people have ~always wanted more than what cable can provide and thus the operators needed to be strategic about the allocation of bandwidth between DL and UL, hence the asymmetry.
Comment · 3 years ago
Huge fan of RASP et al. If you enjoy this space, might be fun to take a glance at some of my work on HandCrafted Transformers [1] wherein I hand-pick the weights in a transformer model to do long-handed addition similar to how humans learn to do it in gradeshcool.
[1] https://colab.research.google.com/github/newhouseb/handcraft...
Comment · 3 years ago
I'm a little skeptical of the paper in question: https://arxiv.org/pdf/2301.00250.pdf
If you look at the training and test set info:
> We report results on two protocols: (1) Same layout: We train on the training set in all 16 spatial layouts, and test on remaining frames. Following [31], we randomly select 80% of the samples to be our training set, and the rest to be our testing set. The training and testing samples are different in the person’s location and pose, but share the same person’s identities and background. This is a reasonable assumption since the WiFi device is usually installed in a fixed location. (2) Different layout: We train on 15 spatial layouts and test on 1 unseen spatial layout. The unseen layout is in the classroom scenarios.
Depending on how they selected various frames -- let's just say it was random -- the model could have learned something to effect of "this RF pattern is most similar to these two other readings I'm familiar with" (from the surrounding frames) and can therefore just interpolate between the resulting poses associated with those RF patterns (that the model has compressed/memorized into trained weights).
If you look at the meshes between the image ground truth and the paper's results, you'll see that they are strikingly similar. I find this also suspect because WiFi-band RF interacts a lot more with water than with clothes and so you would expect the outline/mesh to get the "meat-bag" parts of you correct but not be able to guess the contours of baggy clothes. That is... unless it has memorized them from the training set.
Comment · 3 years ago
Right, there are many folks (dozens of us!) yelling about logit processors and building them into various frameworks.
The mostly widely accessible form of this is probably BNF grammar biasing in llama.cpp: https://github.com/ggerganov/llama.cpp/blob/master/grammars/...
Comment · 3 years ago
You still charge through the USB-C / Thunderbolt ports (and use a USB-C to USB-C cable with the provided power brick to do so). The MagSafe plug is just for convenience (and perhaps slightly higher wattage than is normally specced for USB-C on the beefier MBPs).
Comment · 3 years ago
An example from an earlier comment of mine on a different thread (assuming I've understood correctly):
> let's say we had a grammar that had a key "healthy" with values "very_unhealthy" or "moderately_healthy." For broccoli, the LLM might intend to say "very_healthy" and choose "very" but then be pigeonholed into saying "very_unhealthy" because it's the only valid completion according to the grammar.
That said, you can use beam search to more or less solve this problem by evaluating the joint probability of all tokens in each branch of the grammar and picking the one with the highest probability (you might need some more nuance for free-form strings where the LLM can do whatever it wants and be "valid").
Comment · 3 years ago
I think this is likely a consequence of a couple of factors:
1. Fancy token selection w/in batches (read: beam search) is probably fairly hard to implement at scale without a significant loss in GPU utilization. Normally you can batch up a bunch of parallel generations and just push them all through the LLM at once because every generated token (of similar prompt size + some padding perhaps) takes a predictable time. If you stick a parser in between every token that can take variable time then your batch is slowed by the most complex grammar of the bunch.
2. OpenAI appears to work under the thesis articulated in the Bitter Lesson [i] that more compute (either via fine-tuning or bigger models) is the least foolish way to achieve improved capabilities hence their approach of function-calling just being... a fine tuned model.
[i] http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Comment · 3 years ago
Also important to call out that anytime you have a freeform string it's pretty much an open invitation for the LLM to go completely haywire and run off into all sorts of weird tangents. So these methods are best used with other heuristics to bias sampling once you get to free-form text territory (i.e. a repetition penalty etc)
Comment · 3 years ago
The way LLMs work is they output probabilities for every _token_, so you don't really need to backtrack you can just always pick a token that matches the provided grammar.
That said, you might want to do something like (backtracking) beam-search which uses various heuristics to simultaneously explore multiple different paths because the semantic information may not be front-loaded, i.e. let's say we had a grammar that had a key "healthy" with values "very_unhealthy" or "moderately_healthy." For broccoli, the LLM might intend to say "very_healthy" and choose "very" but then be pigeonholed into saying "very_unhealthy" because it's the only valid completion according to the grammar.
That said, there are a lot of shortcuts you can take to make this fairly efficient thanks to the autoregressive nature of (most modern) LLMs. You only need to regenerate / recompute from where you want to backtrack from.
Comment · 3 years ago
I think you mean "run as part of the sampler," the tokenizer (and tokenization) is fixed for a given model. The sampler blob would basically:
1. Modify the output token probabilities to fit any arbitrary use case
2. Perhaps do trigger some sort of backtracking / beam-search
(I'm not Grant but we've chatted on twitter and built similar things)
Comment · 3 years ago
I've been trying to wrap my head around this and my layman understanding is that there's an assumption (but maybe not baked into any requirements/standard) that use of the hardware key is locked behind either a biometric check (FaceID/TouchID/etc) or password. In other words, there might be an implicit second factor baked in to the passkey itself.
Comment · 3 years ago
Modern LTE/5G provides pretty accurate location out of the box without GPS so if you really wanted to hide you would really need to just shut off the entire modem. Fortunately, the PinePhone has hardware switches to do this!
Comment · 3 years ago
This is great! I built a house and DIY'ed this because our site was 2 hours away from where we were living at the time.
It was clear (our) GC was not used to this because they were constantly telling us that things were happening when they very clearly weren't (thanks to the live video we had from to the site).
Comment · 3 years ago
AidKit | Remote | Full-time | https://aidkit.org | $130k-180k + equity | Social Impact / GovTech | TypeScript
AidKit runs the largest guaranteed income programs in the country. We replace convoluted workflows of glued together spreadsheets, bank portals and human tedium with one unified platform to allow organizations that do good, to do so faster and more effectively. AidKit is in many ways a technological petri dish to explore how we can bring the leverage of a bespoke software team to those without a software background through domain modeling and low-code tooling.
We're looking for an Engineering Manager to help AidKit's engineering organization grow from a tight knit team of folks always jumping in to do what it takes to one where everyone has the space and support needed to do their most leveraged work. Full post here: https://aidkit-inc.rippling-ats.com/job/637223/engineering-m...
We're bootstrapped and profitable.
If you're interested, please reach out to me at [email protected]!
[ my public key: https://keybase.io/benzn; my proof: https://keybase.io/benzn/sigs/ijTuK7E61Do30-K2liLR8gjVuthKb_5Z0_z0m4kwRKM ]