jah242
Joined 6 years ago
Karma
102
Submissions
37
Member since
May 23
Recent submissions
Comment · 3 months ago
I think an under-discussed aspect of the frontier-versus-commodity intelligence debate is that the rate at which real economic tasks adapt in complexity may be just as important as, if not more important than, the lag between open and closed model capability.
Comment · 4 months ago
I wrote this up while building Sonto.
Essentially, I think the discussion around AI agents may be over-indexing on autonomous task horizons. METR-style task horizons clearly matter, but if the industrial revolution is anything to go by, redesigning the production process itself may be just as important for knowledge work: finance, accounting, law, research, etc.
In the industrial revolution, many of the economic gains didn’t come from adding machines into artisan workshops, but from inventing new industrial processes. For physical tasks, that often meant shrinking the length of autonomous human operation and reorganising production into smaller, shorter, more standardised subtasks: division of labour, specialised tooling, QA, repeatable handoffs.
My suspicion is that knowledge work may end up looking similar. The interesting question may be less “how long can one agent operate autonomously?” and more “how can we define small, verifiable subtasks that agents can perform with successive nines of reliability?”
Interested in any thoughts or critiques of the analogy.
Comment · 1 year ago
Interested to hear people's thoughts on why even though AI developments seem so revolutionary it may actually still not be the explosion of growth and societal change that often seems accepted in AI circles.
Comment · 3 years ago
This is text + image -> text but pretty cool and still might be of interest to you:
Comment · 3 years ago
This is text + image -> text but pretty cool and still might be of interest to you:
Comment · 3 years ago
This might be of interest to you (Google are getting there :))- https://palm-e.github.io
Comment · 3 years ago
Google actually recently went some steps further and combined the PaLM LLM (bigger than GPT-3.5) with a 22 billion parameter Vision Transformer to do this -
Comment · 3 years ago
Whilst maybe GPT-4 will change this, I think it is important to remember that these general ChatBots are not the way we have generally trained LLMs to write the best code. In fact, coding is one of the few areas where training specifically just using source code and maybe some stack overflow (not all natural language on the internet) leads to better results on the previous iteration of LLMs (GPT-3 wave). So the real test will be whether the GPT-4 wave of specific coding LLMs i.e GPT-4-Codex can 'actually write code' see:
AlphaCode Codex CodeGen
Comment · 3 years ago
Whilst not the same I recommend you look at the DeepMind Gato paper to see surprisingly (relatively) simple multi modal can be - https://openreview.net/forum?id=1ikK0kHjvj
Essentially to merge lots of modalities they just go 'let's convert all modalities into integers in the same given range', e.g the word 'me' = 1001, up in Atari = 11002, joint torque of right motor of robot = 33000 and so on.
From the paper:
There are infinite possible ways to transform data into tokens, including directly using the raw underlying byte stream. Below we report the tokenization scheme we found to produce the best results for Gato at the current scale using contemporary hardware and model architectures.
• Text is encoded via SentencePiece (Kudo & Richardson, 2018) with 32000 subwords into the integer range [0, 32000).
• Images are first transformed into sequences of non-overlapping 16 × 16 patches in raster order, as done in ViT (Dosovitskiy et al., 2020). Each pixel in the image patches is then normalized between [−1, 1] and divided by the square-root of the patch size (i.e. The tokenized result is a sequence of integers within the range of [0, 1024). 16 = 4).
• Discrete values, e.g. Atari button presses, are flattened into sequences of integers in row-major order.
• Continuous values, e.g. proprioceptive inputs or joint torques, are first flattened into sequences of floating point values in row-major order. The values are mu-law encoded to the range [−1, 1] if not already there (see Figure 14 for details), then discretized to 1024 uniform bins. The discrete integers are then shifted to the range of [32000, 33024).