atgctg

  Size of KV cache = 2 * (num_layers) * (num_kv_heads * dim_head) * seq_length * precision

  8-bit Gemma 27B KV cache = 2 * (46) * (16 * 144) * 1e6 * 1 byte ≈ 200 GB

Note that this doesn't take further optimizations into account that Google might be using.

Formula: https://developer.nvidia.com/blog/mastering-llm-techniques-i...

Gemma 27B config: https://huggingface.co/google/gemma-2-27b/blob/main/config.j...

Open source AI is the path forward about.fb.com

2,360 atgctg 2 years ago 887 comments

The Perceptron Controversy yuxi-liu-wired.github.io

4 atgctg 2 years ago 0 comments

Comment · 2 years ago

Works using math CSS injection [1]:

    ```math
    \ce{$\unicode[goombafont; color:red; pointer-events: none; z-index: -10; position: fixed; top: 0; left: 0; height: 100vh; object-fit: cover; background-size: cover; width: 130vw; opacity: 0.5; background: url('https://github.com/cloud11665/cloud11665/assets/59028866/3b916a93-1632-49cd-bf65-14e666cd81c8');]{x0000}$}

[1]: https://raw.githubusercontent.com/cloud11665/cloud11665/mast...

Comment · 2 years ago

Tiktoken added support for GPT-4o: https://github.com/openai/tiktoken/commit/9d01e5670ff50eb74c...

It has an increased vocab size of 200k.

Comment · 2 years ago

Seems like they are working on adding that capability:

> We're exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT.

Link to section: https://cdn.openai.com/spec/model-spec-2024-05-08.html#dont-...