jdspiral
Joined 1 year ago
Karma
54
Submissions
6
Member since
Mar 6
Recent submissions
Comment · 1 year ago
I moved the app, it’s now tokenizer-machine.streamlit.app.
Comment · 1 year ago
So I've taken the feedback and realized that I was misleading on the name and title. I'm updating the project accordingly.
Comment · 1 year ago
Thanks! Yes — that’s on the roadmap, along with some other cool visualizations I’m working on. Machine translation is definitely something I want to work on: showing how models align meaning across languages using shared embeddings and attention patterns. I’d love to make that interactive too.
Comment · 1 year ago
Yes, tokenization and embeddings are exactly how LLMs process input—they break text into tokens and map them to vectors. POS tags and SVOs aren't part of the model pipeline but help visualize structures the models learn implicitly.
Comment · 1 year ago
I built a tool called Meaning Machine to let you see how language models "read" your words.
It walks through the core stages — tokenization, POS tagging, dependency parsing, embeddings — and visualizes how meaning gets fragmented and simulated along the way.
Built with Streamlit, spaCy, BERT, and Plotly. It’s fast, interactive, and aimed at anyone curious about how LLMs turn your sentence into structured data.
Would love thoughts and feedback from the HN crowd — especially devs, linguists, or anyone working with or thinking about NLP systems.
GitHub: https://github.com/jdspiral/meaning-machine Live Demo: https://meaning-machine.streamlit.app