chenglong-hn
Joined 2 years ago
Karma
313
Submissions
68
Member since
Oct 21
Recent submissions
Comment · 4 days ago
Comment · 9 days ago
We have plenty demos of the interactions here!
- https://microsoft.github.io/flint-chart/#/interactions/advan...
Comment · 3 months ago
no yet! that's a good direction!
Comment · 3 months ago
Mostly for expressiveness and reliability & cost trade-off. Flint has advantage of being an intermediate language that allow agents to generate good-looking stuff without additional refinement loops, since the compiler derives lower-level geometric constraints from semantic types.
Would especially be handy when we are building some agents that produce charts that serve end users (they want faster and more reliable experiences!).
Comment · 3 months ago
Both, it's a combination of latency / reliability / expressivenss. I think when models are more powerful, we will also more naturally want to create more expressive stuff.
Comment · 3 months ago
This is awesome!
Comment · 3 months ago
We struggled the same, especially when deploying an agent to end users, they are even more frustrated with any mistakes from agents than us!
Comment · 3 months ago
This is a formal language that compiler can take! The natural language and clarify loop helps the LLM to generate Flint spec that can render.
Comment · 3 months ago
We made an intentional decision to make it non-compositional for now, and we are using chart-type based approach to handle more expressive charts. (e.g., CDF charts)
But! We have a new project coming in to work on annotation machanism and interactivity to make it supporting presentation tasks.
Comment · 3 months ago
Thanks a lot for the suggestion, I do think we will need to work on accessibility support in Flint, this is a perfect place to centrally handle accessibility issues.
I added an issue to track this
Comment · 3 months ago
We will link it on github and the project page!
Comment · 3 months ago
We find an additional dimension where previous language doesn't explicitly use (semantic type) as the key part of the specification, which turned out to be the root reason of why certain low-level parameters are set.
Effectively Flint is designed to specify charts using another set of semantic parameters over geometric parameters, which happens to be something AI are pretty good at! (They can based on field names + sample values to infer the semantic types reliably than guessing geometrical params.
Btw, for others reading this comment, this is the XKCD we are talking about! https://xkcd.com/927/
Comment · 3 months ago
That's the best option for now. But it can also be frustrating to ask AI to do small edits on D3 just to fix some idioms (like switching order etc) and they kept messing up with other stuff accidentally. Thus I still believe have a language with native representation for these diagram concepts would be helpful.
Good news is that AI do make new language a bit more accessible than before! If your agents can use it well and you can steer easily, it will naturally be good adoption.
We find the issue is that the language agents use to create interactive visualization is kinda low-level. The logic for specifying data encoding, style and interactions are all entangled (if you swap x,y axis, you are likely going to redo your interaction and layout again). Thus, despite agents have been working hard, it is essentially doing lots of stuff bottom up. We decide to design a language that can separate the visual logic and the interaction logic to make specification much cleaner, manageable, and friendly (to both human and AI). The magic here is that we are now disentangle the logic at the spec level, but make the compiler to do the heavy lifting (sorry, compiler!) to wire up logics to support easy composition and adaptation of the logics to create a expressive interactive charts. We used a trick of bidirectional transformation to help the compiler maintain the mapping between pixel/visual space to data semantics so the specs can stay at a higher-level. We made a video to explain the ideas.
https://youtu.be/JAJy7ToczMs
Some principles we had in the design:
- the language should be high-level and the agents can easily user them in chats/apps declaratively (with many presets).
- the language should not give up expressiveness. We also provide a programming API, so that you can use it to create charts in your app/dashboards for external interactions. The expressive API also makes it possible to compose very cool bespoke interactions (the spinning globe in the website).
To try it out:
- You can try Flint in your agent app. Simply call your agent to create it as an MCP app in your chat! (https://microsoft.github.io/flint-chart/#/mcp)
- If you are building applications, ask the agent to use the library. Flint is especially suitable for building agentic apps where you want agents to bring up an interactive charts quickly in the user session following their asks.
- Btw, the flint mcp app allows you create interactive dashboard in the chat as well.
Finally, I'm thinking that despite AI agents are now so good at coding, that they seem to be able to create EVERYTHING. It might not just be the time to give up designing new languages to make it easy to describe and create stuff. In the end, language is still the way for us to communicate with the computer, I think the alignment of programming language between human and AI in could probably make building, steering easier.