jehna1
Joined 14 years ago
Joined 14 years ago
Here's an example I finished just a few minutes ago:
https://github.com/jehna/plant-light-holder/blob/main/src/pl...
https://arxiv.org/pdf/2405.15793
It uses smart feedback to fix the code when LLMs occasionally do hiccups with the code. You could also have a "supervisor LLM" that asserts that the resulting code matches the specification, and gives feedback if it doesn't.
For small scripts I've found the output to be very similar between small local models and GPT-4o (judging by a human eye).
Babel first parses the code to AST, and for each variable the tool:
1. Gets the variable name and surrounding scope as code
2. Asks the LLM to come up with a good name for the given variable name, by looking at the scope where the variable is
3. Uses Babel to make the context-aware rename to AST based on the LLM's response
I'm currently working on parallelizing the rename process, which should give orders of magnitude faster processing times for large files.
1. It asks the LLM to write a description of what the variable does
2. It asks for a good variable name based on the description from 1.
3. It uses a custom Babel plugin to do a scope-aware rename
This way the LLM only decides the name, but the actual renaming is done with traditional and reliable tools.
1. I ask LLM to describe what the meaning of the variable in the surrounding code
2. Given just the description, I ask the LLM to come up with the best possible variable name
You can check the source code for the actual prompts:
https://github.com/jehna/humanify/blob/eeff3f8b4f76d40adb116...
API access of ChatGPT mode is needed as there are many round trips and it uses advanced API-only tricks to force the LLM output.