LakshyAAAgrawal
Joined 8 years ago
Karma
91
Submissions
47
Member since
Jul 15
Recent submissions
Comment · 5 months ago
Can a single LLM-based optimization system match specialized tools across fundamentally different domains? We show that when optimization problems are formulated as improving a text artifact evaluated by a scoring function, a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs-achieves state-of-the-art results across six diverse tasks. Our system discovers agent architectures that nearly triple Gemini Flash's ARC-AGI accuracy (32.5% to 89.5%), finds scheduling algorithms that cut cloud costs by 40%, generates CUDA kernels where 87% match or beat PyTorch, and outperforms AlphaEvolve's reported circle packing solution (n=26). Ablations across three domains reveal that actionable side information yields faster convergence and substantially higher final scores than score-only feedback, and that multi-task search outperforms independent optimization given equivalent per-problem budget through cross-task transfer, with benefits scaling with the number of related tasks. Together, we show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm, unifying tasks traditionally requiring domain-specific algorithms under a single framework. We open-source optimize\_anything with support for multiple backends as part of the GEPA project at https://gepa-ai.github.io/gepa/blog/2026/02/18/introducing-o... .
Comment · 6 months ago
Large language models (LLMs) are trained for downstream tasks by updating their parameters (e.g., via RL). However, updating parameters forces them to absorb task-specific information, which can result in catastrophic forgetting and loss of plasticity. In contrast, in-context learning with fixed LLM parameters can cheaply and rapidly adapt to task-specific requirements (e.g., prompt optimization), but cannot by itself typically match the performance gains available through updating LLM parameters. There is no good reason for restricting learning to being in-context or in-weights. Moreover, humans also likely learn at different time scales (e.g., System 1 vs 2). To this end, we introduce a fast-slow learning framework for LLMs, with model parameters as "slow" weights and optimized context as "fast" weights. These fast "weights" can learn from textual feedback to absorb the task-specific information, while allowing slow weights to stay closer to the base model and persist general reasoning behaviors. Fast-Slow Training (FST) is up to 3x more sample-efficient than only slow learning (RL) across reasoning tasks, while consistently reaching a higher performance asymptote. Moreover, FST-trained models remain closer to the base LLM (up to 70% less KL divergence), resulting in less catastrophic forgetting than RL-training. This reduced drift also preserves plasticity: after training on one task, FST trained models adapt more effectively to a subsequent task than parameter-only trained models. In continual learning scenarios, where task domains change on the fly, FST continues to acquire each new task while parameter-only RL stalls.
Paper: https://arxiv.org/abs/2605.12484v1 Blog: https://gepa-ai.github.io/gepa/blog/2026/05/11/learning-fast... Twitter: https://x.com/KushaSareen/status/2054586907904901245?s=20
Comment · 8 months ago
Thank you so much for the kind words! Didn't realize it got truncated: https://gepa-ai.github.io/gepa/blog/2026/02/18/introducing-o...
Comment · 8 months ago
We open-sourced optimize_anything, an API that optimizes any text artifact. You provide a starting artifact (or just describe what you want) and an evaluator and it handles the search.
import gepa.optimize_anything as oa
result = oa.optimize_anything( seed_candidate="<your artifact>", evaluator=evaluate, # returns score + diagnostics )
It extends GEPA (our state of the art prompt optimizer) to code, agent architectures, scheduling policies, and more.
Two key ideas: (1) diagnostic feedback (stack traces, rendered images, profiler output) is a first-class API concept the LLM proposer reads to make targeted fixes, and (2) Pareto-efficient search across metrics preserves specialized strengths instead of averaging them away.
Results across 8 domains show optimize_anything can create: - learned agent skills pushing Claude Code to near-perfect accuracy simultaneously making it 47% faster, - cloud scheduling algorithms cutting costs 40%, - an evolved ARC-AGI agent going from 32.5% → 89.5%, - CUDA kernels beating baselines, - circle packing outperforming AlphaEvolve's solution, - and blackbox solvers matching and outperforming Optuna.
Comment · 8 months ago
We built optimize_anything, an API that optimizes any artifact representable as text — code, prompts, agent architectures, configs, even SVGs. It extends GEPA (our prompt optimizer, discussed here previously: https://arxiv.org/abs/2507.19457) far beyond prompts.
The API is deliberately minimal. You provide what to optimize and how to measure it:
import gepa.optimize_anything as oa
def evaluate(candidate: str) -> tuple[float, dict]: result = run_my_system(candidate) return result.score, {"error": result.stderr, "runtime": f"{result.time_ms}ms"}
result = oa.optimize_anything( seed_candidate="<your artifact>", evaluator=evaluate, )
The evaluator returns a score plus diagnostic feedback (we call it "Actionable Side Information" — stack traces, rendered images, profiler output, whatever helps diagnose failures). An LLM proposer reads this feedback during a reflection step and proposes targeted fixes, not blind mutations. Candidates are selected via a Pareto frontier across metrics/examples, so a candidate that's best at one thing survives even if its average is mediocre.
Two ideas distinguish this from AlphaEvolve/OpenEvolve/ShinkaEvolve-style LLM evolution: (1) diagnostic feedback is a first-class API concept rather than a framework-specific mechanism, and (2) the API unifies three optimization modes — single-task search (solve one hard problem), multi-task search (solve related problems with cross-transfer), and generalization (build artifacts that transfer to unseen inputs). Prior frameworks only express mode 1.
We tested across 8 domains. Selected results:
Coding agent skills: Learned repo-specific skills push Claude Code to near-perfect task completion and make it 47% faster Cloud scheduling: Discovered algorithms that cut costs 40%, topping the ADRS leaderboard over expert heuristics and other LLM-evolution frameworks Agent architecture: Evolved a 10-line stub into a 300+ line ARC-AGI agent, improving Gemini Flash from 32.5% → 89.5% Circle packing (n=26): Outperforms AlphaEvolve's published solution Blackbox optimization: Generated problem-specific solvers matching or exceeding Optuna across 56 EvalSet problems CUDA kernels: 87% match or beat baseline; multi-task mode outperforms dedicated single-task runs
``` pip install gepa ```
Blog with full results and runnable code for all 8 case studies: https://gepa-ai.github.io/gepa/blog/2026/02/18/introducing-o...
GitHub: https://github.com/gepa-ai/gepa
Comment · 1 year ago
Dear Tom, Thanks a lot for trying out GEPA and writing about your experience in the blog!