renchuw

Create SQL by over fitting decision tree on data, then optimize the boolean representation.

Well, this method is based on the assumption that embeddings can accurately represent the texts and their structural relations are preserved.

So long as you have all the random seeds fixed, I think reproduction should be straight forward.

Thanks for the feedback! The reason the "code" part is more complete than the "research" part is because I originally planned for it to just be a hobby project and only very later on decided to perhaps try to be serious and make it a research work.

Not trying to make excuses tho. Your points are very valid and I would take them into account!

Correct.

Hi, OP here. I would kind of have to disagree here. You raised some interesting points, but I don't think something can be qualified as *moat* if it is overcome-able by just sharing the use cases. For example, we all know Google's use cases is to search, but no one has built one as well as they do. Their moat is in their technology and brand recognision.

This would be an inner loop process. However, the selection is way faster than LLMs so it shouldn't be noticable (hopefully).

Hi, OP here. I would say not really because the goals are different. Although both uses retrieval techniques, RAG wants to augment your query with factual information, where here we retrieve in order to evaluate on as few queries as possible (with performance guaranteed by bayesian optimization)

I designed 2 modes in the project, exploration mode and exploitation mode.

Exploration mode uses entropy search to explore the latent space (used for evaluating the LLM on the selected corpus to evaluate), and eploitation mode is used to figure out how well / bad the model is performing on what regions of the selected corpus.

For accurate evaluations, exploration is used. However, I'm also working on a visualization too s.t. users can see how well the model is performing at what region (courtesy of gaussian process models built in by bayesian optimization) and that is where exploitation mode can come in handy.

Sorry for the slightly messy explanation. Hope it clarifies things!

Perhaps I should clarify it in the project README. It's the phase to evaluate how well your model is performing. So the pipeline goes training -> evaluation -> deployment (inference) corresponding to the datasets in supervised training, training (training) -> evaluation (validation) -> deployment (testing).