jackmpcollins

Joined 5 years ago

Karma

111

Submissions

Member since

Jun 16

View on Hacker News

Recent submissions

Comment · 1 year ago

Seems to me that integrations will be the most important component of tools like this. As an engineer I get my context from video calls with customers and other engineers, slack messages, emails, docs online, using the product myself, etc. So an auto-engineer should do the same.

Comment · 1 year ago

I've already merged my first Engine PR! Being able to review PRs like normal and it updates its work is very cool.

Comment · 1 year ago

I believe we'll see two main types of AI/LLM libraries/frameworks emerge like has happened for ORMs (at least in Python). - The "Sqlalchemy" level: just bridge the gap between code and LLM - The "Django" level: additionally handle how data is indexed for RAG, standard patterns of connecting the pieces, possibly GUI, support for plugins.

It seems langchain and llamaindex are aiming to be the latter, and I'm building https://magentic.dev to be the former. I'd be interested to get your take on whether these abstractions would allow you stray from the narrow path while still being helpful!

Comment · 1 year ago

I would love some feedback on my library magentic which aims to have the best features of both approaches! https://github.com/jackmpcollins/magentic

Comment · 1 year ago

If you are using Python, check out the package I've been building, magentic https://github.com/jackmpcollins/magentic It supports structured outputs and streaming, and aims to avoid making unnecessary abstractions (but might require some more understanding of LLM patterns as a result).

Also recently released is pydantic-ai, which is also based around pydantic / structured outputs, though works at level of "agents". https://github.com/pydantic/pydantic-ai

Comment · 1 year ago

I'm building magentic for use cases like this!

https://github.com/jackmpcollins/magentic

It's based on pydantic and aims to make writing LLM queries as easy/compact as possible by using type annotations, including for structured outputs and streaming. If you use it please reach out!

Comment · 1 year ago

I'd be really interested to hear what abstractions you would find useful for RAG. I'm building magentic which is focused on structured outputs and streaming, but also enables RAG [0], though currently has no specific abstractions for it.

[0] https://magentic.dev/examples/rag_github/

Comment · 1 year ago

I've been building magentic [0] around similar ideas! Structured outputs and keeping control flow in python is definitely the way to go to keep things lightweight and easy to integrate. Having state management built in is cool! I'd be interested to hear what kinds of projects this has enabled you to build.

[0] https://github.com/jackmpcollins/magentic

Comment · 2 years ago

I'm working on a very abstraction light package for LLMs called magentic. The goal is to handle the main boilerplate of building with LLMs, but leave the abstractions up to the user. I'd be interested to hear your opinion on it vs others.

https://github.com/jackmpcollins/magentic

Comment · 2 years ago

I'm building magentic, which aims to be a middle ground between the current "no framework" and "too much framework" options. It handles retries, logging/tracing, concurrency with asyncio, and other boilerplate, without mandating any prompts or particular patterns. Please check it out for the next comparison!

https://github.com/jackmpcollins/magentic

Comment · 2 years ago

Pydantic also have support for parsing partial JSON. https://docs.pydantic.dev/latest/concepts/json/#partial-json...

  from pydantic_core import from_json

  partial_json_data = '["aa", "bb", "c'  
  
  result = from_json(partial_json_data, allow_partial=True)
  print(result)  
  #> ['aa', 'bb']

You can also use their `jiter` package directly if you don't otherwise use pydantic. https://github.com/pydantic/jiter/tree/main/crates/jiter-pyt...

Comment · 2 years ago

That gif is really cool! I built a Python package magentic [0] which similarly parses the LLM streamed output and allows it to be used before it is finished being generated. There are plenty of use cases / prompts that can be refactored into a "generate list, then generate for each item" pattern to take advantage of this speedup from concurrent generation.

[0] https://magentic.dev/streaming/#object-streaming

Comment · 2 years ago

It's a great start but there's a little more work to do for full OpenAI API compatibility, namely streaming support and the tool_choice parameter. Making it fully compatible would allow it be swapped in directly to frameworks like langchain and magentic [1] that I am building).

[1] https://github.com/jackmpcollins/magentic/issues/207

Comment · 2 years ago

Thanks for the explanation! Do you see auto-optimization as something that is useful for every use case or just some? And what determines when this is useful vs not?

Comment · 2 years ago

Looks quite like magentic [0] that I've been building, though broader in scope? I'm (clearly) a huge advocate of pydantic, structured outputs, and keeping control flow in python code (rather than inside abstractions / "chains") as much as possible, so it's great to see those values here too! I'd be interested to hear what you consider in vs out of scope for mirascope and the long-term vision. Also would be cool for one of us to do a mirascope vs magentic comparison blog post.

[0] https://github.com/jackmpcollins/magentic