This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen:
> Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978...
> Judge: There was no case Johnson v. Smith in 1978.
LLMs hallucinate, and there is absolutely no space for hallucination in a court of law. The legal profession is perhaps the closest one to computer programming, and absolute precision is required, not a just-barely-good-enough statistical machine.
> Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978...
> Judge: There was no case Johnson v. Smith in 1978.
LLMs hallucinate, and there is absolutely no space for hallucination in a court of law. The legal profession is perhaps the closest one to computer programming, and absolute precision is required, not a just-barely-good-enough statistical machine.