MadScientistMatt said:Beer Baron said:In reply to Keith Tanner :
I'm aware of that case. That's something that can and certainly will get fixed, much like AI not being able to draw hands.
And I agree that you will still need a lawyer who understands things to make sure it works. That's why AI won't replace lawyers. It will replace the menial tasks that support lawyers.
I'm not sure if improving large language models can fix the sort of problem that brief had. A LLM can create something that looks like other legal briefs. But it has non capacity to sift through actual cases to cite. It just knows that legal briefs end in a block of text in a certain format. So it draws up something with the same format as a block of citations, but doesn't, and can't , verify that they even exist.
A program that can look through a file of case decision and see if they support a case is an entirely different algorithm.
Right. Not developing a LLM to replace the work of a Lawyer to prepare and review a legal brief. Developing one to replace the work of a Clerk to find, collate, and present a master list of cases for the lawyer to review and use to prepare the brief.
"Jimmy! I want a list of all cases related to training squirrels for industrial maintenance tasks on my desk by the time I get back from lunch!"
AI will not be able to understand if prior case law supports a current case. It will still take a reasoning person to understand the case law and its implications. But it would be entirely within the scope of an LLM to be trained to return results that relate to particular key words or topics.
You also need to train the LLM to only return actual cases. That's not a massive hurdle. LLM's started out super racist and sexist like the Redditors they learned from. It didn't take a lot to train those LLM's to not return racist responses. It wouldn't take much to train a specialized Law Clerk LLM to only return actual case law.
Then the human lawyer reviews the lists of related cases and uses their reasoning and judgment to decide which ones actually support their case, and then prepare the brief themselves.
