News

If you’ve visited a business website lately, you’ve probably encountered an AI-powered chatbot. They’re designed to answer simple queries and process transactions without a human agent. But surveys show many customers don't like them, and that can lead to frustration and even loss of business.

For public interest organizations rolling out self-help bots to help people navigate the legal system, the stakes are higher: giving wrong information on matters like evictions, domestic violence, and child custody can lead to serious consequences.

So when Legal Aid of North Carolina launched an AI chatbot called LIA (legal information assistant) on its website, it partnered with Duke Law professor Keith Porcaro and students in his algorithmic auditing class for an assessment.

 Porcaro and his class analyzed data from user sessions during LIA’s first six months, providing LANC with an audit report with feedback and recommendations that helped the organization improve the product’s usefulness to the low-income North Carolinians it serves. LANC also authorized public release of the audit to help other organizations build and deploy similar AI-driven legal information tools.

The report “helped focus LIA’s evolution from a minimum viable product into a robust and continuously improving public resource,” said LANC board chair Jeff Kelly.

Emerging legal technologies like AI chatbots have enormous potential to improve access to justice but must be deployed with special care within the legal system, where providing wrong answers to users could jeopardize their rights, Porcaro says. 

“There's so much pressure out there to ‘do something with AI’ that it’s easy to just do something simple. Organizations risk skipping the step of asking ‘What does a good version of this look like, and how do we know that it's going to be good?’

“We need data to measure whether it’s good or not. And then we can make a decision about whether to continue, change, or stop it. What I care about is that these things work, and we need to invest as much time as we can in figuring out how and whether they work in order for them to actually get better.”

Building best practices for designing and deploying legal AI

Reliable, trustworthy legal AI tools should incorporate confidentiality and competence — two core principles of legal professional responsibility — in their design, Porcaro said. That means ensuring client data isn’t used for other commercial means as well as protecting it from government subpoenas, for example. 

“Right now, a lot of these tools are being presented as just providing information,” Porcaro said. “But that's not really what people expect when they go to a legal aid organization or a legal services provider. People do disclose sensitive information, so having good filters and good data governance practices is critical.”

AI legal tools also need mechanisms to stop users from committing errors that may lead to serious consequences, he said. When users describe their problem to a bot, especially one of a personal nature, they may tell a disordered story with irrelevant details. That could cause the bot to return information that’s imprecise, irrelevant, or incorrect. A user who acts on wrong information could jeopardize their legal rights.

“Clients bring messy stories, they bring emotional stories, they bring incomplete stories. 
That fringe, for these tools, can get really messy,” Porcaro said. 

“As we're building these tools, we need to shift the way we think about deploying software, from where users are left on their own no matter what, to making sure that even if a user misinterprets something, or even if they give a story that's jumbled, that it’s not going to be entirely on them to catch a mistake before they file at court and lose their rights,” Porcaro said. 

In an audit report for the Nevada Administrative Office of the Courts, which launched a chatbot to direct self-represented litigants to reliable information on family law matters, Porcaro’s class developed and added a new feature to its methodology, rating the chatbot’s answers to 1,462 user sessions on a five-point scale from “helpful and actionable” for correct answers that included an actionable next step, down to “harmful” for answers that were incorrect or misleading and might cause the user harm if followed. 

About 46% of the chatbot’s answers were rated helpful and actionable, while 1% were deemed harmful. The audit’s recommendations included improving the chatbot’s ability to detect high-risk situations and deliver more specific resources to the user, such as suggestions to call 911 or child protective services.

Ultimately, courts and organizations must decide whether the benefits of deploying emerging technology like an AI chatbot justify the risks. Porcaro is promoting best practices like pre-launch stress testing and a commitment to constant monitoring and improvement to ensure that AI legal tools hold up under real-world use.

“This is an experimentation phase. The answer is not AI or nothing. The answer is how it measures up compared to all the other information or tech interventions you could use,” Porcaro said.

“If you're helping a certain number of people but a small slice of people are being harmed, is that an appropriate tradeoff? I don't know that we've really found the right balance of benefit and harm yet. But the goal is to add enough detail so our partners can make that decision.”