At a Glance
- Alaska’s probate system gets a generative AI chatbot, AVA, after a year-long development.
- The project was intended to finish in 3 months but now exceeds 1 year 3 months.
- AVA will aim to replicate a human self-help facilitator for probate cases.
- Why it matters: Alaskans will have faster, cheaper access to probate guidance, but accuracy concerns remain.
Alaska’s court system has spent more than a year building a generative-AI chatbot called AVA to help people navigate probate. The project, originally slated for a three-month rollout, now faces delays and accuracy challenges that could affect users’ legal decisions. As the launch approaches in late January, officials weigh the benefits of AI against the risks of misinformation.
From Vision to Reality
The idea for AVA began as a grant-funded experiment to bring AI into probate. The National Center for State Courts provided the initial grant, and lawyer-technologist Tom Martin of LawDroid led the technical build. The goal was to offer a low-cost, self-help experience similar to the court’s staffed family-law helpline.
- Aubrie Souza (consultant, NCSC)
- Stacey Marz (admin director, Alaska Court System)
- Tom Martin (lawyer, LawDroid)
- Jeannie Sato (director of access to justice services)
Accuracy Hurdles
Ensuring 100 % accuracy is essential, but the team encountered several problems. Early versions of AVA were overly empathetic, prompting users to remove condolences. More critically, the bot hallucinated facts, such as claiming Alaska had a law school that does not exist.
Aubrie Souza stated:
> “We had trouble with hallucinations, regardless of the model, where the chatbot was not supposed to actually use anything outside of its knowledge base.”
Stacey Marz said:
> “If people are going to take the information they get from their prompt and they’re going to act on it and it’s not accurate or not complete, they really could suffer harm. It could be incredibly damaging to that person, family or estate.”
Tom Martin explained:
> “Different models have almost different types of personalities. For a legal application, you don’t want that. You want it to be rule-following but smart and able to explain itself in plain language.”
Testing and Costs
The team initially designed a 91-question test but found it too time-consuming. They narrowed the set to 16 questions that cover common, complex, and basic probate queries.
| Test Size | Questions |
|---|---|
| Original | 91 |
| Refined | 16 |
Cost is a major advantage. Under one setup, 20 AVA queries cost about 11 cents. However, the system will need regular updates as models evolve.
Tom Martin noted:
> “We anticipate needing to do regular checks and potentially update prompts or the models as new ones come out and others are retired. It’s definitely something we’ll need to stay on top of rather than a purely hands-off situation.”
Looking Ahead
AVA is slated for a late-January launch, provided all testing clears. The team acknowledges the bot’s current limitations but hopes future model improvements will raise accuracy and completeness.
Stacey Marz said:
> “We did shift our goals on this project a little bit. We’re not confident that the bots can work in that fashion, because of the issues with some inaccuracies and some incompleteness. But maybe with increasing model updates, that will change, and the accuracy levels will go up and the completeness will go up.”
Key Takeaways
- AVA is a generative-AI chatbot for Alaska probate, delayed to late January after a year-long build.
- Accuracy and hallucinations remain major hurdles; the team uses a 16-question test to monitor performance.
- The system costs only about 11 cents per 20 queries, but will require ongoing updates as AI models change.
While AVA promises faster probate guidance, its journey underscores the challenges of deploying AI in high-stakes legal contexts.
