AI Can Recommend, but Who Decides?

While reviewing state legislation for CAID's AI Policy Tracker, I noticed that several proposals allow AI to help with important tasks while still leaving the final decision to a person. This seems like a reasonable balance: AI can make a process faster, but a person remains responsible when the outcome could affect someone's finances, rights, or access to services. However, the bills also made me wonder whether simply having a person involved is enough.

Florida HB 527, introduced in 2026, applied this idea to insurance claims. The bill would have allowed insurers, healthcare organizations, and workers' compensation carriers to use AI to help process claims. However, AI couldn't be the sole reason for denying a claim or reducing payment. A qualified professional would have needed to independently review the facts, examine the policy terms, check the accuracy of the AI's output, and make the final decision. The bill died in the Senate Rules Committee, but it shows how human review could add a layer of protection when an automated recommendation has a direct financial impact on someone.

Alabama HB 524 used a similar idea in a different setting. The proposal would require state agencies to use AI to review rules and guidance documents for outdated language, duplication, legal conflicts, and unnecessary costs. Agency employees with legal or subject-matter expertise would then have checked the AI's analysis and decided whether its recommendations should be used. The bill specifically stated that a rule couldn't be changed or repealed based only on an AI analysis. Although the proposal didn't pass, it recognized that decisions affecting public rules should remain with people who understand the law and can question the system's conclusions.

Hawaii HB 487 offers a less formal example of keeping AI in a supporting role. The proposal would have used AI to identify state laws and administrative rules that appeared outdated, duplicative, or inconsistent with other laws. The Legislative Reference Bureau would then have reported its findings and recommendations to lawmakers, who would still decide whether anything should change. Unlike Alabama HB 524, the bill didn't establish detailed requirements for reviewing the AI's analysis. The proposal did not advance, but it shows another way lawmakers have considered separating AI-generated findings from final government decisions.

What stood out to me is that putting a person in the process doesn't automatically mean the review will be meaningful. A 2024 issue brief from Georgetown University's Center for Security and Emerging Technology discusses "automation bias," or the tendency for people to rely too heavily on an automated system, even when other information suggests it may be wrong. The report explains that this problem can be influenced by the reviewer's experience, the design of the technology, and the organization's training and procedures.

This means that human review can't just be another box to check. A reviewer needs enough knowledge, information, time, and authority to disagree with the AI. Florida HB 527 required an independent review, while Alabama HB 524 required reviewers with legal or subject-matter expertise. Those details matter because a person who simply approves an AI recommendation isn't providing much protection. Overall, these proposals show that lawmakers are trying to keep human responsibility in decisions assisted by AI. None of the bills became law, but they raise an important question for future policy. If AI gets something wrong, is there a person who can recognize the mistake, correct it, and be held responsible? AI may help make decisions faster, but it shouldn't make accountability disappear.