Where To Draw The Line With AI

This voice experience is generated by AI. Learn more.This voice experience is generated by AI. Learn more.Anu Choudhury is the Chief Technology Officer of Reputation, a global leader in reputation intelligence.

gettyCompanies of all shapes and sizes are embracing AI, and many are seeing real benefits. But there are certain tasks AI should not do on its own, and that’s where I, as a CTO, draw the line.

For example, I don’t want AI posting publicly on behalf of my company, or any other company for that matter, on its own. This means no autonomous AI comments on the internet, nothing pushed live on LinkedIn, without a human in the loop to check it first.

Unfettered AI like this is a liability waiting to happen. We all know AI has a tendency to hallucinate, and it’s your good name that suffers if your AI loses the thread and says something that isn’t true. That’s a serious risk because once your reputation gets damaged publicly, it’s hard to win back trust. You sure won’t do it by deleting that one rogue post.

Here’s what unfettered AI can look like in the wild. Let’s say a customer gives you a one-star review because they were unhappy about a particular service. Your AI agent might see that review and respond right away, professionally. But in its quest to always be helpful, it might also invent a discount code out of thin air. Because nobody at your company is monitoring the AI’s outputs, that unauthorized discount code is now sitting out there in the wild for everyone to see. And the original customer? They get even angrier when they try to use the code and it doesn’t work.

That’s not a situation any company wants. And it’s why my number one rule for AI is to have a human in the loop for any public-facing outputs, such as review responses and published content. I’ll hold onto this rule at least until AI can earn my trust and prove to me that it can generate appropriate answers again and again, at scale.

Don’t get me wrong. I’m not an AI luddite. I actually think most workflows are better off fully automated, with no human intervention required.

Take a task like ticket-routing. Many companies have internal queues where hundreds of tickets get assigned to team members every day. I’m all for optimizing that process with AI. It doesn’t need a human in the loop. That’s exactly the kind of high-volume, low-risk workflow where AI creates significant time savings, and there’s no good reason to slow that down with a human overseer.

My point is that, before you trust a particular workflow to AI, you need to think through what happens if something goes wrong. What’s the blast radius? What would a mistake do to your company’s reputation? Would it be easy to revert, no real harm done, or would it be the kind of impact that takes serious time and effort to undo? The answers to questions like these will tell you where it’s best to draw the line with AI.

Before an AI-powered feature gets built, there’s a bunch of questions that need to be answered. How does the feature handle security? How does it interface with external sites? What does it cost to run? These considerations need to be part of the architecture at design time, not bolted on as an afterthought.

Take something as simple as token usage. AI that burns through massive amounts of tokens when no one is watching is a real problem. But with the right checks and balances, you can safely let the AI run on its own because you’ve put guardrails in place to prevent it from exceeding a predetermined cost ceiling. Or take those tasks that risk exposing data or creating a security gap. AI should never be allowed to run them without some kind of human oversight.

This is also where AI observability comes in. In plain terms, this is the ability to see inside an AI system’s decision-making and understand why it returned a particular answer. In other words, when the AI hallucinates, can you catch it? And do you have the systems built in to explain why it gave a bad answer?

Most companies don’t, which is why AI observability matters. If an AI answer isn’t right, you should be able to pinpoint the reason why. And you can’t do that without AI observability in place from day one.

I still see resistance to AI adoption inside organizations. Especially among engineering and product teams, the primary fear is that AI is coming for their jobs. My take is that AI can probably do some part of your job very well. But there are other parts that humans will always be able to do better, and those are the parts I tell my team to really lean into. For instance, AI will never really know your customers the way you do, and it’s this kind of knowledge that makes all the difference. I tell my team it’s on them to bring that knowledge to the table and use it to guide our AI.

The secondary fear is that AI will break everything. That’s a legitimate concern, and it’s exactly why a human-in-the-loop approach is vital: to prevent AI from making mistakes and quickly correct them if it does.

What you can’t do is stick your head in the sand and say, “I’ll never let AI deploy anything.” That’s not an option today. The key is knowing precisely where AI can be deployed safely, where it can’t and making sure a human is the one actually pushing the button when it matters most.​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?