Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon

with Aishwaria Raanti and Kiti Bottom

11 Jan 2026 12 min read 1h 15m

TL;DR

Building successful AI products requires treating them fundamentally differently from traditional software—specifically by accepting non-determinism in both user inputs and AI outputs, and gradually increasing agent autonomy only after proving reliability. Most failures stem from teams jumping to full autonomy immediately rather than starting with high human control and low agency, then incrementally building flywheels for improvement.

Key Moments

Aishwaria Raanti

“Most people tend to ignore the non-determinism. You don't know how the user might behave with your product and you also don't know how the LLM might respond to that. The second difference is the agency control trade-off. Every time you hand over decision-m capabilities to agentic systems, you're kind of relinquishing some amount of control on your end.”

Explaining the two fundamental differences between AI and non-AI product development

▶ 0:08

Kiti Bottom

“when you start small, it forces you to think about what is the problem that I'm going to solve. In all this advancements of the AI, one easy slippery slope is to keep thinking about complexities of the solution and forget the problem that you're trying to solve.”

Discussing why starting with minimal AI autonomy helps teams stay focused on real problems

▶ 0:30

Kiti Bottom

“It's not about being the first company to have an agent among your competitors. It's about have you built the right flywheels in place so that you can improve over time.”

Addressing the common mistake of rushing to deploy fully autonomous agents

▶ 0:42

Aishwaria Raanti

“Leaders have to get back to being hands-on. You must be comfortable with the fact that your intuitions might not be right and you probably are the dumbest person in the room and you want to learn from everyone.”

Describing the leadership mindset shift required for successful AI product teams

▶ 1:03

Aishwaria Raanti

“Persistence is extremely valuable. Successful companies right now building in any new area. They are going through the pain of learning this, implementing this and understanding what works and what doesn't work. Pain is the new mode.”

Concluding thoughts on the reality of building AI products in 2025

▶ 1:16

About the guests

›

Aishwaria Raanti is an early AI researcher at Alexa and Microsoft who has published over 35 research papers. Kiti Bottom works on codecs at OpenAI and has spent the last decade building AI and ML infrastructure at Google and Kumo. Together, they've led and supported over 50 AI product deployments across Amazon, DataBricks, OpenAI, Google, and both startups and large enterprises, and teach the top-rated AI course on Maven.

Takeaways

Start with high control, low agency Don't deploy fully autonomous AI agents on day one. Begin with AI providing suggestions to humans (high control), then gradually increase autonomy as you prove reliability. This pattern applies across customer support, coding assistants, and marketing tools—each version adds more autonomous capabilities only after the previous version performs well.

Non-determinism requires behavior calibration Unlike traditional software, AI systems produce different outputs for similar inputs. Rather than trying to predict all behaviors upfront, build feedback loops where you observe actual usage, log human decisions, and continuously improve. This flywheel approach is what separates successful AI products from failures.

Leadership must become hands-on with AI Executives need to spend time daily learning AI capabilities and limitations, not delegating understanding to engineers. Leaders with old intuitions about software development will make wrong calls about AI products. The winning organizations have leaders who rebuild their intuitions by actively engaging with the technology and admitting when they're learning alongside their teams.