↓Skip to main content
Rushi LuharEngineering, AI & the human side.

AI Technology

The 80/20 Inversion

· 2 min read

For some conversational experiences we build at Jeavio, the first version our clients use is a Slack bot.

It gives subject matter experts a familiar place to try real requests and tell us what works. We capture their feedback and refine the conversational UX before deploying a standalone application.

That’s one example of the 80/20 inversion I’ve noticed in our AI projects over the last three years.

The 80/20 inversion: traditional development puts 80% of effort into build and QA and 20% into feedback and triage; AI development reverses this, with 80% on evals, experiments, and feedback. These are illustrative splits, not measured data. A Jeavio loop moves from a Slack bot with SMEs through feedback and evals to a standalone application, with evaluation before and after release.

View the full-size infographic.

Traditionally, we’ve spent most of our effort on design, development, and QA. Once the application was live, we’d triage feedback, feature requests, and bugs. Think 80% building and 20% iterating, as a rough description rather than a measured split.

With these conversational AI products (or AI products in general), the initial build becomes a much smaller part of the work. Most of the effort goes into refining evals, running experiments, and incorporating what we learn from use.

Conventional software needs that feedback too. What stands out about AI software is how much we still have to learn about its behavior when people bring their own requests. Evals help us check whether our changes improve the experience without breaking something that already worked.

But there’s preparation behind the speed. When AI also helps write the software, teams need to decide which harness and models to use, how to share context and communicate changes, and how to keep PR reviews from becoming a nightmare. Generating code faster is of limited use if nobody can understand what changed.

Early customer feedback doesn’t replace pre-release evaluation, of course. But it does change how I think about planning the work:

  • Establish the development workflow.
  • Get a useful first version in front of key users as soon as possible.
  • Leave substantial room to learn from what they do with it.