Recently, I was brought into a large Japanese enterprise AI project midway through a PoC.
The team hadn't been getting the results they hoped for. I was asked to take a look.
I went in expecting to find the usual issues: prompts, retrieval, data structure, AI architecture. And yes, I found plenty to improve.
But after understanding how the operation actually worked, I arrived at a different conclusion.
This problem should not be solved by making the AI better.
What the operation actually looked like
The operation involves producing complex estimates for very large projects — estimates, detailed supporting data, historical project records, and a significant amount of information managed across spreadsheets. Inconsistencies happen. A customer asks why a number appears one way in one document and differently in another. Answering that requires someone to investigate, understand what happened, and formulate a response.
The AI use case seemed obvious: give AI access to all that information and let it answer the questions. A lot of work had already gone into testing that approach before I arrived.
The question I kept coming back to
Why are we spending so much effort teaching AI to investigate inconsistencies that our own process created?
What if instead the question was: if we designed this operation today, with AI available from the start, how would we capture what the customer needs, build an accurate estimate, maintain consistency across every document, and answer questions correctly?
The trap of incremental improvement
The company already has a long-term AI transformation plan. (The document was 90 pages in PowerPoint!) Hundreds of people are involved. In that plan, each existing activity has been mapped with a projected productivity improvement — reduce this by 20%, automate that by 30%, add up the savings.
Sure, it lets them start small and build momentum. But it assumes something important: that today's operating model should still be tomorrow's operating model.
I don't think incremental improvement is wrong. Large enterprises have legacy systems, contracts, messy past documents, security requirements, existing teams and customers. You cannot simply throw everything out.
But I think the order matters. Start with the outcome. Ask what the process would look like if you designed it today with AI.
Sure, I can and I will improve the AI performance for my client for the specific part of the giant AI transformation plan. But I see more value in making sure these inconsistencies don't happen in the first place. And I'm certain AI can make that happen.
It isn't really about AI
As I am involved with Japanese enterprises lately, I also started to think this is tied to the corporate culture.
Over the decade I have seen so many Japanese companies try to "digitalize" their operation and fail, as they only aimed to digest the full operation into smaller pieces, automating them with RPA, resulting in a giant operation built from accumulated "efficiency improvement" pieces with very little consideration of the user experience. I've always played the role of visualizing the overall goals rather than replacing small manual work with RPA, but the voice was rarely heard.
Now with AI, I feel I'm having a déjà vu.
It's not so much about AI, it's about the process design. And it's strongly tied to the corporate culture.
I've seen it work, too
I've seen amazing transformations as well — especially in small sized companies. In the next article I'll write about success stories.
I hope the companies you work for are seeing successful AI adoption. If anyone wants to share your experience, feel free to reach out.