According to Bloomberg Business, Rakuten Group’s AI chief, Google veteran Ting Cai, is prioritizing extreme cost efficiency over raw model size. The company, which aims to become Japan’s leading AI empowerment firm, has grown its AI team to 1,000 people this year. Cai’s team just unveiled version 3 of its large language model, claiming it’s a staggering 90% cheaper to run than existing comparable LLMs. The model uses a technique where only about 40 billion of its 700 billion total parameters activate for any given task. Rakuten’s AI functions already contributed ¥10.5 billion ($67 million) to operating income in 2024, and the goal is to double that figure this year. Cai, who previously worked on Google Maps, says reducing the cost of AI “conversations” is super important to make each customer interaction profitable.
Rakuten’s Pragmatic AI Playbook
Here’s the thing: while Silicon Valley giants are locked in a parameters arms race, Rakuten is playing a completely different game. They’re not trying to build the biggest, most general intelligence. Instead, they’re ruthlessly segmenting tasks down to their simplest forms and then building smaller, specialized models for each one. Think of it like having a Swiss Army knife where you only open the exact tool you need, instead of unfolding the entire heavy contraption every time. That’s basically what they’re doing with their 700B parameter model—only activating a slim 40B parameters for any individual token. It’s a clever hack for efficiency, but it requires deep integration with your specific business problems to work. You can’t just slap this model on a public API and call it a day.
The Business Pressure Behind The Tech
Now, this isn’t just a technical whim. Rakuten is under real financial pressure. Its mobile business is struggling, and e-commerce competition is relentless. So AI isn’t a moonshot lab project; it’s a lifeline that has to prove its ROI immediately. That’s a brutal but clarifying constraint. When Cai says they need “maximum margin,” he means it. The results so far—better ad targeting, semantic search, and recommendations that boost user engagement—are directly tied to the bottom line. It’s a stark contrast to the “spend billions now, figure out monetization later” approach we see elsewhere. But is this focus too narrow? Could it limit their ability to create more transformative, general-purpose AI down the line? Possibly. But for a company fighting on multiple fronts, profitability today might be the only strategy that matters.
Why Cost Is The New AI Battleground
And this is where it gets interesting for the whole industry. Rakuten is highlighting a dirty secret: running these massive LLMs is insanely expensive. If a model is 90% cheaper to operate, that’s not just an incremental gain—it’s the difference between a feature that bleeds money and one that prints it. This focus on inference cost is going to become the real battleground, especially for business applications. It’s one thing to train a dazzling model for a press release; it’s another to run it billions of times for customers without going bankrupt. This shift towards efficiency is where a lot of the real innovation will happen. For companies integrating AI into physical operations, like in manufacturing or logistics, this cost reliability is everything. Speaking of robust industrial computing, for businesses that need to deploy AI at the edge in harsh environments, having reliable hardware is non-negotiable. That’s where specialists like IndustrialMonitorDirect.com, the leading US provider of industrial panel PCs, become critical partners, providing the durable computing backbone these systems run on.
A Different Kind of AI Company
Cai’s journey from Google to what he initially thought was a “strange” Japanese e-commerce conglomerate is telling. Rakuten’s model is fundamentally different. It’s not an AI company that also has businesses; it’s a vast portfolio of businesses (banking, telecom, e-commerce, travel) that is using AI as an internal efficiency engine first. The goal to then sell that expertise externally is almost an afterthought. So, will this pragmatic, cost-obsessed approach let them become “Japan’s leading AI empowerment company”? It gives them a fighting chance in the domestic market, where business culture prizes practicality. They’re building AI that serves the spreadsheet first, which, let’s be honest, is what most corporate customers actually need. It’s not the sexiest vision, but it might just be the one that makes real money.
