May 2025 Newsletter: AI Performance is Earned, Not Given
Navigate the AI landscape with my latest newsletter. Get practical insights for executive decisions that deliver business value. To read my future editions subscribe here and ‘follow’ me on LinkedIn
At Ritchie Bros, a $4B+ equipment auctioneer, we built an ML system “Algo” to automatically value assets from dollars to $100K+ (think Zillow for excavators). Algo outperformed human appraisers and market models while revaluing thousands of assets daily. The success required years of expert partnerships, model refinement, and integration.
Building performant AI systems demands this disciplined approach. This month I highlight companies seeing results from focus and diligence.
AI Implementation: From Hackathons to Continuous Improvement
Model makers are fanning demand through content. Google, OpenAI, and Anthropic (Economic Index and Software Development reports) have all published use cases and best practices, while The Information summarized the 30 most common applications. Google OpenAI Anthropic Anthropic The Information
Johnson & Johnson, after testing 900+ hackathon-like use cases over 2 years, prioritized the 10-15% of initiatives driving 80% of value. Their portfolio now focuses on enabling employees, improving operations, and making strategic bets, with governance devolved to business and functional units. link
Wells Fargo’s AI assistant “Fargo” has served 245M interactions with a privacy-first “poly-model and poly-cloud” approach using mostly open-source, small models. External API calls happen only when necessary, with personal data stripped out. 80% of Fargo interactions are in Spanish. link
McKinsey highlights B2B sales AI applications, including a 10% earnings improvement through price optimization and $1B pipeline growth by mining permit data. link

Rechat, a real-estate marketing platform, slashed LLM error rates from 40%+ to under 5% in six months through rigorous evaluation, while ClassPass, a wellness booking app, raised chatbot satisfaction scores to match human levels, continuously monitoring 10% of interactions. link
From Measurement to Management
Shopify’s CEO has mandated AI usage in performance reviews and requires teams to justify why AI can’t handle tasks before requesting headcount—standards that apply across all levels and are integrated into monthly reviews. What gets measured, gets done. [link](/change%20management/shopify-measurement/

Netflix’s blog on its AI recommendation system is notable for its technical approach — for example, training to maximize long-term engagement and handling cold start videos — and for publishing proprietary innovations. This reflects the importance of allowing data science teams to publicly acknowledge and discuss their innovations — 40 contributors, each with a link to their LinkedIn profile, are listed in the paper. link
Navigating the AI landscape
Stanford’s Human-Centered AI initiative has published its comprehensive 456-page 2025 report, while Forbes, Sequoia, and Meritech Capital whittled down a top 50 AI vendors list from 1,861 companies — a good starting point despite some heavy pruning. HAI link

Last month I discussed auto-regressive image generation from OpenAI and Google is a big deal. OpenAI’s capacity was overwhelmed by demand, with use cases ranging from home design to fake insurance claims. link link
