Aug 12, 2026 | DataPro#182

👋 Hello ,

Welcome to DataPro 182. 

A model can look exceptional in a notebook and still fail quietly in production. This week’s expert deep dive gets into why. Yusuf Ganiyu, Senior Data Engineer at AstraZeneca and Founder of Data Mastery Lab, explains why the model may be only 5% of the real ML system, and why feature freshness, training-serving skew, and the choice between batch and streaming can ultimately determine whether predictions remain useful in the real world. His key argument is refreshingly practical: you don’t need to stream everything. You need to know which features become stale fast enough to change a decision. 

Beyond the expert article, this week’s research roundup points to a broader shift across data and AI: infrastructure is being redesigned around agents, local models, governed data, real-time intelligence, and increasingly autonomous workflows.

Here’s what else is inside:

The thread connecting it all: AI engineering is moving beyond the model. The competitive edge is increasingly in the systems around it, from fresh features and governed context to agent orchestration, retrieval, infrastructure, and deployment.

Subscribe | Submit a tip | Advertise with Us

Cheers,

Merlyn Shelley

Growth Lead, Packt

A machine learning model can achieve excellent accuracy in a notebook, pass validation, survive technical review, and still make the wrong decision the moment it meets the real world.

Not necessarily because the algorithm failed.

Not because its weights suddenly changed.

And not because the prediction endpoint went down.

The failure can begin somewhere much less visible: the features reaching the model no longer represent what is happening now.

That was the problem at the heart of “Why Your ML Pipeline Needs Stream Processing,” a free technical webinar hosted on the Packt DataML YouTube channel.

Hosted by Abhishek, the session brought in Yusuf Ganiyu, Senior Data Engineer at AstraZeneca and Founder of Data Mastery Lab, for a practical exploration of batch versus real-time machine learning pipelines.

Ganiyu’s background makes the discussion particularly relevant to production-focused engineers. Alongside building enterprise data systems, he teaches data engineering through Data Mastery Lab and his Code With You YouTube channel.

But this was not a session built around the simplistic claim that every ML team should abandon batch processing and move everything to Kafka.

Quite the opposite.

Ganiyu’s argument was considerably more useful:

You do not need to stream everything. You need to understand which features become stale quickly enough to hurt the decisions your model makes.

From there, the session moved through training-serving skew, feature freshness, a live batch-versus-streaming fraud detection experiment, Kafka and Redis-based feature engineering, production architecture, migration strategy, infrastructure choices, cost considerations, CDC, and the difficult question every ML engineering team eventually has to answer:

When is batch actually good enough, and when does the business need real-time data?

The answer ultimately leads to a provocative reframing of production ML:

The model is not the product. The feature pipeline is the product.

And understanding why Ganiyu makes that claim requires looking at what actually happens to an ML system after the notebook closes.

What Yusuf Ganiyu Set Out to Answer

Early in the Packt DataML session, Ganiyu laid out a deliberately practical agenda.

Rather than spending the webinar comparing streaming frameworks or discussing architecture in the abstract, he structured it around the problems engineers encounter when ML models move into production.

The session would examine:

That last point matters.

The question posed by the webinar title might appear to be:

“Why does your ML pipeline need stream processing?”

But Ganiyu gradually replaces it with a much better question:

“Which parts of your ML pipeline need stream processing?”

That distinction changes the entire architecture conversation.

The Real Production Problem Is Often Not the Model

One of the strongest ideas in Ganiyu’s session is that ML teams can spend enormous amounts of effort optimizing the model while underestimating the system responsible for feeding it.

In development, the workflow can feel deceptively simple.

Prepare the dataset.

Split it into training and test sets.

Train the model.

Evaluate accuracy, precision, recall, or whatever metrics matter for the problem.

Package the model.

Deploy it.

But production introduces a completely different class of problems.

Ganiyu argues that the model itself may represent only around 5% of the complete production system, while the other 95% is made up of features, pipelines, serving infrastructure, and the machinery required to keep those pieces operating together.

The exact percentage is less important than the engineering principle behind it.

A production ML system is not simply:

Data → Model → Prediction

Keep reading: the full webinar recap, examples, and technical takeaways are waiting on Packt Medium.

⭕ Filestore file service runs on Colossus: Google Cloud is giving Filestore an AI-era upgrade. Built on Colossus, the infrastructure behind YouTube, Gmail, and Gemini, Filestore can now scale capacity and IOPS independently. Deep GKE integration and multishares enable efficient container storage, while shared NFS workspaces support millions of collaborating AI agents. Add faster recovery, zero-downtime scaling, IAM controls, and lower TCO for enterprise workloads. 

⭕ Introducing Muse Glimmer: An Open Agentic Model That Runs on Your Device: Meta is bringing agentic AI closer to home with Muse Glimmer, a 30B open-weight model designed for always-on local agents. Built to run on a single consumer GPU, it combines long-horizon reasoning, reliable tool use, coding, multimodal inputs, failure recovery, and 100+ languages. Apache 2.0 licensing, quantization, and upcoming llama.cpp, MLX, and ExecuTorch integrations make local deployment increasingly practical. 

⭕ How ONESTRUCTION built the Ishigaki-IDS foundation model with AWS GenAIIC: AWS and ONESTRUCTION are showing how specialized AI can thrive even when data is scarce. Ishigaki-IDS, built on Qwen3, combines expert-generated synthetic data, continued pre-training, supervised fine-tuning, and RLVR with automated verification. The payoff: near-100% structural compliance, 80%+ content consistency, and support for roughly 120K-token contexts, powered by scalable AWS distributed training infrastructure. 

⭕ Integrating Looker and Gemini Enterprise: Google is turning Gemini Enterprise into a governed front door for business intelligence. With Looker’s semantic layer, employees can query structured and unstructured data conversationally while getting consistent, business-approved metrics. Looker agents connect through A2A, preserve OAuth and row-level permissions, generate deterministic SQL, surface interactive charts, and collaborate with other agents, bringing trusted analytics directly into everyday AI workflows. 

⭕ How Pixieset achieved 35% AI feature adoption by solving the right problem with Amazon Bedrock: Pixieset found an AI use case photographers actually wanted: automating the work around their craft, not the craft itself. Using Amazon Bedrock, it launched AI-generated alt text in four months, hit 750,000 photos in week one, and sustained 35% adoption. The formula was simple: solve tedious work, ship quickly, preserve user control, and let trust build gradually. 

⭕ Deep dive on new AI-powered database agents: Google Cloud wants database management to feel less like firefighting and more like collaboration. Its new Database Onboarding and Observability Agents use Gemini to recommend databases, generate configurations, monitor fleets, correlate telemetry, diagnose root causes, and suggest or execute approved fixes. With support across Cloud SQL, Spanner, AlloyDB, and Bigtable, plus console, IDE, and MCP integrations, they act like always-on virtual DBAs. 

⭕ nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4: NVIDIA is pushing powerful agentic AI onto a single GPU with Nemotron 3.5 Lightning. The 30B-parameter MoE model activates just 3B parameters, supports a massive 1M-token context, and runs on DGX Spark or H100 hardware. Built for autonomous agents, coding, reasoning, and tool use, it combines Mamba-2, attention, speculative decoding, NVFP4 quantization, and multi-environment reinforcement learning for efficient deployment. 

⭕ Accelerate PostgreSQL Migrations with Gemini in DMS: Google Cloud is tackling database migration’s toughest last mile with Gemini-powered code conversion in Database Migration Service. Instead of manually rewriting years of Oracle PL/SQL or SQL Server logic, teams can convert stored procedures, triggers, and functions into PostgreSQL faster. Full-schema context, deterministic rules, AI explanations, automated validation, IAM security, and human review turn months of modernization work into potentially days. 

⭕ Prompt Engineering vs Loop Engineering vs Graph Engineering: What Changes at Each Layer: AI engineering is moving up the stack. Prompt engineering controls a single response, loop engineering turns one agent into a repeatable observe-act-verify cycle, and graph engineering orchestrates multiple agents and workflows. The catch: each layer keeps the one beneath it. Start with prompts, add loops only when “done” is mechanically verifiable, and reach for graphs when work genuinely needs parallel, coordinated agents. 

⭕ BigQuery Search innovations: Unify structured & unstructured data: BigQuery is turning unstructured data into something AI agents can query natively. Autonomous Embeddings now continuously vectorize text and images, while AI.SEARCH delivers natural-language retrieval with up to 133× better slot efficiency. New Hybrid Search combines semantic understanding with precise keyword matching. Together, they simplify RAG pipelines, improve retrieval accuracy, reduce hallucinations, and bring document-to-conversation workflows directly into BigQuery. 

⭕ Introducing LTX-2.5: The Open World Model for Video, Real-Time, & Physical AI: LTX is pushing world models beyond video generation with LTX-2.5, an open-weights model built for film, real-time experiences, and physical AI. It brings sharper high-motion visuals, native multishot consistency, stronger prompt understanding, robotics adaptation, and faster distilled inference. NVIDIA optimizations reduce local VRAM needs, while day-one ComfyUI support gives builders control over hardware, customization, data, and IP.

See you next time!

Copyright (C) 2026 Packt Publishing. All rights reserved.

Our mailing address is:

Packt Publishing, Grosvenor House,

11 St Paul's Square, Birmingham,

West Midlands, B3 1RB, United Kingdom

Want to change how you receive these emails?

You can update your preferences or unsubscribe