The big data paradox: We are drowning in data but AI is still starving

Big data – a term most of us probably forgot in the excitement surrounding AI.

Yet, it’s easy to overlook that without big data, most AI models, especially today’s large language models (LLMs), would not exist at all.

Much has been written about the concern that the world may soon “run out of data” to train future LLMs.

For context, the estimated training data size has grown dramatically: from about 570 GB for GPT-3 to roughly 20 TB for GPT-5. And that’s before considering multimodal inputs and synthetic datasets, which could push the total closer to 500 TB to 1 PB for GPT-5.

At the same time, global data generation has exploded. In 2010, the world produced around 2 zettabytes (ZB) of data. By 2024, that figure had skyrocketed to an astonishing 149 ZB.

So, what’s going on here?

On one hand, researchers say we are running out of usable data for training LLMs. On the other, we are sitting on 149 ZB of data.

To put it into perspective, that’s about 150 million times more than the estimated 1 PB used to train GPT-5.

The truth is that most of the world’s data cannot be used to train LLMs at all.

Why Most Data Is Unusable for LLM Training

  1. Enterprise / Proprietary Data
    • Examples: customer databases, internal reports, financial records, CRM data.
    • Why not usable: This information is sensitive, confidential, and legally protected by data privacy regulations.
  2. Copyrighted Material
    • Examples: books, movies, music, subscription-only articles, proprietary research papers.
    • Why not usable: Copyright laws prevent wholesale scraping and reuse without permission.
  3. Personal / Private Data
    • Examples: emails, medical histories, personal photos, location data from phones.
    • Why not usable: Training on such data would violate privacy rights.
  4. Machine-Generated Log Data
    • Examples: server logs, IoT sensor streams, security monitoring feeds.
    • Why not usable: While massive in volume, this data is typically low-level and repetitive.

The Challenge of Unstructured Data

Adding to this complexity, about 80% of the world’s data is unstructured.

  • Examples: social media posts, images, videos, audio recordings, scanned documents, CCTV footage.
  • Why not easily usable: Unstructured data lacks a clear schema. It requires extensive preprocessing, labeling, or transformation before it becomes useful for AI training.

This is why the phrase “running out of data” continues to pack a wallop: it highlights not the shortage of data itself, but the shortage of data that’s actually usable for advanced AI models.

The Real Meaning of “Running Out of Data”

So, next time you hear that we are “running out of data” for LLMs, it doesn’t mean the planet is literally running short of digital information.

What it really means is this: we are running out of high-quality, legally usable, well-structured text and multimodal data suitable for training.

Oh, the world has plenty of data for sure – 149 ZB and counting. The bottleneck is finding the right type of data that can ethically, legally, and effectively fuel the next generation of AI.

If the future of AI is one where we drown in noise but starve for quality, that’s not my idea of a glittering future.

← All blog posts

Turn AI strategy into a working pilot, and prove the ROI.

Start with the free 3-hour AI Strategic Briefing. See agentic AI run real workflows, and leave with an automation roadmap, a prompt library, and a pilot playbook.