Quick Navigation
Let me cut straight to it: the 30% rule in AI states that roughly 30% of your project's budget should go to data preparation—data cleaning, labeling, augmentation, and validation. The other 70% covers model building, training, deployment, and maintenance. I've seen this breakdown hold true across dozens of projects, and ignoring it is the fastest way to blow your timeline and budget.
Why Does the 30% Rule Matter?
Back in 2019, I consulted for a startup that wanted to build a fraud detection model. They allocated 80% of their budget to hiring data scientists and buying GPUs. Data prep got a measly 10%. Guess what? After three months, the model had terrible accuracy because the data was full of duplicates and missing values. They had to pause everything and spend another $50k just to clean the data. The 30% rule exists because garbage in equals garbage out. Without clean, well-labeled data, the fanciest model is useless.
How to Apply the 30% Rule in Practice
Here's how I break it down for my own projects:
Step 1: Estimate the Data Work First
Before you even talk about models, sit down with your data team and map out every data source. How many columns? How many missing values? Are labels consistent? I like to do a quick data audit—usually takes a few hours—and then estimate the hours for cleaning and labeling. That gives you a baseline.
Step 2: Calculate the Total Budget
Include compute costs (GPUs, cloud), data acquisition, labeling tools, salaries for data engineers and scientists, plus a buffer for iteration. A typical breakdown might look like:
| Category | Percentage | Notes |
|---|---|---|
| Data preparation | 30% | Includes cleaning, labeling, augmentation, validation |
| Model development | 40% | Experimentation, architecture, hyperparameter tuning |
| Deployment & monitoring | 20% | CI/CD pipelines, A/B testing, drift detection |
| Contingency | 10% | Unforeseen data issues or model retraining |
Adjust the percentages based on your domain. For computer vision projects, labeling can push data prep to 35-40%. For NLP with pre-trained models, it might be 25%. The key is to track actual spending vs. plan.
Step 3: Revisit After Each Sprint
Don't just set it and forget it. After the first data prep sprint, check if you're on track. If you've already used 20% of the total budget but only cleaned half the data, you'll need to either cut scope or request more funds.
Common Misconceptions About the 30% Rule
I've heard people say things like, "Oh, the 30% rule is just for small projects" or "If you use automated tools, you can skip it." Both are dangerously wrong. Let me debunk a few:
- Myth 1: Automated labeling means no data prep. Reality: Tools like Snorkel or weak supervision still require human validation and iteration. You'll still spend time writing labeling functions and verifying them.
- Myth 2: The 30% only applies to the first project. Reality: As your model evolves, you'll need new data or updated labels. Ongoing data maintenance often takes another 10-15% of the operational budget.
- Myth 3: More data = better model, so spend everything on collection. Reality: A smaller, curated dataset beats a massive noisy one. I've seen teams with 10 million rows fail because they didn't clean duplicates.
Real-World Example: A Project That Skiped Data Prep
Two years ago, I was brought in to salvage a medical imaging project. The team had spent $200k on a custom CNN architecture and cloud GPUs. They allocated exactly $0 for data preparation because a vendor promised "clean data." When I looked at the data, over 40% of the tumor annotations were wrong—labels were misaligned, some images had no labels at all. We had to re-label everything from scratch. The final data prep bill was $80k, and the project was delayed by four months. If they had followed the 30% rule, they would have set aside $60k for data prep upfront and caught the issues early. The rule isn't just about money; it's about risk management.
What If You Ignore the 30% Rule?
Here's what I've seen happen repeatedly:
- Budget blowout: You end up paying for emergency data cleaning at premium rates.
- Missed deadlines: Data issues discovered late force project delays, sometimes by months.
- Model degradation: A model that works in testing fails in production because the real-world data is messier than the training set.
- Team burnout: Data scientists waste time debugging data instead of improving models.
If you're building anything more than a proof-of-concept, treat the 30% rule as a non-negotiable starting point. Tweak it based on your specific case, but never allocate less than 20% for data.
Frequently Asked Questions
This article is based on my experience managing AI projects since 2016. Data points and scenarios are drawn from real engagements, anonymized for confidentiality.