Table of Contents
The dashboard shows thousands of competitor prices. Updated this morning.
Your analyst just spent two hours copying them into a spreadsheet.
Why? The export has their column names. Your pricing engine needs yours. The tool can't change the format. There's no setting for it. Support says it's on the roadmap. It's been on the roadmap for months.
The data is right there. You're paying for it. You just can't use it.
When we first heard customers describe this problem, we thought they were talking about missing data. Gaps in coverage. Sites the tool couldn't scrape.
That's not what they meant.
What they described was worse. The data existed. It was accessible. They could see it in the dashboard. They could export it to CSV.
But somewhere between the tool and their actual workflow, it got stuck.
We started calling this Trapped Data. And once we named it, we saw it everywhere.
Before we go further, here's how to know if this applies to you:
Can the data land in your pricing system automatically, in your schema — without someone transforming it first?
Are the products that actually matter to you (variants, bundles, non-standard identifiers) matched correctly to competitors?
Can you set alerts based on YOUR business logic — margin thresholds, seller exclusions, regional conditions — not just "price changed by X%"?
If you answered no to any of these, some portion of your competitive intelligence is trapped.
The tool collected it. You can see it. You just can't use it.
Here's what we had to learn the hard way: this isn't about bad tools.
The tools aren't broken. They're doing exactly what they're designed to do. The problem is structural.
SaaS pricing monitoring tools serve hundreds or thousands of customers on the same codebase. To make that work economically, they have to standardize. They build for common patterns — the use cases that most customers share.
This makes perfect sense from their perspective. Custom configuration is margin-killing. If they built bespoke logic for every customer, they'd need an army of engineers and their pricing would be ten times higher.
So they don't. They build a matching engine that works for most products. They build an export format that works for most systems. They build alert logic that covers most scenarios.
And for many customers, "most" is enough.
The problem is when your needs extend beyond that common pattern.
Through working with dozens of companies switching from SaaS tools to managed solutions, we've identified four specific places where data consistently gets stuck.
Before we talk about what goes wrong, let's be clear about what "matching" means in competitive intelligence.
Competitive price monitoring requires connecting each product in YOUR catalog to the same product on competitor websites. Your "Asiatic Albany Diamond Wool Rug 160x230cm" needs to link to their "Albany Diamond Rug by Asiatic - 160 x 230."
Different names. Different SKUs. Same physical product.
Without this connection, you have a list of competitor prices — but no way to know which of YOUR products they correspond to. You can't compare what you can't match.
How tools handle this: Most use standard identification methods — EAN codes, GTINs, ASINs, UPCs. When a product has a barcode, the tool looks for that same barcode on competitor sites. Simple and reliable.
Where it breaks down: Not all products have universal identifiers. And even when they do, not all competitors display them.
This isn't just about missing barcodes. Matching fails for:
When companies come to us after using dashboard tools, we see this pattern repeatedly: automatic matching works well for standard products but struggles with anything requiring name-based or image-based identification. Manual matching queues grow faster than teams can process them.
In non-barcoded categories, it's common to find meaningful mismatches — especially with variants and bundles. The dashboard shows competitor data for those products. The data exists. But it's matched to the wrong product — or sitting in an unmatched queue that nobody has time to process.
If matching breaks, your data isn't just incomplete — it's misleading. And misleading data is another form of trapped data.
What this looks like in practice: A UK home goods company came to us after using a major competitive intelligence platform. They sell rugs — products with color variations, size variations, and no universal identifiers.
The tool's automatic matching connected almost none of their products. Their team was manually matching every item, or simply ignoring the competitor data they were paying to collect.
When we built custom matching using brand name, product line, size attributes, and image similarity, accuracy improved to a level the team could actually rely on. The products were always on competitor sites. The tool was always collecting prices. It just couldn't connect them to anything in the customer's catalog.
This one is subtle but expensive.
Your competitors' product pages contain more than just "price." They show multiple price tiers, stock status, shipping costs, promotional badges, and regional differences.
Standard tools capture what they're configured to capture — usually a single price field, maybe stock status.
But if your competitor has a complex pricing structure and you only capture one number, what exactly have you collected?
Here's what goes wrong: The tool captures the sale price but not the member price. You see a competitor "drop" to $29.99 and react — cutting your own price. But their regular customers are still paying $34.99 through the loyalty program. You just undercut yourself against a price most shoppers never see.
A specific example: A pet products retailer needed to track competitor prices across four sites. Simple enough, except each competitor had multiple price points on every product page: one-time purchase, first subscription delivery, ongoing subscription price, and loyalty member pricing.
Their previous tool captured "price." One number. They had no visibility into which price tier it represented, and no way to compare their subscription pricing against competitor subscription pricing.
The data existed on every page. The tool visited every page. But the schema couldn't capture the complexity — so the decisions they made were based on incomplete information.
You can see everything you need. Right there in the dashboard.
Getting it out is the problem.
API access? Premium tier, or not available at all. Rate limits that make real-time integration impractical. Export formats that don't match what your systems expect.
In tool-to-managed transitions, we hear the same frustrations: API access gated to enterprise tiers, changes requiring support tickets, configuration options that simply don't exist.
So every week, the ritual:
Time spent on data janitorial work
2+ hours every week
If a human has to open Excel every week just to make the export usable, that's delivery friction — and it scales badly.
There's also a governance problem: even when export works, teams struggle to reproduce last month's numbers because configuration changes aren't logged. The tool becomes a black box that nobody fully trusts.
The math on this: 2 hours weekly × 52 weeks = 104 hours annually.
If your analyst's fully-loaded cost is $50/hour, that's $5,200. If it's $100/hour, that's $10,400.
The hourly rate varies. The hours don't.
One furniture retailer told us they were spending 6 hours per week on data transformation and scraper management. That's 312 hours per year of analyst time spent on what is essentially data janitorial work — not analysis, not strategy, just moving information from one format to another.
The data was there. The decisions it was supposed to inform were waiting. But it couldn't get from point A to point B without a human translator every single week.
This is the one that compounds over time.
When you first signed up, the tool fit your needs. Maybe not perfectly, but close enough.
Then your business evolved.
You started selling in new regions and needed location-specific competitor pricing. The tool couldn't segment by geography.
Your product catalog grew from 500 SKUs to 5,000, and the manual matching workflow that was "manageable" became impossible.
You wanted to exclude gray market sellers from your competitive analysis. No filter available.
You needed alerts when competitors dropped below your minimum margin, not just when prices "changed." The alert system didn't support custom logic.
Each of these requests seemed reasonable. Each one got the same answer: "That's not currently supported, but we'll pass it to the product team."
The tool didn't get worse. You outgrew its fixed architecture.
This is important: the gap isn't about the tool being rigid. It's about flexibility having scope.
These tools have flexibility. You can configure matching rules within their framework. You can adjust alert thresholds from their options. You can choose export formats from their list.
But that flexibility has boundaries. When your specific need falls outside those boundaries, you hit a wall. And most businesses with complex catalogs eventually hit that wall somewhere.
At this point, you might be thinking: "This is one company's perspective. How do I know this isn't just sales positioning?"
Fair question. Here's our evidence:
Our transition data is consistent. Over 20 years, we've taken over competitive intelligence operations from companies using various tools. The gaps they describe follow the same pattern:
Your specific numbers will vary based on your product mix and competitor complexity. But the pattern is consistent enough that we stopped being surprised by it.
The economics make it inevitable. SaaS tools have to standardize to maintain their unit economics. Custom configuration at scale is margin-killing. This isn't a criticism — it's just how the business model works. Which means the gap between "common patterns" and "your specific needs" isn't a bug to be fixed. It's structural.
Direct time costs:
The hours vary by operation size, but here's a representative range for a mid-size catalog:
Indirect costs (harder to measure, often larger):
He wasn't complaining about the tool being bad. He was describing what trapped data actually means: you have something, but you can't act on it.
Here's how to assess where data might be stuck in your operation:
Let me be direct about when trapped data matters and when it doesn't.
If the left column describes your situation, a SaaS tool might be exactly right. Don't overcomplicate it.
The larger your catalog, the more complex your competitors, and the more integrated your systems need to be — the more likely you are to hit these walls.
What does "not trapped" look like?
It means the data matches the way you think about your products — not the way a generic tool categorizes them.
It means exports arrive in the schema your systems expect — column names, date formats, currency handling all done before it reaches you.
It means when you need a new competitor, a new data field, or a custom alert condition, someone builds it. Not "submits a feature request" — builds it.
One home goods retailer in the UAE came to us after their existing setup delivered 60-70% coverage. That sounds acceptable until you realize: the missing 30-40% wasn't random. It was their most important competitors — the ones with aggressive anti-bot measures and complex product structures.
Now they get coverage across all their target competitors, delivered weekly, directly into their PowerBI dashboards. No transformation step. No manual reconciliation. Gaps flagged and backfilled instead of silently missing.
That's what untrapped data looks like: it arrives ready to use, and the decisions it informs actually happen.
We're a managed web scraping service. We build custom scrapers for each client, each competitor, each use case.
That means:
What we don't do:
✕ Self-service tools. You're not logging into a dashboard to build scrapers. We handle the technical work.
✕ "Set it and forget it." Scraping requires ongoing maintenance. Sites change, anti-bot systems evolve, your needs shift. We handle the maintenance.
✕ Every use case. We focus on competitive intelligence for e-commerce — price monitoring, MAP enforcement, assortment tracking. If you need social media scraping or non-commerce data, we're probably not the right fit.