Data Silos are disturbing your view of the world

We've come a long way, baby!

(If you now have the Virginia Slims jingle stuck in your head, we're getting to that)

I know, everyone's in a hurry, but think for a moment, what we achieved:
From punch-cards to magnetic tapes, floppy disks, and hard drives, SQL, NOSQL, data warehouses, data lakes, and data fabric.
Since humans have harvested crops in the Nile delta, we have some kind of records to prove what we have done, what others have done, and what we have moved around.

Every technological evolution promised to improve our experience, faster, better, more accurate.
Now it is 2025, roughly 7000 years after the first poor soul had to create data about crops in ancient Egypt. 7000 years should be more than sufficient to get our data challenges solved, right?

Do you feel we managed to do so?

• Does your marketing and sales team still battle over whose data is less wrong?
• Did you spend 2 hours making sense of attributions that do not make sense
• Do you look at your data but still make critical decisions based on gut feeling because, deep down, you do not trust your data?

You are not alone.

Obfuscating (data) reality

Never before, even in ancient Egypt, have we had so much data at our fingertips. We can see who clicked where and when, and we can try to map an outcome to these actions. Every tool we use collects some sort of data to explain the world.
And let's be honest. We all bought into the story. The days of cigarette smoking, creative marketers who could not tell you if the great campaign had any effect, are long gone. Online marketing is the promised land, everything can be measured. IoT gives us insights into non-digital assets. ERP, CRM, record everything, and make sense of the business world.

There might have been a short period where this was true. Now we deal with different legislatures that have different understandings of what data is and how we can collect and use it. We deal with different platforms, wooing our hard-earned marketing budget in their ecosystem. And all bring the best bang for the buck. If the explanations get more complicated and platforms start arguing over 0.23 conversions, one might have doubts. Is the platform that wants my marketing dollars the right place to judge if the money is spent well?

The End of Platform Trust

Every platform offers its own analytics, some more sophisticated, some more basic. But all have in common that they show you data that only allows one conclusion: This platform is the best thing since sliced bread.
So in the end, we go back to more traditional tools: The clay tablet of our times, Excel.
We export csv, xlsx, sometimes even JSON, and try to make sense of the data.

Some studies say 80% of analysts' time is wasted with data cleaning and prep, VLOOKUP have mercy on our souls.

The Single Source of Truth

Single Source of Truth or SSoT gets thrown around a lot. To a degree, we know from past buzzwords like data warehouse, data lake, chalkboard, clay tablet. All these technological innovations should address the same problem: dealing with disparate data sources and gluing them together so they paint one picture of what is happening and what is being measured.
The business definition captures this very well:

A centralized, trusted repository where all your business data from every source is aggregated, cleaned, and made universally accessible.

Damn, that's a mouthful.

Let's have a look at what it is meant to do, perhaps then it gets clearer why we are still stuck in defining a Single Source of Truth after 7000 years of data wrangling.

Trust & Consistency: When everyone, from marketing to sales to finance, works from the same data, the arguments over "whose numbers are right" disappear. You could say, if no one has access to conflicting data, they cannot doubt your data. The reality is, you need robust data. There is no way around it. Making decisions based on biased or skewed data is worse than flipping a coin.

Efficiency: This is a real one. Everyone who works with data knows the fun of data cleaning, especially when working with data-illiterate people. We've all received an image of a .csv table in a Word document at some point and been asked if we can do something with this data. Data Scientist was the sexiest job of the 21st century until people noticed that you spend 70% of your time dealing with data issues and not building that sexy model.

Having a robust ETL pipeline that frees up analyst time cannot be bought with gold. (Nevertheless, we tried to put a number on it. Try our data hub calculator to determine the cost of data cleaning for you).

Skill gap: Small to medium-sized organizations often do not have a dedicated analyst. Marketing or salespeople need to do it besides their regular job. Given, chatGPT helps a lot with that, but this might be a reason why most reports are not statistically sound and valid.

Same Understanding of the World: With data silos, each organizational department sees only its little piece of the world. Data does not necessarily match up. Marketing starts blasting sales, why they do not turn the leads into revenue. Often, sales has no idea what marketing is talking about because the leads it measures don't show up. If everyone can see what happens outside their small, prescribed part of the world, they get an understanding of why not everything is so black and white. Sometimes, employees might even be empowered to see cross-functional issues and can begin resolving them.

The Mnemonic Data Hub

If you've followed me this far, you might have noticed that we've had a lot of data issues in our careers. Be it on the marketing side, analytics, data science, or machine learning, in government, in academia, in business.

We know the feeling, "knowing' something is off with data, but not being able to prove it. We know how it is to bump heads, because someone is gatekeeping "their" data. Let's not mince words: It sucks.

Now in a start-up role, we see that a lot of organizations that want to stay relevant are trying to adapt to AI. But what does AI need above everything? Correct, data. So we wrestle with organizational departments to get data, perhaps even data dictionaries, navigate internal policies and politics. This is tiresome and can lead to subpar results.

We tried to find the common denominator in data sources we see over clients of all sizes and built a safe and robust way to connect all these data sources to one platform. The connection part was pretty easy. OAuth is a robust, tried-and-tested technology that most players support. You have probably used it today without thinking about it. Whenever you sign in to a service using credentials from another service, you're probably using OAuth. Sign in with Google, Facebook, Apple - Under the hood: OAuth.

Once you have connected your services to the data hub, you select which data you want to share, and off we go.
But what about all the rest, matching data, cleaning it, making it ready for analysis?

Unification Layer

The main problem, as we see it, is not getting some messy, dirty data, but cleaning it and making it accessible for further analyses. This is why we created an AI unification layer.
Cool word, but what does a unification layer do?

In short, it takes data from different sources and merges them. A lot of people might not know that something as simple as "Click" is something completely different when it comes to Google Ads and Meta Ads. While Google is pretty straightforward, a click is when a user clicks on an ad, Meta gets more philosophical: A click can be a like, a reaction, a share, perhaps even a comment. Or perhaps just to expand the text.
Naturally, if you just merge these, you get a very skewed picture of what happens.

Another example is currencies. You sell products in US dollars, Euros, and GBP. If you export the numerical values without accounting for exchange rates, you might be in for a surprise.

We will go into much more detail in another article, describe what we can do out of the box, and what YOU can do (semantic layer).

For now, we keep it at the data hub turns your data into an active asset, not a nuisance.

If you have data challenges, we love to hear about them. Why not drop us a message?