The hard part of mapping customer data for AI agents
For a while now, I've been working on how a new customer's data gets mapped into our platform, so our AI agents can use it.
Earlier, this took us 8-10 days per customer. We review the mapping sheet the stakeholder shares, go back and forth to understand what the data actually means, update our schema if needed, and then build the dbt pipeline with tests.
I think the hard part is mostly that first step: understanding the data. The same thing can be named differently across customers. For one, the account number is called a file number. And sometimes values just can't be right, like a birth date later than today's date.
I'm writing a two-part series on these problems and the questions we try to answer before writing any pipeline. Part 1 is in the comments.















































