Turning Sales Conversations into Research Signal Using AI

Retrieval is only as good as the corpus it searches, and the sales corpus was completely non-standard. Every rep had their own approach to documentation. Some took detailed notes; some left only a date and a company name. Some uploaded transcripts; most didn't.

This mattered because RAG doesn't fix bad data — it faithfully retrieves it. Sparse notes produced sparse embeddings that never surfaced in search. Inconsistent formats meant semantically identical insights were chunked and ranked unpredictably.

Short-term fix — human-in-the-loop validation. Every generated answer included a list of the "most relevant resources" the model drew from. This did two things: it gave researchers and PMs direct access to the underlying conversations, and it let me continuously provide feedback on whether the surfaced resources were actually relevant — which helped refine prompt design and query patterns over time. In practice, retrieval quality depended heavily on how well the model could interpret real-world, messy documentation and still find the right pieces of evidence.

Long-term fix — standardize how intelligence gets captured. UXR and DS teams at Meta already had structured repositories — standardized formats, clean metadata, consistent tagging — so building an AI layer on top of those was relatively straightforward. Sales data required something different: before retrieval, before any of the AI work, we needed a standardized process for how customer intelligence would be captured going forward.

AI did its best at tagging and theming the backlog, but the durable fix was organizational. I worked with sales leadership to establish documentation guidelines — what to capture after each call, how to tag it, what level of detail makes a note useful versus noise. The hardest part was influence without authority: translating what "useful for research and product" meant into something that also made sense for someone managing 40 accounts and a quota.