How to Prepare Unstructured Data for AI

A proof of concept built on 40 hand-picked files can look great. Production is a different matter. Kirk Haslbeck, GM of Unstructured AI at Deasy Labs, walked through what happens when an AI assistant meets an enterprise's real document stores, and how tagging, quality checks and governance applied before the model sees the data change the outcome. Haslbeck spoke at the ALIGN AI Executive Summit New York City 2026 on June 25, in a session sponsored by Deasy Labs titled How to Prepare Unstructured Data for AI.

Haslbeck founded the data quality company OwlDQ, which was acquired by Collibra. He said Collibra acquired Deasy Labs last year to extend its focus from structured to unstructured data.

What went wrong after the proof of concept?

Haslbeck described a large financial services customer that wanted AI assistants for legal and contract operations, answering questions like which supplier agreements are expiring and what their termination clauses say. The customer's AI engineers built a proof of concept themselves, parsing, chunking, embedding and storing 40 curated supplier contracts in a vector database, and got good answers.

The trouble started with real data. Most documents lived in SharePoint, and some companies Haslbeck works with have 40,000 SharePoint sites. Teams didn't know where files were or what they contained. A file named only ".pdf" might be a supplier contract, an HR agreement or an employee contract that should never reach the chatbot.

Why is sensitive data harder in documents?

With structured data, sensitive fields sit in known columns. In documents, sensitive content could be on page seven or page 70, and several harmless sentences can become sensitive together. Every company also defines sensitivity differently. One food company Haslbeck works with considers its recipes confidential, something a general model would not know.

What does data quality mean for unstructured data?

Haslbeck said his structured data quality engine didn't transfer. In documents, quality often means version control: many near-identical drafts where older versions give wrong answers. For contracts that go through redlining, an old version "literally gives you the wrong clause." Quality also covers freshness, completeness of metadata and relevance. Deasy Labs compares documents for near-duplicates, using both file metadata and metadata extracted from the content, to identify the latest version. "Half of our problem is really just removing all the stuff that you really do not want in there," he said.

How does the preparation process work?

Haslbeck demonstrated the workflow:

• Connect and read. The tool crawls sources such as SharePoint, S3 or Dropbox, and users can inspect the OCR, chunks and vectors.
• Build a taxonomy. Teams can start with AI-suggested tags or define their own in plain language, such as legal contracts, NDAs and supplier agreements, then go deeper into subcategories.
• Tag everything. The system applies those tags across the full corpus, including company-specific definitions of sensitive data.
• Create a data slice. Filters narrow the corpus, for example to supplier agreements without sensitive data, and the slice is published for a chatbot to use through an SDK or API.
• Sync metadata back. Tags flow to catalogs and back into SharePoint, so users and tools stay in sync without lock-in.

Because sensitive content is filtered at the preparation and governance layer, he said, teams don't have to rely on engineers adding guardrails to each agent.

Why does this lower cost?

Large context windows tempt teams to send everything to the model. Haslbeck argued that means rereading all documents every time when only the ten most relevant matter. Preprocessing and tagging reduced token costs and response times while improving accuracy, he said.

What were the results?

For the financial services customer, Haslbeck said data preparation took three days. Before, about 2 percent of its data was usable for AI; after tagging, up to 20 percent was. The customer saw higher chatbot accuracy and lower compliance and security risk, and moved a much larger, messier file set into production.

Where to hear more in 2027

Data and AI leaders will keep comparing notes on AI-ready data at the 2027 ALIGN AI Executive Summits, invitation-only gatherings in eight U.S. cities: Atlanta, New York City, Dallas, Chicago, San Francisco, Boston, DC and Los Angeles. Registration is coming soon. See the ALIGN AI Executive Summits page.

Frequently asked questions

Why do document AI proofs of concept fail in production?

They often use a small curated set of files, while real sources contain thousands of unknown, duplicated or sensitive documents.

What is data quality for unstructured data?

Haslbeck said it includes version control, freshness, complete metadata and relevance, not just completeness and accuracy.

What is a data slice?

A filtered, published subset of tagged documents, such as supplier agreements without sensitive data, that a chatbot can use.

This article is based on remarks by Kirk Haslbeck, GM, Unstructured AI, Deasy Labs at ALIGN AI Executive Summit New York City 2026 on June 25, 2026. More coverage: ALIGN AI Executive Summits.

Filed under: Data & AI Infrastructure, AI Governance & Trust, Observability & Data Quality