Most conversations about RAG governance focus on retrieval, because that is where the leak becomes visible. But retrieval can only enforce a decision that someone already encoded into the data. If a chunk arrives at retrieval time wearing no label, the policy engine has nothing to reason about. Sensitivity has to be understood when the content is first processed, not improvised at the moment a query lands. That is why governed RAG quietly begins at ingestion.
The unit of governance is the chunk, not the document
RAG already breaks documents into chunks so they fit a model's context window and so retrieval can find the most relevant passage rather than the whole file. The mistake is to govern at a different granularity than you retrieve at. If retrieval works on chunks but permission works on documents, the two will disagree, and the disagreement always resolves in favor of exposure.
Real documents are mixed. A board pack contains public talking points and confidential numbers. A customer contract contains a standard template and negotiated terms that are commercially sensitive. A single research report contains a published abstract and unreleased findings. Label the whole document as confidential and you lose the useful public parts to over-blocking. Label it as open and you expose the sensitive parts. Neither is acceptable, and both are what you get when the unit of governance is the file.
Classifying at the chunk level dissolves the dilemma. The public talking points stay retrievable for everyone. The confidential numbers carry a sensitivity label that retrieval policy can act on. The same document serves a wide audience and a narrow one correctly, because each piece of it carries its own truth about who may use it.
What a useful classification actually captures
A sensitivity label that only says "confidential" or "public" is a start, but it is rarely enough for AI. Useful classification captures both how sensitive a chunk is and what it is about, because policy often depends on meaning, not just a sensitivity tier. A chunk might be flagged as personal data, as financial information, as legally privileged, or as material non-public information, and each of those categories can carry different rules for different roles and purposes.
This is the difference between a label and a meaningful label. "Confidential" tells you to be careful. "Compensation data, employee-identifiable" tells the policy engine exactly which rule to apply and to whom. The richer the classification, the more precise the retrieval decision can be, and precision is what lets the same knowledge base serve finance, HR, and engineering without three separate copies.
Doing this at the scale of real data
The obvious objection is that no one is going to hand-label millions of chunks. They are not, and they should not have to. Classification at ingestion has to be automated to be real, which means applying classification as content flows through the pipeline rather than as a manual project that finishes never. It also means classification has to be repeatable, so that when a document is updated or re-ingested, its chunks are re-evaluated rather than left with stale labels.
The reason this matters for governance is durability. A label applied once and never revisited drifts away from the truth as the business changes. Classification that runs as part of ingestion, every time, keeps the labels honest, and honest labels are the thing retrieval policy depends on. This connects RAG governance to the broader practice of data governance, where classification and lineage are managed as ongoing functions rather than one-time clean-ups.
The payoff at retrieval
When classification is done well at ingestion, retrieval becomes simple to reason about. The policy engine is no longer guessing. It looks at the chunk's sensitivity, the requester's role, and the purpose of the request, and it makes a clean decision: allow, redact, annotate, or deny. Every hard part of that decision was front-loaded into ingestion, where there was time to do it carefully.
This is why chunk-level sensitivity is the quiet foundation of governed RAG. It is not the part anyone demos, but it is the part that makes the demo safe to ship. The retrieval story it enables is covered in relevance is not permission, and the whole pipeline is laid out in the complete guide to governed RAG.