OpenAI has made file-based retrieval look almost too easy. Create a vector store, upload files, and the platform handles chunking, embeddings, and indexing for you. That is great for getting a demo running. It is not the same as having a knowledge assistant you would trust inside a real business. The same documentation that makes the setup look simple also makes the production challenge obvious: retrieval stops being mainly an API problem and turns into a governance problem. File search can return answers with file citations, and you can also choose to include the underlying search results. That is enough to show something working. It is not enough to prove the answer came from the right version of the right document for the right audience.
That is the real shift once a team moves past proof of concept. Retrieval quality depends less on model prompting and more on the condition of the source material. OpenAI's retrieval guide explains that vector stores sit behind semantic search and that files are automatically chunked, embedded, and indexed when they are added. Semantic search is useful because it can find relevant passages even when the wording in the query and the document do not match neatly. But that same strength becomes a liability if the corpus is full of duplicates, outdated policies, mixed jurisdictions, or drafts with no clear status. A passage can be semantically close and still be operationally wrong.
Why file search becomes a governance project
OpenAI's file search tools expose the controls that matter in production. Responses can include citations, but raw search results are not returned unless you ask for them. The retrieval guide notes that a search returns up to 10 results by default, configurable up to 50, and it supports query rewriting, attribute filtering, and ranking controls such as ranker choice and score thresholds. Those are not minor tuning options. They determine which documents are even eligible, what the model is encouraged to trust, and how much weakly related context you are willing to let it blend into an answer.
The practical question is not whether OpenAI can search files. It can. The practical question is whether your organisation can explain why a given answer appeared, which documents were in scope, how stale content is excluded, and how retrieval is inspected when someone challenges an answer. If that cannot be explained, the system is not really governed yet. It is still an indexing experiment.
The source corpus usually needs work before it is safe to index
OpenAI supports a wide range of file types for file search, including PDFs, HTML, common office documents, code, Markdown, JSON, and plain text, plus text encodings such as UTF-8, UTF-16, and ASCII. That flexibility is useful, but it also makes it easy to upload everything that happens to exist. Most internal documentation is messier than teams expect: repeated exports, obsolete versions, meeting notes sitting beside approved policies, language variants, and files that are technically supported but structurally poor for retrieval. Vector stores can only index what they are given. Cleanup is not a nice-to-have. Naming, deduplication, version rules, language normalisation, and basic document hygiene all shape what the model can later cite with confidence.
There is also a basic operational point teams miss early. File operations are asynchronous. The vector store file reference exposes processing status, last error, and usage bytes, so an uploaded file is not automatically a ready file. If some documents are still processing, some have failed, and others were indexed under an older workflow with unknown chunking behaviour, the knowledge base is already inconsistent before the first user asks a question.
Metadata design is where retrieval becomes governable
OpenAI exposes attributes on vector store files and supports filtering in both the retrieval API and the file search tool. The documentation shows straightforward filters and compound logic using operators such as eq, in, and, and or. This is where a document dump becomes a usable retrieval layer. If you label files by region, business unit, confidentiality, document type, effective date, language, or content status, you can narrow the candidate set before semantic search starts ranking passages. If you skip that step, the model may end up choosing between documents that should never have been competing with each other in the first place.
- Good metadata reflects decision boundaries, not just folder names.
- The most useful attributes are the ones that restrict scope before synthesis begins.
- Filters should map to real business questions such as jurisdiction, audience, freshness, and sensitivity.
The vector store reference also allows metadata on the store itself, along with names, descriptions, status, usage tracking, and expiration policies. That matters because governance happens at two levels: the individual file and the collection that contains it. In many cases, one large store called internal docs is the wrong design. Different departments, languages, confidentiality levels, or lifecycle rules may justify separate stores.
Chunking choices are not neutral
OpenAI's vector store reference says the default auto chunking strategy currently uses a maximum chunk size of 800 tokens with 400 tokens of overlap. It also supports a static strategy, with configurable overlap and a maximum chunk size between 100 and 4096 tokens, where overlap cannot exceed half the chunk size. That is not an implementation footnote. Chunk size affects how much context stays together, how passages compete in ranking, and how clearly a retrieved excerpt maps back to a coherent section of a source document.
A policy manual, a technical runbook, and a slide deck export do not behave the same way under retrieval. Smaller chunks can improve precision, but they can also separate the instruction from the caveat. Larger chunks preserve context, but they can weaken ranking. Overlap reduces boundary problems, but it also increases redundancy. The sensible setting depends on the document class first and the parameter value second.
Lifecycle discipline matters because storage is billable and deletion is not instantaneous
OpenAI's pricing makes sloppy ingestion expensive. File search storage is billed at $0.10 per GB per day after the first free gigabyte, and tool calls are billed separately. The vector store object tracks usage bytes, and vector store files have their own usage bytes as well, which may not match the original file size. Once storage has a meter on it, every duplicate upload, stale archive, and forgotten export becomes both a quality problem and a budget problem.
The lifecycle controls matter just as much. Vector stores support expires_after policies anchored to last_active_at, with a configurable number of days. The retrieval guide states that when a vector store expires, its associated files are deleted and charges stop. It also notes that file removal is eventually consistent, so removed content may still appear in search results for a short time. That is exactly why a production setup needs maintenance rules, replacement procedures, and clear expectations around freshness windows.
What competent implementation looks like
A solid setup usually starts with an audit, not a chatbot. First decide what should be searchable at all. Then normalise filenames, remove duplicates, define version rules, and choose which documents belong in which vector stores. After that, design attributes around how the business actually needs to constrain answers. Tune chunking by document class, not by habit. Expose citations to end users, include raw search results in testing and QA, and review ranking thresholds when the assistant starts sounding plausible but vague.
That is the kind of work Greg at GrN can help with without turning the stack into a science project: audit the source corpus, normalise files, design vector-store structure and metadata, tune retrieval settings, add citation checks, and connect the workflow to a CMS or internal operations system with rules a team can maintain. The underlying technology is already capable. The value comes from making it dependable.
OpenAI has made retrieval much easier to ship. What remains is the part that businesses cannot skip: deciding what knowledge enters the system, how it is segmented, how it expires, how it is filtered, and how the organisation verifies that an answer was grounded in the right material. At that point, this is not just RAG plumbing. It is retrieval governance.
Need help with this kind of work?
Plan your retrieval-governance setup Get in touch with Greg.