Feeding ChatGPT your own data? What a knowledge system does differently
You do not need to train ChatGPT on your data, and in most cases you should not. What you actually want is a system that looks things up in your approved documents for every question, answers only from what it finds and tells you where it found it. The technical term is retrieval-augmented generation, or RAG. We call it a knowledge system.
Why "training" is the wrong picture
A language model like the one behind ChatGPT learned from enormous amounts of text how language works and what tends to appear together. That knowledge is spread across billions of numbers. There is no folder called "VX-400 manual" in there and no record of where a statement came from.
Fine-tuning a model on your own documents mostly changes how it answers. OpenAI lists uses such as a consistent output format or a specific task that a smaller model should handle reliably (OpenAI, Model optimization). It does not turn the model into a reference work for your company, for three reasons:
- No source. The model cannot tell you which document a statement comes from, so nobody can check it in seconds.
- Out of date tomorrow. Every new version of a policy would mean training again, and the old version still lingers in the model.
- No read permissions. Whatever goes into the model can be asked out of it by anyone who uses it. You cannot teach a model that service staff may not see a sales framework contract. Nor can you remove a single document again, for instance when someone requests erasure under Art. 17 GDPR.
What a knowledge system does instead
A better picture is an open-book exam. The model does not need to memorise your documents. It needs to find the right passage, read it and phrase an answer from it. That takes four steps:
- Approve sources. You decide which repositories count: manuals, work instructions, cases from the ticket system. They are split into passages, and each passage keeps its document, page, version and the read permissions from the source.
- Search. For each question the system searches only the passages the person asking is allowed to read. It searches by meaning and by exact wording, so part numbers and names are found too.
- Answer. The language model receives the question and the passages found, not your whole archive. It answers only from them and cites a source for every statement.
- Decline. If it finds nothing solid, it says so instead of producing something plausible.
The fourth step matters most. A knowledge system is only as good as its refusals. "The approved documents do not cover this" is a useful answer, because it shows where your documentation has a gap.
Same question, two answers
Here is an example of how this could play out at a machine builder. A service technician asks: "How often does the gearbox oil on the VX-400 series need changing?"
The general chat does not know the series. It answers with a typical range and advises checking the manufacturer's specifications. Not wrong, but no help either. The knowledge system gives the value from your manual, the exception for dusty conditions and the place where both are written. Anyone in doubt is one click away from page 14.
Is an off-the-shelf product enough?
Sometimes, yes. The big vendors are moving the same way. ChatGPT Business and Enterprise can search company knowledge through connected apps and cite sources (OpenAI). Microsoft Copilot answers from Microsoft 365 content the person has at least view access to (Microsoft Learn).
An off-the-shelf product fits when your documents are well kept in SharePoint and Teams and the sharing there is right. Your own knowledge system makes more sense when:
- important sources live outside Microsoft 365, for example in the ERP, a ticket system or a file share;
- you want to decide where data and model run, up to a fully local AI in your own data centre;
- every answer has to cite chapter and page and decline when there is no source;
- you want to measure quality against your own set of real questions, again after every change.
Either way, overly broad sharing becomes visible through AI. Why that happens and what to clean up first is covered in Who can see what?.
What a knowledge system needs
Maintained sources. What matters is not volume but whether someone looks after the documents. If two versions of a policy sit side by side, you get the wrong answer, now with a citation. So every source needs an owner, and outdated versions stay out. If documents exist only as scans or in duplicate, the work starts with data and document preparation.
Read permissions from the source. The system searches only what the person asking may read anyway, and it filters before the answer, not after. OWASP lists missing access control in the search index as a risk of its own for AI applications (OWASP LLM08:2025). This is where most quickly built chatbots fail.
Updates without a second repository. When a document changes, the index follows, superseded versions drop out and the answer states which version it is based on. Nobody copies content into a second system.
A place for the data. Before the first document moves, a data processing agreement under Art. 28 GDPR is in place. We build knowledge systems on PostgreSQL with pgvector, hosted in Germany, and use language models through EU endpoints, for example from Mistral, IONOS, STACKIT, OVHcloud or your own Azure tenant. A model running in your own building is possible; then the operation lives there too.
A test set. About fifty real questions with known answers and sources, confirmed by your department. It is re-run after every change. A "not in the sources" for a question that has no source counts as a pass.
What it costs, phase by phase
The effort depends on the number and state of the sources, how deep the integration goes and what you need in terms of permissions and traceability. So we build in three phases, and each has a price that is fixed before it starts:
| Phase | What happens | Result | Price |
|---|---|---|---|
| AI strategy | Workshop on real cases, review of sources | Roadmap with effort and benefit | Fixed price |
| Pilot & implementation | One use case, acceptance criteria agreed upfront | Running system with test set | Fixed price per use case |
| Operation | Index, updates, measurement | Stable and measured | Annual share of project price |
Current prices are on the linked pages. On top come running costs for hosting and language model usage, which grow with the number of questions. We name both before you decide. If you already know your first use case and its sources, you can skip the first phase.
Your next step
Collect twenty questions that get asked in your company every week, and next to each one note which document holds the answer and who is allowed to read it. If many questions have no document, the knowledge lives in people's heads; writing it down helps more than AI. If it sits in maintained documents, that is a good candidate for a knowledge system. What this looks like in field service is shown in the AI service assistant for technicians.