Services · Building block

One data foundation every system can trust.

Data preparation for AI: we bring together ERP, Excel, mailboxes and drives, resolve duplicates and keep your data up to date.

Data basesynced 06:00
  • ERPHofmann GmbH, Bielefeldspelling differs
  • Sales spreadsheetHOFFMANN GMBH, 33602no customer number
  • MailboxHoffmann GmbH, Industriestr. 12
Hoffmann GmbHCustomer 10442, from 3 sources
Name
Hoffmann GmbHnormalised
Address
Industriestr. 12, 33602 Bielefeldnewer than ERP
Duplicates
2 entries mergedapproved
Text for the knowledge system
  • Terms of delivery, section 4.2version 3, read by sales
  • Warranty 24 monthsvalid from 1 March 2026, service
  • One customer, one number

    Duplicates are found and merged, spellings and units made consistent. Analyses no longer start with manual work.

  • All sources, one base

    ERP, CRM, spreadsheets and file shares flow into one data foundation that is kept in sync. Nobody maintains a copy by hand.

  • Text ready for knowledge systems

    Sections with source, version, validity and read rights. Old versions stay out, permissions stay intact.

  • Quality in figures

    Fill rate, duplicates and contradictions per field, before and after. From then on, fixed rules check every sync.

Where it gets stuck today.

  1. 50 %

    The same data, five places.

    Customers live in the ERP, in the CRM and in the sales team's spreadsheet, articles in the supplier catalogue as well. Each source spells them a little differently, none is complete, and every analysis starts with manual work.

    Bitkom, Sept. 2026: half of the companies using AI name data preparation a major cost driver (603 companies with 20 or more employees)
  2. 60 %

    The AI project that stalled at the data extract.

    The use case was clear, but the fields were empty, duplicated or contradictory. The model was never the problem.

    Gartner, Feb. 2025, forecast: through 2026, organisations will abandon 60 % of AI projects unsupported by AI-ready data (248 respondents)
  3. The knowledge system finds the old version.

    Manuals and policies sit in several versions on file shares and in SharePoint. Without version, validity and read rights on every section, a knowledge system answers from the wrong version or shows what the person may not read.

Merge

One record from many sources.

We read ERP, CRM, spreadsheets and file shares through an interface or an export and match the entries. What belongs together becomes one record; which source leads for which field is your team's call.

Sourcestoday
  • ERP06:00Customer master, 4,812 recordsInterface
  • CRM06:00Contacts, 6,130 recordsInterface
  • Salesyesterdaycustomers_2026_final2.xlsxFile
MatchingCustomer master
  1. Same VAT ID312 possible duplicatesproposal
  2. Spellings1,104 names normaliseddone
  3. Contradictions25 addresses differ between ERP and CRMto review
Leading recordCustomer 10442
CustomerHoffmann GmbH
Customer number10442, from the ERP
AddressIndustriestr. 12, from the CRM
Merged3 entries from 3 sources
Approve

Clean

Clean fields, by fixed rules.

We fill mandatory fields, make spellings and units consistent and set the rules every field has to meet. What was never captured we name, rather than invent it.

  • Normalise

    Addresses, units, date formats and descriptions in one form. "pcs", "pc." and "pieces" become one unit.

  • Resolve duplicates

    Same VAT ID, similar name, different number: the system proposes, your team decides which record leads.

  • Data quality rules

    Mandatory fields, formats and plausibility checks run with every sync. Violations land on a list, not in the ERP.

Customer masterSample values
  1. Duplicates312 found, 287 mergeddone
  2. VAT IDfill rate 64 % to 97 %filled
  3. Addresses25 contradict the ERPto review
Quality rulesCustomer master

Checked on every sync

VAT ID present and in a valid formatRule 1

Postcode matches the townRule 2

14 violations since yesterdayto review

For knowledge systems

Text in sections, with version and read rights.

We split manuals, policies and case files into sections that can answer a question on their own. Each carries source, version, validity and read rights, and a sync keeps it current.

Service file shareSharePoint

Policies and manuals

Warranty policy v3.docxcurrent

Warranty policy v2.docxold version

Maintenance manual HX-200.pdf84 pages

Warranty policyfor the knowledge system

4. Deadlines

Warranty 24 months from acceptance.Section 1

Version 3, valid from 1 March 2026, read by servicecarried over

Version 2 from 2024replaced, not indexed

Synctoday 06:00
  1. New4 documents, 37 sectionsindexed
  2. ChangedWarranty policy, version 3replaced
  3. DeletedPrice list 2024removed

Approaches

The usual routes to a clean data foundation.

Data preparation rarely needs a big tool. Which route fits depends on your systems, the volume of data and who maintains it later.

  • PostgreSQL and our own jobs

    Our default

    Our stack, run in Germany

    Fits when
    data from several systems has to come together and stay clean. We merge it in PostgreSQL, catch duplicates, check it against your rules and keep the texts for the knowledge system current through a queue; a new state only goes live once it has run through completely.
    Mind
    You decide what counts as a duplicate and which system leads; we build it in.
  • Microsoft Fabric with Dataflow Gen2

    Managed, Microsoft

    Fits when
    You already work with Power BI and Microsoft 365. Cleaning happens in Power Query, which many know from Excel, with a schedule for refreshes.
    Mind
    Requires a Fabric or Power BI Premium capacity; the cost depends on its size.
  • Airbyte

    Open source or cloud, sync

    Fits when
    Many sources should flow into one database regularly. Ready-made connectors, incremental sync with deduplication on a key.
    Mind
    It syncs but does not clean; duplicate rules and quality checks come after it.
  • n8n

    Self-hosted, workflow automation

    Fits when
    A few sources should be matched on events, say a new customer in the CRM that has to reach the ERP straight away.
    Mind
    With large volumes, matching and rules belong in the database; n8n then only triggers them.
  • Master data tools in the ERP

    In the system you have

    Fits when
    Your ERP already brings duplicate checks, mandatory fields and approvals for master data. Use those first and set them up properly.
    Mind
    Only covers data inside the ERP; spreadsheets, mailboxes and file shares stay outside.

Tasks and limits.

What the system takes on

  • Reads ERP, CRM, spreadsheets and file shares through an interface or an export
  • Finds duplicates and proposes what belongs together
  • Makes spellings, units and formats consistent
  • Checks every field against your rules on every sync
  • Splits text into sections with source, version, validity and read rights
  • Reports fill rate, duplicates and contradictions per field, before and after

What your team takes on

  • Decides which source leads for which field
  • Approves merged duplicates
  • Defines figures and quality rules once and in writing
  • Captures what was never captured; the system does not create data
  • Approves which file shares go into the knowledge system

Data & documents

Data & documents is the foundation: merged, cleaned, current. Reading single documents is document intelligence; answering from the prepared text is a knowledge system.

Which one when?

  • Document Intelligence

    Does the data still sit in scans, PDFs, drawings or contracts? Then document intelligence reads it, and this is where the fields go next.

  • AI knowledge systems

    Should questions be answered from the prepared text, with the passage they came from? Then a knowledge system on this foundation.

  • Workflow automation

    Should clean records always take the same route between CRM, ERP and approval? Then workflow automation.

The way there.

From the first call to a system in operation. Each phase stands on its own; none ties you to the next.

  1. Intro call

    free

    30 minutes. We work out where AI fits for you and whether an AI strategy is worth it.

  2. AI strategy

    €1,990

    AI strategy for mid-sized companies: which of your processes suit AI, what the build costs and brings back, and which ones are not worth the effort.

  3. Pilot & implementation

    from €5,000

    AI pilot at a fixed price: one use case, fixed scope and acceptance criteria in writing up front. Built in your systems, tested with your data.

  4. Operation & improvement

    from 20% a year

    Running AI systems: hosting in Germany, monitoring, model and prompt maintenance, ongoing measurement, and the next use case on the same foundation.

Further reading

All posts

Frequent questions

Common questions about it.

The questions that almost always come up.

How do you check whether our data is good enough for AI?

Against the use case, not in the abstract. We look at the fields it needs: are they filled, are they consistent, do they exist in every system involved under the same key. What is missing we name, with the effort to fill it.

Does messy data rule out AI?

No, but it is work with a cost. We state what is missing, what exists twice and what contradicts itself before building. An analysis on unclear data is worse than none, because people believe it.

What if the data sits in scans and PDFs?

Then Document Intelligence reads it first, field by field and with a confidence score. This is where it continues: the fields read are matched against your master data and taken into the shared data base.

How does the data for the AI stay current?

Through a sync that pulls in changes from the sources, hourly, nightly or on every change depending on the source. The quality rules run every time, and replaced or deleted documents are removed from the knowledge system as well.

Does AI need a data warehouse?

Not at the start. The entry point is one question from two or three sources, or one area of master data such as customers or articles. A warehouse earns its place when several departments calculate the same figures differently. Then it is the solution, not the entry point.

Where does the prepared data live?

In your systems, or in an environment that belongs to you. Processing in German data centres or inside your own infrastructure, under a data processing agreement under Article 28 GDPR before any data moves.

The next step

Let us find out where AI pays off for you.

30 minutes, free and without obligation.