SharePoint data migration: moving documents AI can use

Most organisations hold years of documents across SharePoint sites, network file shares and OneDrive accounts. Some of that content is current, much of it is not, and few people can say which is which. A migration to SharePoint Online means every library gets handled once. That makes it the best moment you will get to decide what deserves to move and what does not.

SharePoint data migration is the process of moving documents, lists, libraries, metadata, versions and permissions from one SharePoint environment, file share or tenant into another, usually SharePoint Online in Microsoft 365. A good migration inventories the estate first, cleans it, maps permissions and metadata, moves content in waves, then checks the result before the old system is switched off.

This guide is for the person who owns the migration: an IT lead, a head of data or a digital programme manager who has been asked to move the document estate and wants it to be useful afterwards. It covers the mechanics of the move. It also makes one argument. If you plan to put a search or retrieval-augmented generation (RAG) system over these documents later, the migration is where you fix the problems that would otherwise turn into wrong answers.

We are Shipshape Data, a London AI and data consultancy. We build the data foundation that AI depends on, and document estates are part of that foundation. The advice below is the order of work we recommend to clients before anyone points an AI tool at their files.

What a SharePoint data migration involves

A SharePoint data migration moves content and its context. The content is the files, list items and pages. The context is everything that makes them usable: who can open them, which metadata describes them, which versions exist, which retention rules apply and who created them. Moving files without their context is quick. It is also the most common reason a migration looks finished and still fails the people who use it.

Typical sources are SharePoint Server sites held on premises, network file shares, another SharePoint Online tenant after a merger or divestment, and other cloud storage. The target is usually SharePoint Online and OneDrive in Microsoft 365. As with any software data migration, most of the effort sits in the preparation and the checking, not in the copy itself.

The five workstreams

• Inventory and clean-up: know what exists, who owns it and what should move.
• Permissions: map users and groups, and decide who should see what in the new structure.
• Metadata, versions and retention: carry over the information that describes each file and the rules that govern it.
• Cutover: move content in planned waves, with a short freeze and a clear switch-over date.
• Verification: prove that what arrived matches what left, then retire the source.

Treat the move as a piece of work with an owner, a plan and a written definition of done. The stages in our data migration framework (assess, map, build, reconcile, cut over and support) apply to documents as much as to databases.

Inventory and clean-up before you move anything

Start with a full inventory of the source. Migration tools and the SharePoint admin centre can produce much of this, and a short script against a file share fills the gaps. The aim is a list you can make decisions from, not a report that sits in a folder.

What to record for each site or library

• Owner: a named person who can decide what happens to the content.
• Size and file count, so you can plan waves and estimate run time.
• Last modified date, which is the quickest signal of whether anyone still uses it.
• File types, including blocked types and very large files that need special handling.
• Unique permissions, meaning any folder or file that does not inherit from its parent.
• Path length and characters that the target does not accept.
• Any retention, legal hold or records obligation attached to the content.

Decide: migrate, archive or delete

Sort every library into one of three groups. Migrate the content people use and need. Archive content that must be kept but is no longer worked on, into a separate location with restricted access. Delete content that is redundant, obsolete or trivial, once the owner has agreed and you have checked it is not under a retention obligation.

Duplicates deserve particular attention. Most estates hold several copies of the same policy, contract template or price list, saved in different places and edited at different times. Pick the authoritative copy, record where it lives and remove or archive the rest. If you skip this step, every copy moves, and every copy becomes something a search tool can return.

A migration moves your problems with your files, unless you decide during the move to leave them behind.

Clean-up is the step most often cut when a deadline tightens. We recommend protecting it. It reduces the volume you move, shortens the cutover window and makes every later step simpler.

Permissions and why they matter more for AI

Permissions decide who can open each file. In an older estate they are rarely tidy. Years of urgent requests leave folders shared with individuals who have since changed role, groups that nobody maintains and broad access granted to fix a single problem.

What usually goes wrong

• Broken inheritance: thousands of files with their own permissions, which slow the migration and are hard to audit.
• Orphaned accounts: leavers whose access never got removed, which cannot be mapped to a current user.
• Over-broad sharing: libraries open to the whole organisation because that was the fastest fix at the time.
• Sharing links: anonymous or organisation-wide links that carry access outside the permission model.
• Identity mismatch: on-premises accounts that do not map cleanly to Microsoft Entra ID accounts in the target.

Build a mapping file from source users and groups to target users and groups before the first wave. Use the migration to reset permissions to a simpler model where you can: access granted at site or library level, through groups that have named owners, with unique permissions kept only where there is a reason.

Why AI makes this urgent

Before AI, an over-shared folder was a risk that mostly stayed hidden, because people had to know where to look. A search or RAG system removes that protection. A well-built system respects permissions and returns only what the user can open, but that means it returns everything the user can open, including the salary spreadsheet sitting in a folder shared with the whole company. A system that ignores permissions is worse still.

Search shows people what they are allowed to open. Fix what they are allowed to open first.

The practical rule is simple. Treat the permission model in the target as the access policy your AI tools will inherit, because that is what it becomes. Review sensitive areas such as HR, finance, legal and board material by hand before they move.

Metadata, versions and retention

Metadata

Metadata is the information that describes a file: its author, created and modified dates, content type and any columns such as client, department or document status. Decide which fields must survive the move and test that your tool preserves them, especially original authors and dates, which are lost easily and matter for audit.

Many file shares hold their metadata in folder paths, for example a client name, then a year, then a document type. The migration is a good point to turn that structure into proper columns or managed metadata. Tagged files can be filtered, governed and retrieved accurately. Files known only by their path cannot.

Versions

SharePoint keeps a version history for each file, and old libraries can hold a great many versions. Migrating every version multiplies the volume and the run time. For most content, a sensible cap on recent versions is enough. Where versions are evidence, such as signed contracts or regulated documents, keep the full history and record that decision.

Versions also matter for retrieval. A search index should see the current version of a document, not five drafts. Check how your target and any later AI tool handle version history before you decide what to carry over.

Retention

Retention rules, legal holds and records declarations must move with the content or be recreated in the target before content arrives. Involve whoever owns records management early. Deleting content under a hold, or migrating it to a location without the same protection, creates a compliance problem that no later fix will undo.

Office 365 data migration paths

An Office 365 data migration, now usually called a Microsoft 365 migration, follows one of a small number of paths. The right tool depends on the path, so identify yours first.

• SharePoint Server to SharePoint Online: Microsoft provides the SharePoint Migration Tool (SPMT), which it describes as a free tool for moving content from on-premises SharePoint sites.
• File shares to SharePoint Online and OneDrive: SPMT copies files from regular file shares, and Microsoft's Migration Manager in the SharePoint admin centre handles larger file share moves.
• Site to site within one tenant: content is restructured rather than relocated, often to split an overloaded site or merge several small ones.
• Tenant to tenant: usually after a merger, acquisition or divestment, where identity mapping is the hardest part.
• Other cloud storage to Microsoft 365: content moves from another file platform into SharePoint and OneDrive.

Choosing a migration tool

Microsoft publishes guidance on which of its tools suits each source. Third-party tools also exist, mostly for site-to-site and tenant-to-tenant moves. Choose by fit with your estate rather than by feature lists. The questions that matter are whether the tool supports your source, preserves the metadata and permissions you need, runs incremental passes so a final sync is short, and reports every file that failed, with a reason.

Planning the cutover

Move content in waves, grouped by team or site, so problems surface in a small group first. Run a bulk copy ahead of time, then an incremental pass just before switch-over to pick up recent changes. Agree a short freeze window with each team, tell people where their files now live and set the source to read-only once their wave is done. Read-only matters: it stops people saving new work in the old location and keeps the source available for checking.

Migrating with search and RAG in mind

Documents are unstructured data, and most organisations want to use them with AI sooner or later. Retrieval-augmented generation finds relevant passages in your documents and gives them to a language model to answer from. The answer can only be as good as the passage it retrieves. If the index holds a superseded policy next to the current one, the system may quote either.

Planning the target with retrieval in mind costs little during the migration and a great deal afterwards. These are the decisions we would make before the first wave.

• One authoritative copy of each document, with duplicates archived out of the searchable estate.
• A clear split between live content and archive, so archive locations can be excluded from the index.
• A status field, such as draft, approved or superseded, on documents that people rely on for answers.
• Text that can be read: scanned PDFs and images need text extraction before any retrieval system can use them.
• Descriptive titles and a named owner, so a retrieved passage can be traced and corrected.
• A permission model that you would be content for an AI tool to inherit, because it will.

These choices feed directly into RAG architecture: ingestion, chunking and retrieval all work better on a tidy, tagged, permissioned library. They also support data lineage, because each answer can be traced back to a known document with a known owner and status.

The evidence for this approach comes from systems that answer from a client's own material. Slimstock's custom AI chat answers from Slimstock's own content, providing 24/7 support in 11 languages. That result depends on the content being current and governed. A chat tool over an unsorted estate would not produce it.

Checking the result after cutover

A migration is finished when you can show that what arrived matches what left, not when the tool reports completion. Plan the checks before the first wave so that each wave is signed off the same way.

• Counts: compare file and item counts per library between source and target, and explain every difference.
• Failures: review the tool's error report and resolve or record each failed file.
• Sampling: open a sample of files of each type to confirm they are complete and readable.
• Metadata: confirm that authors, dates and key columns arrived populated and correct.
• Permissions: sign in as test users in different roles and confirm they see what they should and nothing more.
• Links: check that links inside documents and pages point to the new locations.
• Search: run real questions from each team against the new estate and check the results.

The same principle that applies to databases applies here: data reconciliation means comparing two systems and investigating every figure that does not agree. Keep the source read-only for an agreed period after cutover, then decommission it once the checks are complete and the owners have signed off. Plan support for the weeks after go-live, because that is when people find the folder that did not move.

Where this leaves you

A SharePoint or Office 365 migration will move your documents whatever you decide. The choice is whether it also moves the duplicates, broken permissions and untagged files that would undermine search and AI later. Inventory first, clean up with the owners, reset permissions, carry over the metadata and versions that matter, move in waves and check the result before you switch the source off.

If AI over your documents is on the roadmap, plan the target estate as its foundation. The extra work during the migration is small compared with fixing wrong answers once people have stopped trusting them.

Our data migration service moves content onto a governed foundation, reconciled and supported past cutover, and our RAG and vector search work builds retrieval on the estate that results. If you are planning a SharePoint move and want it to serve search or AI later, we will scope it honestly and say if we are not the right fit. Talk to us before the first wave, when the decisions are cheapest to change.

Frequently asked questions