Catalog Migration to Commercetools
A catalog migration from legacy CMS systems onto Commercetools: ~1K SKUs across ~5 markets, delivered in-house by two engineers in about two sprints, displacing a multi-million-dollar external contractor proposal. When I left, it was still the catalog data-integrity backbone.
Results
- Displaced a multi-million-dollar external contractor proposal by building in-house with two engineers in ~2 sprints, on a stack the team already operated.
- Migrated ~1K SKUs across ~5 markets with per-market verification gating each step.
- Unblocked the broader move to the modern commerce platform, which could not proceed on untrustworthy catalog data.
- When I left, it was still running as the catalog data-integrity backbone, instead of being retired as one-off migration tooling.
Context
Catalog data lived in legacy CMS systems that were inconsistent across regions and channels. The commerce platform migration depended on moving this data into Commercetools with clear contracts and validation. Nothing else could proceed until the catalog was trustworthy.
We needed a data model on Commercetools that matched real product structure, a migration service that could load it reliably, and enough validation to make the cutover safe across regions and channels. The immediate question, though, was who would build it.
What I did
- Defined the target data model on Commercetools and the API contracts the storefront and platform would rely on.
- Built a catalog migration service using jOOQ, Netflix DGS, and the Commercetools SDK to transform legacy CMS data and load it into the new platform.
- Built validation and verification checks that gated each region/channel migration step.
- Worked with product and localization teams to handle regional pricing, availability, and content differences.
Detail
Build or buy
A contractor had proposed a multi-million-dollar build for this work. Their proposal was an Airflow plus Java/Maven build: competent and conventional, on a stack our organization did not otherwise operate.
The quote was priced as delivery capacity. What it would have bought was a second platform to operate indefinitely: another deployment pipeline, another PagerDuty rotation, another set of failures nobody on the team had debugged before, for a system sitting on the critical path of every catalog change.
The tradeoff. Outsourcing buys throughput now and pays for it in systems to operate later. Building in-house costs senior engineering time up front and returns a system the team can already debug on call. The pipeline was going to be permanent infrastructure, not a one-off load, so we built it. On the existing Kotlin/Spring stack it was also the lower-risk choice.
Two engineers shipped it in about two sprints: jOOQ extraction from the legacy CMS, transformation, then load through the Commercetools SDK. When I left, it was still running as the catalog data-integrity backbone.
Verification
Read from one system, write to another, count the rows: that framing is how catalogs end up subtly wrong in production. A price that lost its currency, an availability flag that meant something different in the source system, a market that inherited another market's copy.
So every field mapping was stated explicitly, and each region and channel step was gated on verification. Migration ran per market, and a market that failed its checks did not advance.
- Extract from the legacy CMS with jOOQ, against an explicit projection instead of a full-table dump.
- Transform into the target Commercetools model, with every field mapping stated and unmapped source fields raised as an error instead of ignored.
- Verify the transformed set: counts, required fields, currency and locale coverage, referential integrity of bundle components.
- Load through the Commercetools SDK, then re-verify against what the platform actually stored.
- Gate: a market advances only if its checks pass. Otherwise it stops and nothing downstream sees partial data.
Re-verification after load matters because writing through an SDK confirms only that the call succeeded. Defaults get applied, optional fields get dropped, and a product can be created successfully and still be wrong.
Modeling bundles
Commercetools has no native bundle product, and a reference identifies a product without storing the quantity a bundle needs. A bike plus mat plus weights sells as one purchasable unit at one price, and that structure has nowhere to live in the default model.
The two options were to flatten each bundle into a standalone product, or to model bundles as a custom type carrying their components. Flattening loads more simply and loses the relationship to the component SKUs, which leaves inventory, pricing, and downstream fulfillment unable to tell what a bundle contains. I modeled it as a custom type.
{
"key": "product-bundle",
"fieldDefinitions": [
{
"name": "components",
"type": {
"name": "Set",
"elementType": { "name": "Reference", "referenceTypeId": "product" }
},
"required": true
},
{
"name": "componentQuantities",
"type": { "name": "String" },
"required": true
}
]
}Carrying quantities beside the references is a compromise, because the platform cannot enforce that the two stay in step. Validation has to, which is why referential integrity of bundle components is one of the verification checks.
Tech: Java, Kotlin, jOOQ, Netflix DGS, Commercetools SDK, PostgreSQL, GraphQL