An Icelandic retail group brought us into a half-built data lake mid-development. Six months later it was complete, consolidating three business units — with CI/CD on every pipeline, because a platform without deployment discipline is a demo.
The client is a holding company whose group spans retail grocery, e-commerce and pharmacy — three businesses with different margins, different regulations, and the same customers walking between them. The group had started building a data lake to consolidate sales, customer-interaction and inventory data across the units, and the build was already in flight when we arrived.
The engagement was to enhance and complete it: six months, four FTEs, joining mid-development. That framing matters — this was not a greenfield project where you pick the architecture, but a working site where you inherit the decisions already poured into the foundations and finish the building anyway.
The completed platform consolidates sales, customer-interaction and inventory data from all three business units into one lake: Azure Data Lake Storage for the data itself, Azure Data Factory for ingestion from the units' source systems, and Microsoft Purview for governance — so the group knows not just what data it has, but where it came from and who may use it. Azure Databricks sits on top as the processing, analytics and machine-learning layer, turning the consolidated data into something the businesses can actually query and model against.
The second half of the work was less visible and at least as important: introducing CI/CD and a DevOps practice. When we joined, changes reached the platform the way they usually do on projects under deadline pressure — manually, and with fingers crossed. We put Azure DevOps pipelines around every data pipeline, so that deployments became repeatable, reviewable and reversible. Launch day is the easiest day; the next ten years are the product, and CI/CD is what makes the ten years affordable.
Half the platform existed before we did — the first weeks were archaeology: understanding what was built, what was intended, and where the two diverged.
Some early design choices we'd have made differently; rebuilding them all would have burned the budget. Choosing which to keep was the real engineering.
Each unit describes a sale, a customer and a stock item its own way — consolidation meant reconciling three views of the same shopper.
CI/CD lands on people, not just pipelines — the team had to adopt reviews and releases while still shipping to the deadline.
The group now sees its sales, customer interactions and inventory across grocery, e-commerce and pharmacy in one governed place — instead of three unit-level versions that never quite reconciled. With Databricks as the analytics and ML layer, questions that used to span three teams and three exports can be answered against a single consolidated platform.
Just as durable is what the CI/CD work changed about the platform's economics: every pipeline now ships through an automated, reviewable release process, which means the lake can keep evolving after the consultants leave without each change being an act of courage. The project finished in six months from our joining — on a build that was already in motion, which is usually where timelines go to die.
Processing, analytics and machine learning on the consolidated data.
The lake itself — sales, customer-interaction and inventory data across the units.
Brings data in from each business unit's source systems.
Catalogue and lineage — what the group has, and where it came from.
Pipelines around the pipelines: automated, reviewable releases for every change.