Taking geoscientific data from core tray to cloud, at scale

A mining leader investing in faster discovery

Our customer is one of the world's leading metals and mining companies, focused on finding, mining and processing the Earth's mineral resources. Behind every new discovery sits a mountain of geoscientific data, and with faster and more efficient ways to explore, its exploration division is generating volumes more data every year.

That data comes in many forms such as spectral readings to high-resolution drill core imagery, and from many sources including field devices, aircraft, third-party APIs, databases, and internal applications running both on-premise and in the cloud.

The organisation launched a program of work to ingest and provision this geoscientific data for its exploration teams. They engaged Mechanical Rock to help put trusted and well-organised data in front of geoscientists fast enough to shape their decisions in the field.

A workflow tool that outgrew its foundations

The exploration team relied on an in-house application to manage its core scanning workflows. Drill core is scanned with a laser-based scanner that detects lithology, mineral composition and spectral properties, and the results are processed into data that geoscientists use to understand what lies underground.

Core scanning is a manual process with plenty of room for human error, whether that’s a tray labelled wrong, data uploaded against the wrong record, or a scan swath which has captured a bit of the plastic tray along with the core. Broken core also needs core loss marked, contaminants such as wood have to be excluded and project settings were sometimes misconfigured at site level.

The application was initially set up in Excel and SQL where it queried its own PostgreSQL database separate from the organisation's Databricks platform. As volumes grew, it began to slow creating downstream delays and errors.

Four priorities anchored this piece of work:

1. Replace the legacy application with apps hosted natively in Databricks
2. Improve on the data correction workflows, rather than replicate them like-for-like
3. Automatically reprocess data once it has been corrected, so downstream data products stayed accurate
4. Give geoscientists new ways to explore core data, from colour-coded composition to AI-assisted answers

Building a workflow tool to keep up with the job

Mechanical Rock embedded with the exploration data team, working within the environmental and security constraints set by the organisation's IT and platform teams, and bringing those teams along for the journey.

The Mechanical Rock team then automated core imagery ingestion, built a governed catalogue in Databricks, introduced CI/CD and Infrastructure-as-Code, and produced a migration design for the core scanning platform that became the blueprint for the rebuild.

One home for data, apps and AI

The core scanner vendor now stages its outputs including images, tabular data, point clouds, RGB photos of the core and spectral data directly into the mining company’s S3 buckets. From there, data moves through a full medallion architecture in Databricks, from bronze to silver to gold, by the time it has been captured, processed and ingested.

Rather than sending users out to a separate application and database, Mechanical Rock rebuilt the workflow tool as Databricks Apps, hosted right next to the data so apps, data, jobs and governance now sit in one place and are integrated with the wider company ecosystem.

The platform also now brings in data from the team's AI-driven core imaging technology, and curated outputs are exported to 3D geological modelling tools for drillhole visualisation. Together, they support the whole value chain, from core scanning through to modelling.

Fixing data at the source

Data correction sits at the heart of the new Databricks app, designed to catch upstream human errors before they reach a geoscientist’s screen.

Users can correct scan swaths directly: trimming out tray material captured in a scan, demarcating core loss where core is broken, and excluding contaminants such as wood that shouldn't be in the swath. Mislabelled trays and data uploaded against the wrong record can be fixed in the same place, and project misconfigurations made at site level can be corrected without a workaround.

Saving a correction triggers Databricks jobs that reprocess the affected data and regenerate image data from the scanner vendor's platform. Geoscientists can then get corrected data without manual handoffs across disconnected systems.

From core tray to confident decisions

For an exploration team, data quality is the difference between second-guessing a scan and trusting what they see while decisions are still being made. By moving its core scanning workflows into Databricks Apps, the company now has a platform where errors are fixed at the source and where the new apps can do far more than the old tool ever could. Core composition can be visualised and colour-coded, so patterns in lithology and mineralogy stand out at a glance rather than hiding in rows of numbers. And AI capabilities go a step further, letting users query the data in plain language.

Most importantly, geoscientists, subject matter experts, technical contributors and data engineers now all work from the same trusted data, on a platform that scales with the next generation of core analysis technology and puts better answers in front of the people searching for the next deposit.

How can Mechanical Rock help with your data platform?

At Mechanical Rock, we help companies deliver cloud-native solutions, leverage data platforms and unlock AI so systems can run securely, reliably and at scale.

Whether you're looking to set up a data platform or get more out of your existing one, we can help.

Let's have a chat today >