Data Modernisation in Geological Survey of Queensland
Context and challenges:
The Geological Survey of Queensland (GSQ) has been the custodian of geological data, reports, scientific surveys and resource commodity information for over 150 years. During this time changes in legislation, commodity focus, technology, data formats, storage methods and databases meant that the information available to the resources industry was quite variable. The GSQ maintained several separate on-premises legacy systems to hold all their data, which required a number of fulltime staff to respond to data requests from industry. The costs and staff time for maintaining these systems and processes was high compared to the cost savings and efficiencies achievable through consolidation and modernisation of the system.
The Geoscience Data Modernisation Project (GDMP) solution:
In August 2020, the GSQ released a new modernised data ecosystem which had:
- condensed the seven systems to a single modern database, with a single common schema
- standardised data formats, vocabularies, templates, and requirements
- implemented an end-to-end, automated, cloud-based platform solution
- significantly reduced storage costs, manual data handling and time spent by staff
- constructed a lodgement portal to receive legislated report submissions from industry
- implemented an open data portal for industry to search for and download data of interest
- made 150+ years of geoscience data easily available for self-serve by industry
- significantly upskilled staff technical capability and digital literacy and enhanced data culture in the GSQ
- enabled ongoing feature developments for continual system improvement
The above data flow diagram shows the simplified process in which data is collected, stored and publishing within GSQ.
Since the launch of the GSQ Open Data Portal (ODP) and the GDMP solution, over 170 thousand public datasets, containing almost 1 million individual public files and totalling over 60 Terabytes of data, are available online and can be easily accessed by industry. This represents more than 90% of the data held by GSQ.
Data explorers, researchers and miners from 176 countries have visited the ODP, and more than 36 terabytes of data have been downloaded in the two years since the system launched to the public.
GSQ’s Open Data Portal is the single point of truth for all Queensland geoscience data. It aims to help unlock the full value of geoscience data to enable industry success.
Improving data discoverability and next steps:
The Queensland Government is committed to improving data discoverability through metadata harvesting and sharing links to GSQ open datasets on the government’s primary open data platform, data.qld.gov.au. At the time of writing, this work is well-advanced,
GSQ’s Open Data Portal has a simple user interface packed with powerful search functionality. Data is tagged by categories and can be filtered by type, dates, spatial coordinates, formats and more. Users can quickly drill down to find specific data of interest and download it all at once. The ODP is also accessible via API for easy automation. New datasets and functionality are being added all the time, so there is always something new to find.
For further information on the GSQ’s Open Data Portal system, contact the Geoscience Information team at GSQOpenData@resources.qld.gov.au .
You can visit the portal here - Welcome - GSQ Open Data Portal
The above picture is a field photo of metasedimentary rocks from the Mary Kathleen Region, North West Queensland.