The Bureau of Meteorology (BoM) is considering replacing its data extraction software system with an advance extract, transform and load (ETL) software system.
With the new system, the BoM aims to extract water-related data in a range of formats from an increasing number of sources, to produce spatial data. It is in line with the gradually expanding water information programme.
“The need to have consistent and efficient means by which to process and ingest this data obtained from a wide range of sources has grown substantially over the last few years and is expected to expand yet further over coming years,” agency documents stated.
The documents further mentioned, “To date the Bureau has implemented several bespoke applications to undertake these operations, whereas there is now a desire to move towards the use of non-bespoke approaches where it makes good economic sense to do so.”
According to agency documents, the system will enable developers to create processes to automatically collect data from a range of data sources and formats, perform specified operations and transformations on these data, and feed the data into a data warehouse.
The system will also have suitable management interfaces for processes, provide the means to deploy and execute these processes centrally, and operate on software platforms currently supported by the BoM. The agency anticipates the contract to be completed by June 30, 2012.
The bureau recently outlined plans to invest in a new integration framework to cope with the expanded number of sources producing spatial data. The solution will assist with the processing of geospatial and related non-geospatial to support business and system processes.
