This article was written by Michael Giudici, the Surveyor-General of Tasmania and chair of the GDA2020 Implementation Working Group (GMIWG). It was published in issue 101 of Position magazine. Stay tuned for an update on GDA2020 in the upcoming issue of Position.
As the GDA2020 implementation has been progressing and technical transformation parameters developed and released, several legacy issues pertinent to the web mapping environment have emerged.
The capacity and understanding by spatial data users to transform discrete data sets between datums has been increasing over the last two years, particularly as that relates to the desk top environment, and software houses have responded with upgrades and help notes to assist with such transformations.
However, a particular problem arises with transformations undertaken in the web mapping arena, , one that demands a concerted international effort to overcome. This article describes that problem and the steps that ICSM and others are taking to resolve it.
The dilemma’s history
The so-called ‘Web Mercator Dilemma’ is the result of activities and decisions that date back to the late 1980s. At that time, the US Defence Mapping Agency identified the World Geodetic System 1984 (WGS84) as the single earth-centred geodetic system for use worldwide, for all Department of Defence mapping, charting, navigating and geodetic activities.
With the advent of web mapping, WGS84 became the default, or ‘hub’ or ‘pivot’ datum for these applications. This means that transformation between datums often included transforming first to WGS84 and then to the target datum. This reduced the number of datum to datum parameters that had to be stored, but also resulted in the quality of transformations being limited by the quality of the transformation to WGS84. WGS84 is often described in Web Mercator, a variant of the Mercator Projection, and became the de facto standard for web mapping when it rose to prominence following Google Maps adopting it in 2005. It is used by virtually all major online map providers such as Google, Bing Maps, OpenStreetMap, Mapquest and ESRI.
So this ‘hub’ datum has worked pretty well considering what it was designed for, up to now. However with the introduction of high precision, commodity positioning systems, and the massively increased uptake of web mapping applications, changes are now required.
In 1994 when GDA94 was established, WGS84 could be considered equivalent to GDA94. However, like GDA2020, GDA94 is a plate fixed datum, so it is fixed to the Australasian tectonic plate, and moves with that plate. WGS84 on the other hand, is an earth-centred reference frame and therefore changes over time in relation to plate fixed datums. So, the name ‘WGS84’ provides only part of the picture: the full and accurate description is ‘WGS84 (G1762)@2019.5’. In other words, the full definition string accounts for ‘G1762’, the GPS week when that variant of WGS84 was defined, and the time-stamp for the coordinates themselves..
The dilemma thickens
The next part of the story concerns the European Petroleum Survey Group, or EPSG, which was set up in 1986 at about the time the Department of Defence adopted the WGS84 approach. The EPSG defined and disseminated the EPSG geodetic parameter set which is a widely-used database of earth ellipsoids, geodetic datums, geographic and projected coordinate systems, units of measurement, aetc. The functions of this group became part of the Geodesy Subcommittee of the International Association of Oil and Gas Producers (IOGP). The Geodesy subcommittee of the IOGP consists of specialists working in applied geodesy, surveying and cartography, and kept the name of the EPSG Registry to avoid confusion. The various parameters are assigned ‘codes’ or ‘identifiers’, and these are what are drawn upon by the software houses for their web mapping applications.
Since, as mentioned above, in 1994 ‘WGS84’ and, GDA94 were nominally equivalent, the EPSG transformation parameters between WGS84 and GDA94 was made Null. It still is. This has not been a major issue in the intervening years due to the purposes for which this system was however over time of course the plate fixed datum and the earth centred reference frame have diverged.
So, whilst software houses can correctly introduce the transformation parameters between the new GDA2020 and GDA94 for their desktop applications, once data is re-projected to a web environment, due to the Null parameter issue, data transformed from GDA2020 will not align with data transformed from GDA94. This has the potential to cause significant problems with display and analysis of datasets.
Organising for a solution
ICSM has been working with the EPSG compilers to introduce ‘work arounds’ and additional metadata to alert users to this issue, however a more definitive solution that provides for rigorous transformations in the web mapping environment is now being pursued.
Since this issue is an international one, discussions about the best solution have been occurring in a number of countries. Through the combined efforts of a group of European based specialists in spatial data management and development who work in the open source environment, the necessity of drastic improvements to the handling of coordinate systems in the web and open source environments was realized, and a ‘Barnraising’ effort was organised to raise funds for the required works in May 2018. Twenty organisations responded to contribute $144,000 towards this work program, which exceeded the target funding of $125,000. Among the organisations contributing are ESRI ($30,000), Land Information New Zealand ($7000) and Safe Software ($31,000) which demonstrates the international nature of the challenge, and the desire for a single robust solution and not a plethora of bespoke solutions separately developed by individual organisations. For a detailed discussion of the ‘Barnraising’ parameters, google ‘GDAL Coordinate System Barn Raising’.
In essence, the Barnraising project is aimed at bypassing the need to use WGS84 as a ‘hub’ or ‘pivot’ datum, and draws upon the new capabilities of the PROJ5 library for performing conversions between cartographic projections.
ICSM has been monitoring progress on this work and in early 2019 commissioned an Australian company, North Road Pty Ltd, to undertake a program of work aimed specifically at operationalising the international effort for the Oceania region. ICSM has also engaged Mercury Project Solutions to contribute to the work packages and project manage the program on ICSM’s behalf. The work packages will concentrate on improving the handling of all the transformation elements of QGIS, an open source GIS package, so that the benefits may flow to all software developers via the open source platforms. The work packages can be summarised as follows:
- Engage and influence the maintainers of the OSgeo4W and QGIS installers to ensure users will automatically have the new transformation grids included in installs. Secondly to liaise with the GDAL Barnraising team to address Oceania needs with respect to EPSG transformation data bases, and change the QGIS user interface to warn users when grids are not available and present steps for remediation. This was successfully completed in March 2019.
- Upgrade all QGIS code paths to completely respect the transformation context, including the datum transformation settings set by the user (and in the future the temporal components of GDA2020 transformations). The GDA2020 transformation is the first global use case that needs this functionality and may therefore contribute to global fixes. Place code blocks to prevent future code changes that could ‘undo’ the correctly handled transformations, such that any future features added to QGIS will work correctly with GDA2020 regardless of who develops them. This phase was completed in April 2019.
- Upgrade QGIS coordinate transformations to utilise the new PROJ5 API with a graceful fallback to the version 4 API wherever version 5 is not yet available. This will result in more accurate transformations avoiding the WGS84 ’Pivot’ and will be ready to support future time dependent coordinates and transformations. Also, upgrade to PROG6 support from February 2019,with graceful fallbacks to earlier versions. This is underway as of May 2019.
- Research into desktop GIS application handling of 4D coordinates and coordinate transformations and into the changes required to expose these to users. Who is doing what, looking at use cases, leading to a better understanding of where ICSM may target future investments towards standards and policy. This will include a stocktake of current and expected future approaches, software and standards, and recommended options for ICSM. A joint North Road and Mercury Project Solutions proposal has been submitted to the ICSM in May 2019.
- Document lessons learned, best practice and challenges for (Oceania) user and developer communities, specifically in the use of PROJ and GDAL. Share experiences with the community, produce a white paper or blogpost for the ICSM website and promote to the Oceania developer community through appropriate forums. This final stage ispost project, scheduled for mid to late 2019.
Readers who may be unfamiliar with some of the terminology used in this article will find a large amount of content and discussion on GitHub and the Barnraising websites. ICSM has produced a number of good resources for understanding the GDA2020 Datum and continues to develop fact sheets and communication products.
If there is one key take home message from this discussion, it is the phrase that ICSM has been emphasising since the early days of the datum modernisation project, and that is, ‘Know your data, know your datum!’
Stay up to date by getting stories like this delivered to your mailbox.
Sign up to receive our free weekly Spatial Source newsletter.