This article was first published in Issue 124 (Apr/May 2023) of Position magazine.

NSW’s statewide Digital Elevation Model improves approximate AHD heights on public record.

By Jonathon Smith and Volker Janssen

The Survey Control Information Management System (SCIMS) is NSW’s database containing more than 250,000 survey marks on public record across the state. It was recently updated with Australian Height Datum (AHD) values sourced from a state-wide Digital Elevation Model (DEM) for more than 127,000 survey marks with existing Class U or null AHD heights. Almost 19,000 marks received an AHD height for the first time, while 100 gross AHD height errors in SCIMS were identified and corrected.

The updated AHD heights are displayed in SCIMS to the nearest metre and provide important benefits for industry, such as enabling the calculation and reporting through SCIMS of the Combined Scale Factor (CSF) and supporting datum modernisation to further improve user access to survey information.

DCS Spatial Services, a unit of the NSW Department of Customer Service (DCS), provides a state-wide DEM with a vertical uncertainty of ±0.9 m at the 95% confidence level (CL) and a horizontal grid density of 5 m. While it has been available through ICSM’s Elevation Information System (ELVIS) as 2 km by 2 km data tiles for some time, the DEM can now be queried directly through a publicly accessible Application Programming Interface (API).

This article outlines how the new interface was used to query the coordinates of all marks in SCIMS to retrieve AHD heights from the elevation model, assess the accuracy of these AHD heights and update SCIMS with DEM-sourced heights for suitable survey marks to yield a homogeneous dataset of known provenance and verifiable quality across NSW.

Background

As surveyors and spatial professionals, we know that height and elevation data is crucial for a vast number of applications and that our survey control underpins this data. DCS Spatial Services delivers various imagery and elevation products as part of its ongoing custodial responsibilities regarding the NSW Foundation Spatial Data Framework.

Accurate and reliable orthorectified aerial imagery and high-resolution elevation data are critical to effective planning, decision making, change monitoring and risk mitigation and are utilised by government, industry and the community. Reliable and quality-assured survey control is fundamental to ensuring the integrity of this data, which contributes significantly to economic, social and environmental sustainability in NSW.

One of the products provided by DCS Spatial Services is a state-wide DEM with a vertical uncertainty of ±0.9 m (95% CL) and a horizontal grid density of 5 m. It was produced by a combination of category 1 LiDAR, category 3 LiDAR, 10cm ground resolution imagery and 50cm ground resolution imagery.

This DEM can now be queried directly through a new publicly accessible API to return an AHD height at a specified location.

Automated AHD height retrieval

AHD heights from the state-wide DEM were extracted via an in-house-developed Python script using the ‘public/NSW_5M_Elevation’ API, available on the NSW Spatial Information Exchange (SIX) platform.

To obtain a sufficiently large dataset for evaluation of the accuracy of the returned data, every survey mark in SCIMS (including witness marks, destroyed marks and interstate marks along the borders) was submitted to the API for height retrieval. This included 301,200 marks, using their horizontal GDA2020 positions for interrogation.

Due to the enormous number of queries required, and to prevent overload of the server, the submission data was separated into sets of 50 marks for asynchronous retrieval, with a wait timer introduced between sets.

Retrieval was an iterative process, as failure rates for the server’s identify function were up to 50% of the submitted set at times (likely caused in part by inferior internet connections while working from home during the COVID-19 pandemic). Failed retrievals were re-added to the submission set for the next iteration and automatically resubmitted until completion. This process took approximately eight days of continuous processor time.

We found that 872 of the submitted positions returned no data, with 46 of these located on Lord Howe Island (not covered by this DEM nor true AHD). The remainder (apart from a few anomalies) were located along the Queensland, South Australian and Victorian borders.

Closer inspection revealed that all these locations were outside the extent of the DEM. Note that SCIMS includes several interstate survey marks close to the NSW border and that the Australian Capital Territory is entirely covered by the DEM.

Quality assessment of the DEM

The lower-accuracy AHD heights retrieved from the DEM were compared to existing high-accuracy AHD heights of survey marks in SCIMS that satisfied the following criteria:

• Established (Class D or better) horizontal GDA2020 coordinates.
• Accurate (Class B/LD or better) AHD height.
• Mark located at or near ground level.

The prerequisite for established horizontal coordinates is as important as an accurate AHD height because local terrain undulations can quickly alter the height returned from the DEM. Similarly, selecting marks that are not on the natural ground surface renders the comparison invalid.

In this instance, 34% of the survey marks in SCIMS (102,437 of 300,328 that returned heights) met the required criteria. For example, Figure 1 illustrates the distribution of these marks across Greater Sydney.

The threshold for a successful comparison was set at ±0.9 m, which is the quoted vertical uncertainty of the DEM (95% CL). We found that the calculated height differences between the DEM and SCIMS were within this threshold for 95,866 marks, i.e. 93.6% of the comparison set (indicated in blue in Figure 1).

This pass rate is slightly lower than the quoted vertical DEM uncertainty, for two main reasons:

• The comparison did not consider the survey marks’ vertical position above or below ground level (typically up to 0.2 m in either direction). This was deemed unnecessary as the result was fit for purpose and this information was only available for 3.7% of the comparison set.
• The horizontal density of the DEM is 5 m, so AHD heights at marks located on undulating terrain may show some discrepancy, depending on their position relative to the DEM sample points (used to interpolate the height at the specified position).

For at least the last five years, DCS Spatial Services has recorded the mark-to-ground-level offset at each survey mark occupied or inspected as part of normal field operations. To examine the effect of including such metadata, we repeated the analysis for a subset of 3,849 marks.

This smaller sample exhibited a pass rate of 97.1% when the mark-to-ground-level correction was applied and a 96.2% pass rate when it was ignored. Together with the earlier analysis, this result was deemed fit for purpose, confirming the stated DEM uncertainty.

Interestingly, 960 (14.6%) of the 6,571 marks that failed to meet the ±0.9 m threshold (indicated in orange in Figure 1) are located within 20 m of the centreline of a major highway or motorway. This can be explained by rapid changes in topography often occurring along the cross section of the road corridor, including embankments and cuttings.

Furthermore, some of these roads are extremely steep, such as the Great Western Highway between Penrith and Glenbrook. Finally, 1,082 (16.5%) of all the marks failing to meet the threshold are specified as ‘destroyed’ in SCIMS, indicating that their AHD height may relate to a time prior to road or other construction earthworks altering the topography.

Comparison to independent DEM

Following initial height retrieval, we noted that the DEM returned an AHD height that was significantly different (> 20 m, e.g. a typical contour) from the value in SCIMS for 787 survey marks with an existing Class U AHD height.

This was investigated by querying Geoscience Australia’s national 1-second Shuttle Radar Topography Mission (SRTM) DEM for the entire dataset via another API. We then compared the two DEMs to each other and to SCIMS. Wherever the NSW DEM value differed from SCIMS by more than 10 m (in 4,690 cases), the 3-way comparison was recorded.

Any AHD height difference exceeding 20 m between the two DEMs was manually investigated, resulting in 45 of 108 marks (41.7%) being excluded from the following SCIMS update. All these excluded marks were located where an open pit mine had subsequently been created.

Any other large differences between the two DEMs were a result of the coarser resolution of the SRTM DEM (1 arcsecond equates to approximately 30 m). For example, for trigonometric station (TS) pillars located on the side of a cliff, the SRTM DEM sometimes returned the height partway down the cliff.

Most remaining large differences between the NSW DEM and SCIMS appeared to be the result of transcription errors in the SCIMS height (e.g. 1,000 m instead of 100 m) or rounding to the nearest contour when the heights were initially entered into SCIMS.

Updating SCIMS with DEM-sourced AHD heights

Reliable and quality-assured survey control is fundamental to ensure the integrity of the imagery and elevation products delivered by DCS Spatial Services. However, this connection can work both ways as these products can then be used to improve survey control information on public record.

In this case, we updated SCIMS with suitable DEM-sourced AHD heights (at Class U) to yield a homogeneous dataset of known provenance and verifiable quality across NSW. This essentially improved approximate AHD height values that were initially obtained from the nearest contour on 1:25,000, 1:50,000 and 1:100,000 topographic maps to DEM-sourced values with sub-metre uncertainty.

Noting the DEM uncertainty, survey marks were only selected for AHD height update if their existing AHD height in SCIMS was null or Class U. Furthermore, it is important to consider that SCIMS holds records for a wide range of different monument types.

A location descriptor also indicates whether the mark was placed in the ground or on a structure. As such, further filtering was applied to limit the height update to only those marks that are likely to be at (or near) ground level.

Consequently, several monument types (e.g. chimney, mast, reservoir, tower) and mark location descriptions (i.e. building or structure, silo, other structure) were excluded from the update. Any TS whose name included the word ‘tower’ was also excluded.

Note that ‘reservoir or tank’ should also have been on the list of excluded mark locations but was unfortunately missed. This resulted in 30 trig stations on reservoirs incorrectly receiving a height at ground level, which has since been revisited and corrected.

After filtering and before the SCIMS update was executed, a final test was performed to check for trends. This revealed that the data was normally distributed and 99.3% of the AHD heights included in the update were within 20 m of their existing SCIMS values (see Figure 2).

A 20 m error in height corresponds to approximately a 3 parts per million (ppm) error in the reduction of ground distances to the ellipsoid between two marks, which was deemed acceptable and fit for purpose.

As previously mentioned, height differences larger than 20 m were generally attributed to:

• Transcription or rounding errors in the existing SCIMS value (corrected by the update).
• Marks located where an open pit mine had subsequently been created (excluded from the update with mark status updated as ‘destroyed’).

SCIMS update results

This SCIMS update delivered DEM-sourced AHD heights at Class U for 127,154 survey marks, of which 18,854 marks (14.8%) received an AHD height for the first time (see Figure 3).

Putting this large number into perspective, 42.3% of the survey marks that returned a DEM-sourced AHD height during the initial retrieval were updated. This is a huge improvement in the access to reliable, approximate AHD heights of known quality in SCIMS, ensuring that nearly every survey mark in NSW has an AHD value of 0.9 m uncertainty or better.

When inspecting Figure 3, it is worth noting the near-perfect straight line of AHD height updates to survey marks running from north-west to south-east through the centre of the state. Despite appearing to be an artefact, this is a series of marks located along a gas pipeline easement, which connects to the main distribution network on the east coast.

Benefits

These updated AHD heights provide several important benefits such as enabling better calculation and reporting of the Combined Scale Factor (CSF) with confidence at virtually all survey marks (99.98%) in NSW through SCIMS. CSFs are now typically up to 1.5 ppm better because heights have been improved from 10-metre to sub-metre accuracy.

The DEM-sourced AHD heights support our datum modernisation efforts through the ongoing readjustment of legacy terrestrial data for inclusion in the growing GDA2020 state adjustment by facilitating the rigorous reduction of terrestrially measured distances to the ellipsoid. This translates into more SCIMS marks being assigned a Positional Uncertainty (PU), directly benefitting the profession.

Furthermore, this process identified and corrected 101 extremely large AHD height errors (Class U) in SCIMS. It follows that retrieved DEM-sourced heights can now also be used to identify gross errors on SCIMS marks with existing accurate AHD heights. This further contributes to our ‘Saving AHD’ efforts, which aim to ensure that users have continued and easy access to reliable physical heights and their uncertainties across NSW (see Position 115, Oct/Nov 2021).

Finally, with SCIMS now holding AHD heights of known quality state-wide, ellipsoidal height was derived by applying AUSGeoid2020 at all applicable survey marks with existing null ellipsoidal height values.

Another SCIMS update then provided GDA2020 ellipsoidal heights for 267,581 survey marks for the first time, ensuring that practically all SCIMS marks now also have an ellipsoidal height. Publishing these values allows surveyors and other users to easily verify that they have set their height datum and/or applied AUSGeoid2020 correctly during both field operations and office processing and reductions.

Jonathon Smith and Dr Volker Janssen work at DCS Spatial Services, a unit of the NSW Department of Customer Service.