Moving closer to practical automation of the Scan-to-BIM process.

By Jidnyasa Patil and Mohsen Kalantari

Creating accurate building information models (BIM) from point cloud data of existing buildings remains a slow, manual and expensive process. This process, known as Scan-to-BIM, is an important challenge for the architecture, engineering, construction and surveying industries.

In this article, we discuss how our research at the Digital Twin and Land Tenure Lab at UNSW Sydney combines artificial intelligence with geometric processing to automate Scan-to-BIM.

We focus on two questions:

• Does better AI-based semantic segmentation always produce better 3D reconstruction?
• And does the answer remain the same for different building elements such as walls, doors and windows?

Our results show that high segmentation accuracy is useful, but it is not enough. For a reliable BIM, the geometry of building elements must also be preserved and reconstructed accurately.

What is Scan-to-BIM?

Laser scanners and photogrammetric systems can capture existing buildings as dense 3D point clouds. However, a point cloud is not yet a BIM.

A BIM contains structure objects and information. It identifies a wall as a wall, records its dimensions and position, and defines how it relates to floors, doors, and windows. This information can support building maintenance, renovation, accessibility assessments and energy modelling.

Scan-to-BIM generally has four stages: pre-processing, semantic segmentation, extracting measurements and 3D reconstruction. Semantic segmentation assigns a category such as wall, floor, ceiling, door or window to each point. Extracting measurements involves separating instances for each of the categories and their spatial relationships to reconstruct building elements.

Deep learning has significantly improved point cloud segmentation. However, a critical question remains: does a model that classifies more points correctly also create a more accurate BIM?

How did we investigate this question?

We compared five deep learning models: Swin3D, Point Transformer V3, Point Transformer V1, PointNeXt and PointMetaBase. We trained and tested them using a modified version of Stanford Large-Scale Indoor Spaces dataset. Furniture categories were grouped as clutter so the models could focus on key architectural classes.

After segmentation, we used different geometric methods for different elements. For walls, we used RANSAC to detect planar surfaces. For doors and windows, we used DBSCAN to group segmented points into instances, followed by bounding boxes to extract position, width and height. These measurements provide the parameters required for BIM objects. The workflows use open-source tools, including Open3D and IfcOpenShell.

What are our findings for walls?

Swin3D produced the best wall segmentation result, with a wall Intersection-over-Union score of 88.95%. It also produced the most accurate reconstructed walls, with an average deviation of 7.3 centimetres from the reference point cloud.

However, PointNeXt produced the lowest wall segmentation score among the five models, at 82.25%, but achieved a joint second-best reconstruction result, with an average deviation of 7.7 centimetres.

PointNeXt did not classify every wall point correctly. However, it preserved enough wall geometry for RANSAC to detect the correct plane. The reconstruction algorithm did not need every point to be correct. It needed enough correctly located points to identify the main structure.

This shows that better identification does not automatically guarantee better reconstruction. Semantic segmentation focuses on assigning the correct label to each point, while reconstruction focuses on recovering the overall shape, dimensions and position.

What happened with doors and windows?

Our second study extended this investigation to architectural openings. Doors and windows are more difficult than walls because they are smaller, less common and often affected by occlusion. Windows are especially challenging because glass may be transparent or reflective, and windows are often co-planar with surrounding walls.

For doors, Swin3D achieved the highest detection rate at 96.9%, followed by PointMetaBase at 92.9%. For windows, the best detection rate was 75%, achieved by both Swin3D and Point Transformer V3.

Across all five models, the same pattern was clear. Walls were the easiest elements to segment, doors were more difficult, and windows were the most difficult.

The geometric results again showed that segmentation and reconstruction rankings were not always the same. Swin3D produced the strongest door geometry, while Point Transformer V3 achieved the highest 3D overlap for windows. PointMetaBase also produced strong door geometry despite not having the highest overall segmentation score.

We then applied the models to 11 rooms from the Matterport3D dataset without fine-tuning. These residential environments differed from the training spaces, so performance decreased. Even so, incomplete segmentation could still produce useful geometry when the overall shape of an opening was preserved.

Why does this matter?

Much of the research on automatic Scan-to-BIM has focused on improving semantic segmentation and point-level accuracy. These developments are important, but segmentation only identifies the probable class of each point. It does not produce a complete building model.

Reconstruction converts segmented data into usable building information by determining the geometry, placement and relationships of individual elements. Our results show that a model with a lower segmentation score can still support accurate reconstruction if it preserves the main geometric structure.

Segmentation accuracy alone should therefore not be used to judge a Scan-to-BIM workflow. The final model must also be assessed for geometric accuracy, completeness and validity. Research should place greater emphasis on reconstruction algorithms and on the quality of the resulting BIM, rather than focusing mainly on small improvements in point-level classification.

What comes next with Scan-to-BIM?

The next challenge is to develop reconstruction methods that can make reliable use of imperfect predictions. They must handle missing points, occlusions and variations in point density while still extracting accurate measurements.

They must also reconstruct the building as a connected system. Walls need to intersect and connect correctly, while doors and windows need to be associated with their host walls. The resulting elements must follow the spatial and topological rules required for a valid BIM.

Future studies should test complete workflows on a wider range of buildings and scanning conditions, and extend them to floors, ceilings, columns and other elements.

The aim is not to reduce the importance of semantic segmentation, but to balance it with stronger reconstruction methods. By treating segmentation as an input to reconstruction, rather than the final outcome, we can move closer to practical automation of Scan-to-BIM.

Jidnyasa Patil is a PhD candidate in Surveying and Geospatial Engineering at UNSW Sydney. Her research focuses on developing novel methods that integrate deep learning and geometric algorithms to automate the Scan-to-BIM process for existing buildings.

Mohsen Kalantari is an Associate Professor of Geospatial Engineering and Deputy Head (Education) at the School of Civil and Environmental Engineering, UNSW. He also is a Co-Founder of a tech start-up on Scan-to-BIM called Faramoon. Over the last 22 years, he has undertaken research on cadastres in collaboration with land registration and mapping agencies, municipalities and surveying companies across Australia and overseas.

Further reading:

For more details on the methods, experiments and results discussed in this article, please refer to the authors’ research papers. The first paper examines the relationship between semantic segmentation accuracy and wall reconstruction. The second extends the investigation to the detection and geometric reconstruction of doors and windows.

Jidnyasa Patil and Mohsen Kalantari, “Automatic Scan-to-BIM: The Impact of Semantic Segmentation Accuracy,” Buildings, 2025.

Jidnyasa Patil, Arcot Sowmya and Mohsen Kalantari, “Automatic Scan-to-BIM: The Impact of Semantic Segmentation Accuracy on Opening Detection,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2026.