Since the launch of OpenStreetMap (OSM), the quality of its data has been a prominent study subject. Among the various aspects of data quality, data completeness – i.e., whether all real-world geographic features in a given extent exist in the dataset – remains a central element and a prominent challenge [1]. Existing approaches to assessing the completeness of data in OSM can be broadly grouped into three main groups: extrinsic comparison against a reference dataset [2-5]; intrinsic analysis of some proxy variable in the data [6-8]; and statistical methods that include curve fitting and Deep Learning algorithms (DL) [9-12].
Extrinsic analysis, though the best estimator, requires an authoritative dataset, thus limiting this approach’s applicability in regions where official or proprietary data are unavailable, outdated, or costly. Intrinsic methods compare OSM's edit history (e.g. edit frequency, contributor counts) with human development proxies (such as night lights or GDP) and allow for better scalability and flexibility between regions but tend to reflect local activity more than mapping outcomes. Statistical modeling approaches, such as fitting Sigmoidal curves to cumulative road lengths [10] and applying DL to detect missing features [11], require good training data and may not translate well over regions of different characteristics.
This study presents a novel intrinsic completeness metric based on the evolution of mean feature geometry, that requires only aggregate feature statistics and no external source. The method uses cumulative time series data, but differs from prior studies, e.g. [7, 11], by focusing on the average relative change in the cumulative value, instead of the absolute value, accounting for both feature additions and geometry alterations. The underlaying assumption draws on the works by Corcoran et al. [13] and Strano et al. [14], showing that road networks are typically mapped in a process in which exploration is inherited by densification. In other words, contributors initially add the largest, most prominent features (major roads, large and central buildings) and gradually move on to smaller and more detailed elements (minor roads, small structures). Thus, we can expect the average size (area\length) per added feature to decrease over time, until it converges to some general threshold such that the data can be considered saturated (or "complete enough" for usage). Simply put, we expect to see the temporal signature of the average feature size per new addition (denoted aa_t) to stabilize over time. For comparative analysis between regions, the metric is normalized to the largest average addition in each time series (denoted naa_t).
A complete region can be identified using this method if the series reaches a defined stable saturation period, meaning naa_t<α for at least β years, α and β being pre-defined thresholds. This creates a robust method that integrates both a standard adjustable threshold that accounts for different mapping dynamics and enables a completeness metric of different mapped elements via their geometric attributes. To make the method more robust, we define two subsequent calibration tests in relation to the saturation period: testing for an external disruption that may have altered the signal periodically (e.g. a data export or HOT mapping party) and bounding the relative growth, i.e. the relative addition in the saturation period, against another threshold γ. These tests are defined to detect false positives.
As an initial exploratory assessment of its behavior, we implement the method for a sample of 446 regions categorized by distinct mapping dynamics [15], allowing us to test the method’s sensitivity to large scale events. Aggregated mapping pattern statistics, i.e. total number and cumulative length of roads (way features, highway= tag) and total number and cumulative area of buildings (way features, building=) per month, were extracted using the Ohsome API [16]. Our implementation sets α=0.1,β=2 and γ=1.5 as threshold values, meaning all the relative additions were less than 10% from the highest addition for a period of 2 years or longer, bound to less than 50% relative addition in said period.
Initial results are promising though somewhat restrictive, with 17% of all regions labeled as complete for building features and 8% for roads, meaning it can be acute to different feature types' mapping trend. These results were analyzed through two validation tests – analyzing completeness in relation to feature density and analyzing a randomly selected set of locations - and a statistical analysis of the sampled region's attributes. Test 1 found that both complete and incomplete regions displayed similar density distributions, suggesting robustness to both rural and urban areas. Test 2 showed that even for a randomly selected group of regions and cities from around the world, the sensitivity to different feature types holds. The statistical analysis results show that some complete regions had a very long stability period despite having a consistent relative addition. However, in both the original sampled data and the random control group in test 2 a dominant reason for incompleteness was the unbounded relative addition, meaning saturation periods were long enough to typically result in a consistent addition that fails the calibration test, perhaps due to an ill representation of the data (see figure A). This marks the method restrictive, albeit robust. Finally, analyzing completeness frequency by mapping dynamics and spatial extents showed both distributions to maintain a relatively uniform shape, implying the method robust to both attributes (see figure B).
The study contributes to OSM Science in several ways. First, it introduces the geometric properties of mapped features — specifically, their average added size — as a completeness signal, a dimension absent from existing literature. The method can be implemented for various way feature types and applied for different confidence intervals (using the α,β and γ thresholds), allowing a robust and flexible completeness assessment. Second, the metric's data requirements are exceptionally lightweight: it needs only aggregate statistics (feature count and total length or area) at regular time intervals, with no extrinsic datasets or further intrinsic attributes. Moreover, it seems to be indifferent to case study attributes. This makes it computationally inexpensive and universally applicable in principle. The results presented here are only exploratory, meant to understand how this method behaves. Next steps will focus on validating and refining it further, using extrinsic data sets such as Overture maps and a building dataset mapped by the Survey of Israel. Furthermore, the limitations of the metric, mainly the restrictiveness of the method, will be investigated in greater depth.