As cities are one of the most obvious phenomena on the Earth's surface arising from human activities, human productivity has increased significantly since the industrial revolution, which has led to the expansion of population and the acceleration of urbanization (Mumford, 1968; Sanchez-Rodriguez et al., 2005). The rapidly expanding built-up urban area has serious impacts on regional and global changes by modifying the characteristics of the underlying surface while exacerbating human activities such as fossil fuel combustion (Seto et al., 2012; Rodriguez et al., 2018). With complex interactions happening in global environmental changes, the evolution of urban scale and spatial distribution is an important part of global change research (Solecki et al., 2013; Seto and Ramankutty, 2016; Goldewijk et al., 2017; Bai et al., 2018; Kuang et al., 2021). Long-term data on historical urbanization trends and patterns will be conducive to contextualize the current urbanization, as well as to predict future trajectories of its process. In particularly, China has a history of urban construction for thousands of years, and it is also one of the countries with a relatively fast urbanization process in the world today (Gong et al., 2019; Liu et al., 2020). However, China's industrial revolution did not start until the end of the 19th century, while the pattern of cities in late imperial China in the Ming and Qing dynasties (1368–1911) laid the foundation for Chinese cities in modern time (Skinner, 1977).

The data used for the study in the historical period must take into account the availability and integrity even though there are many methods and indicators to assess the level of urbanization. The widely used data are the statistical material about the population and area of cities for the study of urbanization before the industrial revolution (Doxiadis, 1970). Significantly, population is an effective indicator of the level of urbanization for most current studies to estimate the historical urbanization levels (Chandler, 1987; Reba et al., 2016; Leyk et al., 2020). However, in the case of late imperial China, population is not fully applicable to the study of China's urbanization during the Ming and Qing dynasties for obvious limitations of and flaws in the data when the data on urban population usually originated from the regional level which included cities; thus few separate statistical data on the number of urban residents could be found, although the official demographics of China during this period were detailed and generally credible (Ho, 1959; Perkins, 1969; Cao, 2001a). For example, William Skinner (1977) used population as the key indicator to measure the urbanization of China in the 19th century. However, since China did not have reliable urban population data (UPD) until 1953, Skinner had to work backward in time, extrapolating better, more recent data to somewhat earlier dates and building up a consistent time series culminating in the fairly hard data for 1953. Skinner selected 1893 as the representative year and created a comprehensive file of over 2500 data cards designed to cover every city and town. Based on this database of more than 150 attributes (mainly including administrative level, circumference of city wall, postal status, population estimates, trade statistics, and steamship or rail traffic), cities were classified. Then he defined the urban population class intervals so that the upper boundary of each class was twice the lower boundary, and the following series was used: 1000, 2000, 4000, 8000, 16 000, 32 000, and so on. Finally, Skinner estimated the urbanization process of China in the 19th century. It is acceptable to use data of the 1950s to study the urbanization in the 19th century, but for longer-term research, the credibility and operability of this approach will be greatly reduced. In summary, the flaws in the original materials have led to a great controversy over the different versions of estimates on Chinese urban population during this period (Li, 1997; Cao, 2000; Cao, 2001b).

Another way to explore the urbanization process in the historical period is the restoration of the urban extents or the built-up areas of cities (He et al., 2002; Hedefalk et al., 2017; Lin et al., 2017; Qin et al., 2019; Uhl et al., 2021). However, before the popularization of scientific cartography in the 20th century, maps in China generally lacked the basis of surveying and mapping (Yee et al., 1994; Cheng, 2019) and could not be used to restore the built-up urban areas in the late imperial period precisely. In addition, there was a lack of statistical data on urban area in late imperial China. Therefore, researchers generally use alternatives to represent the built-up areas of Chinese cities in the late imperial period, and one of the most commonly used indicators is the extent of city walls (Skinner, 1977; He et al., 2002; Qin et al., 2019).

How can the extent of a city wall represent the urban extent? Here we must begin by attempting to summarize the city wall building history that existed in imperial China. The city wall is considered to be one of the basic symbols of ancient Chinese cities (Chang, 1986). But to be specific, cities in China were not always walled. In addition, the characteristics of city walls in different eras were not the same. During the 3rd to 10th centuries, small cities in China generally had no walls. Even regional capital cities only built small-scale city walls called Zi-cheng (Zi means small and Cheng means city wall). The Zi-cheng was built around the government and military barracks, just like castles in medieval Europe. Residential areas, markets, schools, and religious buildings were all outside the Zi-cheng (Lu, 2011). From the 10th to 13th centuries, there were some large-scale city walls built around residential areas, but they were generally confined to a few important cities. During the Mongolian-ruled Yuan Dynasty (13–14th centuries), many city walls were deliberately torn down. Only in the Ming and the Qing dynasties (14–19th centuries) did cities generally build large-scale walls to protect governments, temples, granaries, residences, and certain natural resources against invasion, tribal uprising, and peasant rebellion. According to many previous studies (Chang, 1970; Kostof, 1992; Knapp, 2000), city walls in this period were usually slightly larger than the built-up area of the city, and as the suburban areas grew, new and larger city walls were often built. Thus, the city wall in the Ming and Qing periods could be regarded as the urban fixation line which reflected the extent of the city. On the other hand, the Ming period and the first century of the Qing witnessed the extensive construction of city walls. A total of 80 % of cities in China had walls in the 15th century, and in the 16th century, 95 % of cities were walled (see the details in Sect. 5 below). Through the study of the extent of the city wall, it will help us to reconstruct the urban extent in late imperial China in the 15–19th centuries.

Historical materials in the Ming and the Qing dynasties in China recorded the length and construction time of the city wall of each administrative city above the county level in detail, which provided reliable information for restoring the scale of the city walls. Researchers have estimated the built-up area of Chinese cities in the late imperial period by converting the perimeter of the city wall into the area of the city wall (Skinner, 1977; He et al., 2002; Cheng, 2007). However, due to the shape of the city, walls were often irregular, and their construction years were different from each other; the mentioned built-up urban area estimation often produces large errors. In addition, the differences between the extent of the city wall and the built-up urban area have not been much discussed. There is still a lack of city wall and urban extent datasets with high resolution and definite age for late imperial China.

The aim of this project was to collect multiple historical data sources related to the city walls of late imperial China, digitize them, and make the China City Wall Areas Dataset (CCWAD) and China Urban Extent Dataset (CUED) for the late imperial period in the 15th–19th centuries. We used a similar method to produce a dataset of urban extent areas in Northwest China in the Ming and the Qing dynasties (Xue et al., 2018), and in this new database, we improved the research methods and extended the study area across China. Firstly, based on the historical urban morphology theory (Conzen, 1969), we restored the extent and construction time of walls of each administrative city in the Ming (1368–1643) and the Qing (1644–1911) dynasties and made the CCWAD product. Then, we analyzed the years and sites of the construction of the city walls, and we found six representative years that could illustrate the general level of urban extent in China of this period. Based on this strategy, we developed the product of the CUED for 1400, 1537, 1648, 1708, 1787, and 1866 across China. These datasets provide a foundation for understanding cities in the traditional agricultural society, and they will also be helpful in current and future research and practices in urban environmental and cultural sustainability.

This research is aimed at the cities in China in the 15th–19th centuries. The definition of a city is the same as the general research practice of ancient Chinese cities, namely administrative cities, including county, Zhou, Fu, and Ting. In addition, the military cities of the Ming Dynasty, Wei and Suo, and the Eight Banner cities of Manchu of the Qing Dynasty were added.

The research period consisted of the Ming and the Qing dynasties, and there were some differences in the territory of the two dynasties. In order to explore the temporal and spatial characteristics of late imperial China's urban extent, the study area is divided into five sub-regions based on landform types, local socioeconomic history, and ethnic distribution, as shown in Fig. 1. (I) Northeast China mainly covers the area to the east of Daxing'anling mountain and the north of the Great Wall of the Ming Dynasty. This region was sparsely populated until the influx of large numbers of immigrants in the 18th–19th century, and a number of cities were established at the end of the 19th century and the beginning of the 20th century. (II) Inner Mongolia was to the north of the Great Wall and was inhabited by Mongolian herdsmen in the 15th–19th centuries. (III) The traditional agricultural area was densely populated, with many cities and a long history. (IV) Xinjiang was located in the continental interior, and the population was concentrated in oases. It became the territory of the Qing Dynasty after the mid-18th century. (V) Qinghai–Tibet Plateau is mainly located on the Qinghai–Tibet Plateau, which is the highest-elevation plateau in the world. There were some historic cities on the edge of the plateau, but the administrative cities within it were established very late.

Now we attempt to extract urban extent data from CCWAD. It must be emphasized that although city walls could be a helpful indicator for representing the extent of cities, there are always gaps and delays in both definitions and spatiotemporal changes between the city walls and urban extents. The city wall was a functional building with high cost, and it would be built only when it was of vital importance to military and economic defense. Therefore, the extent of the city wall must be adapted to the physical boundaries of the built-up urban area at that time. However, the urban extent would not remain unchanged forever; it would change accordingly with the increase or decrease in urban residents. In contrast, after the city walls were built, the extent of the city walls generally did not change with the built-up areas over time. The overflowing population would build contiguous settlements outside the wall, especially during peaceful and prosperous periods. During these periods, the extent of city walls could not be consistent with the urban land use. In addition, the urban boundaries before the construction of the city wall were practically unknown. Finally, some special cities, such as those established in the northeast of China at the end of the Qing Dynasty and some urban concessions (such as the Shanghai concession) established by foreigners in the 19th century, often did not build city walls.

After considering the relationship between the size of the city wall and the urban extent, we think that the city wall could be regarded as the urban boundary at least during the period when the city wall exerts its functional role, and the closer the time to the construction of the city wall is, the more consistent the size of city walls and the urban extent is. Therefore, as long as the appropriate periods were selected, the extent of city walls in these periods could be very approximately regarded as the urban extent. In small-scale studies, users can refer to the above principles to select proper data from CCWAD and regard the size of city walls as the urban extents.

CCWAD may enough to satisfy the demand of local and case studies. However, long-term and large-scale urban extent data are highly desirable for urban studies. Since city wall can be regarded as a helpful indicator of the extent of cities, we hope to provide an acceptable national-scale urban extent dataset based on the CCWAD. This is the China Urban Extent Dataset (CUED). To make CUED, it is necessary to extract some suitable representative years from CCWAD to make the dates of city boundaries be in close proximity to the time that most of the city walls were built. This requires statistics and analysis of the city walls' area, the number of walled cities, and the total number of all cities.

We plotted the time series of the number of city walls built (Fig. 7b), the total number of cities (Fig. 7d), the total number of cities that built the city wall (Fig. 7e), and the percentage of the total number of cities (Fig. 7c). It can be seen in Fig. 7b that there were some connections between the number of wall constructions and the area of the extent of the walls. The periods of more construction were often of faster area growth, and the periods of less construction were always of area decline or unchanged. In 1368, there were 1375 cities in China, of which 851 had city walls, accounting for only 62 % of the total (Fig. 7c, d, e). However, in the year 1393, 70 % of cities had city walls; in 1469 it reached 80 %, in 1540 it was 90 %, and in 1576 it was 95 %. Since then, even though the number of cities fluctuated to a considerable extent, the proportion of cities with walls to the total cities remained stable between 95 % and 97 % for a long time. But after 1868, this percentage began to decline, and after 1900 it dropped sharply.

According to the above facts, we selected six base years when the area of the city wall extent was closest to the urban boundary from the six time periods (i.e., 1368–1404, 1405–1564, 1565–1662, 1663–1727, 1728–1860, and 1861–1911) to produce the CUED product for the 15th–19th centuries. The selection criteria for the representative years are as follows. Firstly, the proportion of cities with walls to the total cities should be higher. The proportion should generally be more than 90 %, except in the 14th and early 15th centuries. Secondly, after the city walls were built, the extent of the city walls generally did not change with the built-up areas over time, so the representative years should be within only 1 or 2 years after the end of a large-scale construction activities of the city wall period. In addition, the representative year should be selected at a moderate level of changes in the extent of the city wall within the period. Finally, the representative year should avoid major political and military events and severe natural disasters in order to reflect the general level of urban development in that period.

Therefore, we selected 1400, 1537, 1648, 1708, 1787, and 1866 from CCWAD as the representative years to develop the CUED product for the 15th–19th centuries. In these representative years, the extent of city walls and the urban extent were relatively close at the national level. CUED provides the long-term and national-scale urban extent data.

Based on the CCWAD product, we plotted the time series of the changes in the area of the city wall's extent. Taking the area of the city walls in 1368 (= 1087.06 km2) as the initial value, Fig. 7a reflects the changes in the area of the city wall area during the Ming and Qing dynasties in China. It can be seen that in the 14th–20th centuries, the extent of the city wall area grew at a slow rate. The smallest area of the city wall was in 1373 (= 1040.98 km2), and the largest area was in 1911 (= 1367.22 km2). According to the change in the slope of Fig. 7a, the area change in the city wall extent can be divided into six periods. Period 1368–1404 was in the early years of the Ming Dynasty: many cities were abandoned due to years of war, which led to a decline in city wall areas. However, these cities were quickly rebuilt, and many military cities were built, making the built-up area soon exceed the level of 1368. At the beginning of the 15th century, the Ming Dynasty abandoned the area north of the Great Wall, and most of the cities in this area were abandoned. After that, in the period 1405–1564, the area of city wall extents grew slowly. From the middle of the 16th century, the situation in the north and southeast was tense, and many cities there built outer city walls, which accelerated the growth of the area of the city walls (period 1565–1662). In the middle of the 17th century, the area of the city walls fell again partly because of the war in the late Ming and early Qing dynasties and also because the Qing government abolished many military cities built by the Ming Dynasty (period 1663–1727). The growth of the area of the city walls in the period 1728–1860 was very slow. Until the middle of the 19th century, the government opened up immigration to the northeast of China, and the area of city walls began to grow rapidly.

Figure 10 based on the CUED product shows the urban extent areas in some provinces in each representative year. Combined with Table 1 and Fig. 1, it can be seen that provinces in the northeast of Region III had the largest urban extent area in the late imperial period in the 15th–19th centuries. Hebei, where the capital Beijing was located, had the largest urban area. Jiangsu and Shanghai, an economically developed area, ranked second, and Henan, a populous province, ranked third. Shandong, Shanxi, and Zhejiang also have large urban areas. During the study period, the urban extent of the above provinces increased steadily or slowly, but Zhejiang province decreased slightly in 1708. That was because the Qing Dynasty issued an order to demolish some coastal cities at that time. The urban extents of other provinces in Region III were roughly the same. Among them, Anhui, Guangxi, Hubei, Hunan, Jiangxi, Sichuan and Chongqing had a long history of land development, and the urban extent remained stable during the 15th–19th centuries. Fujian, Guangdong, and Hainan decreased slightly in 1708 for the same reason as Zhejiang. Yunnan and Guizhou provinces developed intensively and built a number of cities in the early Ming Dynasty. In the middle and late Ming Dynasty, the urban extent of Shaanxi, Liaoning, Gansu, and Ningxia increased rapidly because of the severe military pressure faced by nomads at that time. Taiwan began large-scale development only after the 18th century, and some small cities were built mainly on the west coast.

Jilin and Heilongjiang, located in Region I, had no administrative cities in the Ming Dynasty. After the mid-18th century, with the influx of immigrants, a number of cities were established. Inner Mongolia, located in Region II, had a certain number of cities in the Yuan Dynasty (1271–1368) and the early Ming Dynasty, but by the middle of the Ming Dynasty, these cities were gradually abandoned. It was not until the late 18th century that Inner Mongolia rebuilt some cities with the influx of immigrants. Xinjiang, located in Region IV, was not under the rule of the Ming Dynasty. In the late 18th century, the Qing Dynasty completely conquered Xinjiang and established a number of administrative cities, and the cities of Qinghai in Region V were located in the valleys of the Yellow River and Huangshui River.

The datasets include the CCWAD for 1368–1911 and the CUED for 1400, 1537, 1648, 1708, 1787, and 1866 are publicly available and can be downloaded from 10.6084/m9.figshare.14112968.v3 (Xue et al., 2021).

For CCWAD we provide a shapefile file (referring to files with .cpg, .shp, .dbf, .shx, .sbn, .sbx, and .prj extensions). Appendix A provides an introduction to the attributes of CCWAD. For CUED we provide six shapefile files (referring to files with .cpg, .shp, .dbf, .shx, .xml, .sbn, .sbx, and .prj extensions). Appendix B provides an introduction to the attributes of CUED.

Ultimately, we view CCWAD and CUED as a beginning compilation of a richer historical, city-level urban database for late imperial China. Despite the current reliability gaps, these datasets provide a spatially explicit, long-term historical record of walled cities and urban extent of China, especially since no alternative geocoded dataset at such a resolution exists. As a result, this dataset could be used as a foundation to build a full and accurate record of built-up urban areas through history, creating systematic, global built-up area data to measure urban growth on a long timescale.

However, we caution potential CCWAD and CUED users of the following limitations and dataset details:

The urban extent dataset (CUED) is derived from the city wall dataset (CCWAD). Strictly speaking, the extent of the city wall cannot be completely equal to the size of the urban extent. The data may better reflect the urban extent in the year when the city wall was built. The lifetime of each urban extent provided by the CCWAD is a period of time, and the urban extent of any year within the time period can be determined. However, if the year to be determined is too far from the year of the construction of the city wall, the actual urban extent may have a large difference from the wall's extent. Before the construction of the city wall, in fact, we could hardly know the actual size of the urban extent, and only the later wall's area was referred to. More often, after the city wall is built, as time goes by, the area farther away from the city gates and the center gradually becomes uninhabited and even becomes cultivated land; the area with convenient transportation outside the city gates forms new built-up areas. Therefore, we recommend that potential CCWAD users should be careful not to be too far away from the year of construction of the city wall when choosing the research years. This was why we generated six representative years in the CUED product in 15th–19th century China.