the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 24 Jun 2024
A derecho climatology (2004–2021) in the United States based on machine learning identification of bow echoes
Abstract. Due to their persistent widespread severe winds, derechos pose significant threats to human safety and property, and they are as hazardous and fatal as many tornadoes and hurricanes. Yet, automated detection of derechos remains challenging due to the absence of spatiotemporally continuous observations and the complex criteria employed to define the phenomenon. This study proposes a physically based definition of derechos that contains the key features of derechos described in the literature and allows their automated objective identification using either observations or model simulations. The automated detection is composed of three algorithms: the Flexible Object Tracker algorithm to track mesoscale convective systems (MCSs), a semantic segmentation convolutional neural network to identify bow echoes, and a comprehensive algorithm to classify MCSs as derechos or non-derecho events. Using the new approach, we develop a novel high-resolution (4 km and hourly) observational dataset of derechos over the United States east of the Rocky Mountains from 2004 to 2021. The dataset is analyzed to document the derecho climatology in the United States. Many more derechos (increased by ~50–400 %) are identified in the dataset (~31 events per year) than in previous estimations (~6–21 events per year), but the spatial distribution and seasonal variation patterns resemble earlier studies with a peak occurrence in the Great Plains and Midwest during the warm season. In addition, around 20 % of damaging gust (≥ 25.93 m s-1) reports are produced by derechos during the dataset period over the United States east of the Rocky Mountains. The dataset is available at https://doi.org/10.5281/zenodo.10884046 (Li et al., 2024).
- Preprint
(5825 KB) - Metadata XML
-
Supplement
(9705 KB) - BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on essd-2024-112', Anonymous Referee #1, 26 Aug 2024
The general idea of an AI-based objective tool to identify derechos is fascinating, and such a tool could be very useful. The approach taken in this paper has promise and it is exciting that some reasonably good results are obtained. However, I feel like there were some poor choices made in the design of the tool that harm its performance, and I list those below. In addition, I wonder if the timing of a paper like this is not good, in light of the fact that a very new physics-alone-based definition of a derecho is being proposed by some severe weather scientists (e.g. Squitieri 2024, abstract 16A.3 for AMS 28th Conf. on Sev. Local. Storms). In my own work with derechos and severe thunderstorm winds in recent years, I was aware that many in the community feel the definition much change from its current unusual focus on after-the-fact wind measurements. In meteorology, we normally define things based on their physics, and it sounds like that is what is being proposed by the severe weather community within SPC, which typically has been tasked with identifying derechos. It seems it would be much better for the AI work in this present study to be better coordinated with the group proposing the new definition, so that the AI tool is designed to work with the new definition. As it stands, I fear the AI tool will be obsolete from day 1, particularly in light of my concerns listed below. The Squitieri 2024 abstract indicates that the new definition reduces the number of derechos per year substantially, which would make the present study’s overestimate of derechos even worse. In summary, I believe the problems associated with the design of this tool are so severe that the paper cannot be published in its current form. The AI tool is really just identifying bow echoes that produce strong winds, not derechos. This presents numerous fatal problems in the discussion of results.
I am curious since prior definitions of derechos have included the length requirement (roughly 400 miles of damaging winds, or 650 km, in the more recent stricter definition of Corfidi et al. 2016 that you cite), why you would shorten the MCS longevity requirement to just 6 hours? If one assumes a typical MCS translation speed of 50 km/h, then an MCS lasting only 6 hours would only move over an area 300 km long, or less than half the pathlength needed in this prior definition of derechos. Even with the older 400 km pathlength requirement that you mention, many events would not meet it if moving at 50 km/h. Also, even if one acknowledges that derechos often move rather quickly (I believe the 2020 Midwestern one moved at around 90-100 km/h, which is about as fast as they can move), a 6-hour lifetime would still not result in a damage path quite long enough to match the 650 km definition. It is true that MCSs can produce significant damage with wind swaths shorter than 650 km, but if that is the focus of your work, you should not be referring to it as a derecho climatology. A high wind-producing bow echo climatology would be more appropriate for the title. In fact, in light of my opening paragraph and the ongoing efforts to change the definition of a derecho substantially, this change in title might avoid even more serious problems. It much better defines what your tool does since it is broader in its wording. I would strongly suggest that change.
I am troubled by your use of the surface wind station database for the winds used in your system. This is NOT how derechos are classified operationally now. Instead, classification is based on the Storm Events Database (SED), which I would assume includes more actual measurements (such as home weather stations) and estimates of wind speed based on damage. In fact, roughly 90% of all the severe wind reports in that database do not involve a measurement and are instead estimates based on damage. The definition of a derecho that I found on SPC’s website does not restrict derecho classification to just measured winds. The Storm Events Database is rather robust for the period 2004-2021 that you are using, and thus it is puzzling why you would not have used that for your training? Your use of just the surface station database is even more puzzling considering my comment earlier that your reduction in the longevity requirement for MCSs would lead your system to call many events derechos that would not meet the past requirements of a swath of severe winds at least 650 km long (or even 400 km long in many situations). Thus, you made one choice that really makes it easier to call a system a derecho, but then this choice of where you obtain wind information would do the opposite, making it much more difficult for systems to meet prior thresholds to be called a derecho. Tirone et al. (2024) used the SED in their training of a ML tool, so there is no obstacle to using that information as a source of thunderstorm wind information. I believe you need to test the sensitivity of your results to your use of a very limited database of thunderstorm winds, by examining changes that happen if you switch to using the SED. I see that later starting at about line 350, you provide good explanation which I think deals with my concern. It seems you acknowledge the deficiencies in your choice of this surface data network for wind and thus make numerous modifications to try to account for the deficiencies (like lower wind thresholds, use of broader ellipse containing reports, etc). You talk some about how damage estimates take some time to be performed, but it is unclear if this is the primary motivation for your use of the problematic surface wind network dataset. I believe you need to provide some of this justification earlier when you first mention that surface network, since most readers familiar with severe weather reports will question why you are not using the SED. Derechos are often classified now within 24 hours of their occurrence as the process of gathering both the measurements and having some estimates is quick. From what I have read so far, I do not see a reason why 24 hours is too long for what you are doing. You need to make a stronger case for your use of the surface wind measurements. Are you planning for your tool to be used operationally in a setting where it must alert forecasters the moment that a system has reached the requirement to be called a derecho? I guess I do not see why this would be so urgent. With all ongoing derecho research that I am aware of, your tool could easily be applied one day after an event, or even a week or month afterward, so there does not seem to be a valid reason to avoid using the SED.
When you refer to bow echo samples on line 180, it would be helpful to know the time resolution of the radar images you use. In recent years, radar reflectivity images often update nearly every minute or two. For readers to be able to put into context your 566 positive identifications, they need to be able to figure out how many images in total might be getting evaluated in the normal lifetime of a derecho. It is possible the citation about Gridrad would mention the time frequency of the Gridrad products, but this is simple information to supply in your own paper and is absolutely necessary. For a derecho lasting 8 hours, if radar images are available every 5 minutes (the traditional frequency of NEXRAD scans), there would be 96 for one case, and thus your 566 total bow echo scans from 54 events would be a tiny fraction of the lifetime of the derechos. If the images are hourly, then 566 implies every hour of every derecho must include a bow echo. Thus, this information is critically important.
I believe the results you state starting at about line 422 reflect some of the harm done by choices you made that I take issue with in my earlier comments. You do mention your reduction in wind threshold, but this could likely have been avoided had you used the SED instead of the surface wind network. Likewise, your overestimate compared to other studies is probably influenced by your unusual choice to reduce the MCS longevity criterion. You should try using a longer threshold than 6 hours to see how the numbers change. I do not believe you have made a case as to why you needed to reduce it to 6 hours. If your line 433 is implying that there may be derechos in the SPC database that are not really derechos since they did not come from organized convection, I highly doubt that. SPC usually avoids even showing wind damage reports unless they are clearly due to convection. There is no way I can conceive that a system would get listed as a derecho, because of that path length requirement), if it did NOT come from an MCS. Perhaps I have misinterpreted what you say in line 433, but if so, you need to be more direct in explaining why your MCS criteria is so important. From what I know about the SPC database, all events being called derechos had to be MCSs. For your AI work, obviously you need to have some criteria to ensure a system is an MCS, but when humans classify derechos, they are automatically ensuring this. Thus, I think the only impact of your MCS criteria would be if you are comparing your number of cases to numbers in other studies that used AI to classify derechos (and presumably to explain why you should end up with a smaller number than other studies that may not have bothered to ensure the winds were happening due to an MCS). In the context here, in your paper, however, I do not believe it makes sense.
Your discussion around line 450 again seems like it is needed there only because you did not use the SED database. If you had, you could avoid having to explain so many possible caveats.
Once again, starting around line 552, this discussion is reflecting the serious flaw in your design of your experiment. You chose to define derechos in a way inconsistent with the already problematic fact that at least two standard definitions exist. It is like you chose to study cats, but are calling them dogs, and now you are having to explain numerous differences between your study and prior ones. Because you used a different definition, it is impossible to know how serious the differences are. As I stated earlier, you are really just identifying bow echoes that were associated with strong winds. Not derechos. The question here becomes, is there a physical reason why strong wind bow echoes do not show the NW-SE swath of enhanced occurrence that is present for derechos? Or is there some fundamental issue with AI that is resulting in the difference. We cannot know because you chose to compare apples to oranges. The sensitivity test you do on line 554 is interesting and may offer some insight into my question above. But I believe your neglect of using the SED still complicates the interpretation you are providing here.
In lines 593-594, it would have been good to try to compare your rather large percentage of wind reports being due to derechos and DFs to an estimate of what prior studies showed. I have a feeling your number is much higher, which is consistent with the fact that you are actually studying all bow echoes that produce strong wind, and not true derechos.
Citation: https://doi.org/10.5194/essd-2024-112-RC1 -
RC2: 'Comment on essd-2024-112', Anonymous Referee #2, 28 Aug 2024
This article describes the development of a machine learning approach to create a derecho climatology across the United States. The novelty and originality of the work should be praised. The authors for the the most part have a well reasoned approach and methodology to creating this dataset, however there are a few major items of concern that stood out during this review:
I struggle with understanding which definition of a derecho the authors are using and also relying on to classify a feature as a derecho. In the background/introduction the authors present a history on the evolution of the definition of a derecho. I encourage the authors to keep this in the introduction, but I also encourage to authors to present the definition of a derecho they chose for their methodology clearly and provide additional reasoning on why the this specific definition was chose. The authors should really try to use a definition that most closely represents the official definition used by the National Weather Service and/or Storm Prediction Center. Using a definition that either has a shorter length requirement (or longer one) would impact the number of derechos that are classified in your results.
I do not understand the inclusion of surface wind speed observations in this manuscript. Derechos are classified operationally through the Storm Events Database (i.e. local storm reports), not through surface wind observations.
The organization of introduction needs quite a bit of improvement as well. It was very difficult to follow in terms of readability, partly compounded by the presentation of all the definitions of derechos. The introduction also presents Figure 1 which is a very very busy figure and in its current form, takes away from the paper. I recommend the authors overhaul the section to provide clarity on previous research, the definition of a derecho and motivation for their great ideas as far as developing this database.
It is difficult to evaluate the results that are presented, especially with the current derecho definition that is used. The current definition that is used (and with sfc wind obs) makes the number of derechos classified by this current form of research difficult to believe. Hopefully an overhaul in the definition used will provide a more realistic number of derechos identified. I do like Figures 9, 10, and 11 in presenting the results. These are great and easy to interpret graphics. I encourage these graphics to stay but adjusted with potential adjustments from the reviews. I would like to see potentially see how the next iteration of these graphics compare to actual confirmed derechos from the same time period.
Citation: https://doi.org/10.5194/essd-2024-112-RC2 -
RC3: 'Comment on essd-2024-112', Anonymous Referee #3, 25 Sep 2024
Major Comments
This paper presents a novel, machine-learning scheme to objectively identify derecho-producing convective systems. The paper reads well and, not withstanding the minor comments indicated below, to this reviewer seems both well-organized and well-presented. I have only two main comments.
First, being familiar with the vagaries of the severe-weather report database, I support the authors' use of the Integrated Surface Database (ISD). While the ISD arguably is subject to its own limitations, the quality-control algorithms used offer a higher and more universal level of uniformity than that associated with the severe-weather report database. The large number of derecho-producing convective systems identified with the current approach of the authors compared with those of previous studies largely reflects the rather low wind threshold employed; use of a somewhat higher threshold (and/or duration threshold) would lower the number of events identified. Obviously the true frequency of derecho-producing events remains unknown; the lower frequencies suggested by previous studies may be somewhat low.
The "true frequency" point brings into mind the main purpose of the present study --- objective identification of derecho-producing MCSs. The number of systems identified is sensitive to the underlying definition used in the scheme. This is where difficulty has arisen in the past and to some extent continues with the present paper. The omission of "forward propagating" from the current definition of the authors (page 516 ff) is problematic. Sustained forward-propagation is a fundamental aspect of derecho-producing convective systems. In absence of such a criterion one could argue that the approach described in the present paper is closer to that of a bow-echo detection scheme. Derecho-producing convective systems could be described as arising from bow-echo producing processes --- including rapid, sustained forward propagation --- that remain active for extended periods of time. I suggest re-visiting the abortive attempts made (lines 517-519) to identify the presence of forward propagation and refine the ML approach used here.
Minor comments (numbers refer to line numbers in version of 24 June 2024)
40. Change "magnitude" to "impact"
190. Consider adding a parenthetical description of "skip connections"
201. Clarify what is meant by "more distinct"
267. Define or reference "binary cross entropy loss"
307-310. Well-stated
312. Not completely sure what is meant by the "upper" and "lower" parts of the table
366. Add parenthetically, "Derecho feature" after "DF"
371-372. Not sure that this criterion would always be helpful...
385. Makes sense!
433-434. Agree with this focus; consider also maintaining the forward-propagating aspect.
455. Should "as" be "than"?
480. Consider parenthetically adding "cold-cloud shield" given that acronym has not been used since line 129.
520. The addition of a simple schematic to illustrate the angles mentioned would be helpful
525-526. Good to see this explicitly stated
556. Should "west-east" be "northwest-southeast"?
588. Should "2014" in Figure 12 caption be "2004"?
676. Capitalize "Weather" and "Review"
712. Add publication in which this manuscript appeared
731. Add year of publication (2020 (?), per line 234)
738. Capitalize "Python"
791. Capitalize "Atmospheric Sciences"
Citation: https://doi.org/10.5194/essd-2024-112-RC3
Data sets
A derecho climatology over the United States from 2004 to 2021 Jianfeng Li et al. https://doi.org/10.5281/zenodo.10884046
Bow echo detection and segmentation Andrew Geiss et al. https://doi.org/10.5281/zenodo.10822721
Video supplement
bow echo segmentation scheme Andrew Geiss https://youtu.be/iHWY_OhaVUo
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 529 | 130 | 127 | 786 | 39 | 18 | 20 |
- HTML: 529
- PDF: 130
- XML: 127
- Total: 786
- Supplement: 39
- BibTeX: 18
- EndNote: 20
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1