Juliacon 2024

The 3D Lightning Mapping Array (LMA) is a regional network for the detection of lightning channels as they propagate through the cloud. Designed 25 years ago by New Mexico Tech, more than a dozen LMAs are now being operated across the globe. This talk will explain how I use Julia for processing LMA data processing, the visualization, and performance considerations. The power of Julia allows additional (novel) data analysis to be done while allowing a responsive interactive data browsing experience.

The functionality of the LMA visualizer is already rich, and it is my intention to create a package for release to the greater community within a year or so.

Background

An LMA is a regional network of VHF antenna stations spaced 5-50 km apart that detect lightning pulses, typically in the 60-66 MHz band. The pulses are timed by GPS. Time-of-arrival reconstruction results in 3D pulse locations, revealing the lightning leader channels (of negative polarity) inside the cloud and toward ground. The detection range of a sensitive network can reach over 250 km from the center. More than a dozen LMAs are now operational world wide. They are used for lightning science, nowcasting of severe weather and airspace safety.

In 2011, the Ebro 3D Lightning Mapping Array in Spain was the first LMA to be installed outside the USA, consisting of 12 stations in 2012-2014 and was subsequently split in half in 2015 to facilitate a LMA in Colombia. In 2023 and 2024, the system was upgraded to 15 stations spread out mostly across western Catalonia. It has detected very interesting cases of supercell storms with rising bubbles of lightning activity and even a “lightning hole” in hailstorms, as well as very large horizontal flashes that lasted for 6 seconds before they ceased, spanning 130 km. More information on the Ebro LMA, real-time monitoring and the eLMA services can be found at https://elma.upc.edu.

An interactive lightning visualization and analysis tool for the LMA is being developed using Julia and Makie. It is a suite of tools that focus on (a) lightning flash analysis and general browsing of activity, (b) storm activity evolution analysis, and (c) network performance analysis based on the data.

An automated leader speed tool was previously programmed in Scilab, using Theil-Sen slope estimator of points, which has been successfully implemented in Julia, reducing waiting times from many minutes to generally less than 10 seconds. Leader speed turns out to be the best tool for diagnosing the overall electrical charge structure of a thunderstorm.

New tools include a flash grouping algorithm based on DBSCAN. Noisy points that do not belong to lightning leaders are filtered out, and these themselves can hold information which is subsequently going through another DBSCAN with different settings, to isolate sparse, local, but continuous activity such as sparks near the cloud top and sparks at wind turbines. Another DBSCAN setting allows to isolate the leader branches within flashes.

The tool also can read cloud-to-ground stroke data from other networks, and Meteosat Third Generation Lightning Imager or GOES Geostationary Lightning Mapper (GLM) to be compared with LMA.

A grid-based storm activity analysis tool has now been designed. It is based on local 3D flash density and its evolution in time visualizes trends in 3D activity, rather than the ubiquitously used general flash rate. It uses sparse arrays to manage memory issues, with certain performance considerations.

Lightning leader speed, polarity and the repetition rate of leaders in flashes may hold new information that could turn out to be useful in monitoring the thunderstorm state and potential for severe weather.