2024-08-06 –, Siena
They say everything on the Internet is forever, and while this may be true of your pictures from dinner last night, the reality is that everything on the Internet is NOT forever. In fact, much of the Internet is ephemeral, or flappy; services and hosts will appear online, only to disappear shortly after. This has major implications for research that utilizes Internet scanning and begs the question – how often should we be scanning the Internet, and how does this ephemerality differ across the Internet?
In this talk, I’ll discuss our findings from scanning the Internet every hour for a week. I’ll share some interesting anecdotes about where uptime differed across three main variables: L4 ports, L7 services, and ASNs. I’ll dive into examples where the portion of the Internet was fairly stable (e.g. popular protocols on their standard ports) and where uptime was, well, ephemeral (e.g. TCP SIP, HTTP on non-standard ports). I’ll discuss what these findings mean for the Internet Scanning community as a whole, implications for scanning research, and next steps. My hope is that attendees leave understanding just how ephemeral the Internet is, and what they should do about it.
In order to accurately measure the Internet, we need to understand how ephemeral it is. In this talk, I’ll discuss the results of a week-long experiment where we scanned a representative sample of the Internet every 30 minutes. Specifically, I’ll dive into these three main research questions, and their findings:
What L4 ports exhibit high ephemerality and low ephemerality?
What L7 services exhibit high ephemerality and low ephemerality?
Do we see higher or lower ephemerality on services running on non-standard ports?
What ASNs exhibit high ephemerality or low ephemerality?
Do cloud providers or other classifications of ASNs (according to ASDB) exhibit higher ephemerality than others?
What combinations of these three variables exhibit high or low ephemerality? What can we deduce from these trends?
What is the lifespan of an Internet service across (1) L4 Port, (2) L7 Protocol, (3) ASN, (3) Port + Protocol, and (4) Port + Protocol + ASN? How often do we need to scan ports/protocols/ASNs of interests?
Outline:
Intro – 3 Minutes
– Who I am
- What is Internet Scanning and why it matters in the scope of security research
- Overview of problem: How ephemeral is the internet? This matters because if you are worried about something on SIP, but SIP services go up and down faster than you are scanning them, you won’t catch all the nuance of their behavior. However, we can’t just scan everything all the time – we need to balance data with good Internet behavior.
Analysis Explanation (~15 minutes)
- walk through highlights of the 5 questions above, and their takeaways. These are based off of preliminary results and are subject to change (this research is still cooking :) )
-E.G preliminary results show protocols like SSH, SNMP, and TCP SIP have very high ephemerality, meaning that we see these protocols appear then disappear far faster than their counterparts like HTTP or HTTPS on standard ports. This means researchers looking for interesting aspects online on on these protocols should scan more
- E.G. preliminary results show that cloud providers have far more ephemeral of services than non-cloud provider ASNs. This is actually somewhat intuitive, due to the function of cloud providers (people are purchasing services all the time!). This also means that to better understand cloud providers, we need to scan them more than non-cloud providers, at least for anything temporally related
-E.G while we expected to see more ephemerality in the longtail of ports, that doesn't actually seem to be the case
Conclusion (2 minutes)
- Why this matters for anyone doing Internet scanning research: we don’t want to DDOs the internet constantly and we don’t want to piss people off. Instead, this research toes this balance by quantifying how the Internet acts, and showing how we can modify our research to accommodate the trends we see online. Good Internet stewardship for the win!
-What are next steps or future work for this research (e.g. expanding long tail of ports scanned to find more interesting trends)
Ariana Mirian currently works as a senior security researcher at Censys, where she uses Internet Measurement to answer interesting security questions. Prior to Censys, she received her PhD from UCSD, where her thesis focused on answering the question: how can we use large scale measurement and analysis to better prioritize security processes? When not geeking out about Internet Measurement and security, Ariana is also an avid aerialist and birder.