Confessions of a Linux Drama Queen: Incident Response When Hackers Try to Steal Your Spotlight
2025-10-21 , Europe

It's one of those mornings. You just crushed your early workout, feeling all kinds of invincible, you're halfway through your first sip of coffee, mentally planning your day, when your SOC team drops a bombshell: Suspicious activity has been detected on a critical system. Suddenly, it's not the caffeine waking you up, it's sheer panic!!

But let’s be real—cyber drama is inevitable. What separates the pros from the panicked is how we respond. In the Linux world, post-compromise activity isn’t just a mess; it’s a story waiting to be told. From tracking suspicious IPs and unexpected file creations to analyzing logs and identifying rogue services, our job is to piece together exactly what happened and how.

Because let's face it, while trends come and go, resilience never goes out of style. Join me in this session as we turn the chaos into clarity and decode the drama, and maybe even add a little sparkle to incident response.


  1. About Me

    1. Oops, they did it again - What hackers do after they break in.

    2. Hackers be like "Why are you so obsessed with me" - Understanding the attacker's goals. Pyramid of pain.

    3. Diamonds might be forever, but logs are a girl's best friend - Logs and other mitigation strategies. XDRs are like a beauty bag, they can be customized with all your essentials (correlation searches) and pretty useful in case of an emergency fix! (Note: I will be showing relevant logs needed to detect post compromise activity).

    4. Getting our hands dirty.
      Note: I will be providing demos for these:

    • Start With Your EDR Logs: Uncover the juicy secrets.
      • High Risk Folders: Check high-risk directories for changes, such as /tmp, /var, and /usr/local. Attackers often hide malware in these directories, so flag unusual folder activity.
      • IPs: Use commands like netstat, ss, or ip addr show to monitor unusual outbound or inbound IP connections. Watch for IPs that are outside your organization or connected to known malicious domains. If it feels shady, it probably is!

    • Shell History: Linux’s way of spilling the tea!
      • Focus on logs from /var/log/secure or /var/log/auth.log to track shell access. Check for any unusual command executions that could indicate privilege escalation, file tampering, or lateral movement.

    • Newly Created Services: When new isn’t always better.
      • Attackers may install services under false pretenses. Look for services you didn’t authorize by running systemctl list-units --type=service. If it looks out of place, it probably shouldn't be there! Think of new services like "syscleaner" or "tempd". Sometimes they might look really legit like LenovoAutoUpdater.sh.
      • Dive into configuration files in /etc/systemd/system/ or /etc/init.d/ for service details. If anything looks too good to be true, it probably is.

    • Remote Monitoring & Management (RMM): Legit or suspect?
      • Attackers love to piggyback on legitimate RMM tools like OpenSSH or VNC, but they’ll use them to control your environment. Review /var/log/secure and /var/log/messages for abnormal usage of these tools.

    • Kernel Modules: Spotting the cheap knockoffs.
      • Check for suspicious kernel modules that could indicate privilege escalation or unauthorized system access. Commands like lsmod or modprobe are your best friends for this. If you spot unfamiliar modules, that's a massive red flag!

    • Check Event Logs Like you're reading gossip!
      • Dig into /var/log/syslog and /var/log/messages for general system events. Anything out of the ordinary, like strange restarts or crashes, could be the result of a compromise.
      • Use journalctl to filter and search logs for specific keywords like "failed," "error," or "unauthorized." Think of this as going through the receipts—everyone leaves a trail!

    • Suspicious Locations: The wrong wide of the internet.
      • Keep an eye out for login attempts from unusual geolocations or times. The last and lastb commands can show you the recent logins and failed attempts.
      • Review ss or netstat for abnormal network traffic. If connections are being made to strange, distant locations, you’ve likely got a problem on your hands.

    • A glimpse into the past.
      • Attackers might leave traces of their activity in system caches, especially if they’re interacting with the GUI. Look for artifacts in /home/user/.cache/ or /root/.cache/ directories. Tools like strings can extract any useful text from these cache files—like sifting through old screenshots and desktop snapshots.

    • Monitoring Resource Use and DHCP Logs: Your digital paparazzi.
      • You can use tools like atop to monitor resource usage. Look for CPU or memory spikes that could indicate hidden malicious processes.
      • DHCP Logs: Use logs from /var/lib/dhcp/ or /var/log/syslog to check for network traffic anomalies or devices connecting to your system. An unfamiliar device could be the hacker’s backdoor.

    • Automate the Hunt: Keep it stylish.
      • Set up automated detection with tools like auditd or OSSEC to keep tabs on file changes, service modifications, and unexpected network activity. Automation is like having your personal stylist, everything stays polished while you focus on bigger things.
      • Use security frameworks like AppArmor or SELinux to limit the damage attackers can do. They’re your invisible bodyguards, protecting critical files from being tampered with.

    1. How to shut it down - Mitigation and response.

    2. Final Thoughts:

    • Audit your logs regularly - Think of it as your daily dose of gossip!
    • Baseline, baseline, baseline! - Know what "normal" looks like.
    • Automate where you can.
    • Hardening your systems - Why security is like a skin care routine.
    • Practice incident response drills.
    • Stay in style. Stay up to date.
    • Trust your gut.

Melina Phillips is an Offensive Security Engineer with a background in Security Operations and Incident Detection. She has over ten years of IT experience and six years working directly in cybersecurity, blending hands on blue team work with her current focus on adversary simulation and endpoint compromise.

Her recent talks have been featured at Bsides Cambridge, Security Fest, BruCon and LeHack. She's known for making complex technical concepts accessible without watering them down, and for delivering practical insights grounded in real world attack and defense experience.

Outside of breaking into infrastructure and chasing down Linux threats, she's usually at CrossFit or playing with makeup, ideally not at the same time.