Network Protocols: attacks & security analysis

Network protocols define the rules governing data exchange between systems on a network. Understanding network protocols is critical in cybersecurity because each protocol introduces a specific attack surface. TCP/IP, for instance, relies on a stateless handshake mechanism that can be abused via SYN flood or session hijacking. ARP operates without authentication, making it inherently vulnerable to spoofing. DNS resolves domain names without verifying the integrity of responses, enabling cache poisoning and DNS tunneling. BGP propagates routing information between autonomous systems with minimal validation, exposing global routing infrastructure to hijacking. SMB, widely deployed in Windows environments, has historically been the vector for critical exploits such as EternalBlue. HTTP and HTTPS, despite TLS encryption, remain targets for downgrade attacks and certificate abuse. Each protocol has a defined RFC specification that describes its normal behavior. Deviation from that specification is often the first indicator of an active attack. Analysts and pentesters must understand protocol state machines, header structures, and negotiation sequences to correctly identify both legitimate traffic and malicious manipulation.

Blue Team detection of protocol-level attacks relies on a combination of traffic analysis, log correlation, and behavioral baselining. Wireshark and Zeek are the primary tools for packet-level protocol inspection. Zeek generates structured logs for DNS, HTTP, SSL, SMB, and DHCP flows, making anomaly detection tractable at scale. For ARP spoofing, detection focuses on gratuitous ARP messages and sudden MAC-to-IP mapping changes visible in switch CAM tables or via tools like arpwatch. DNS anomalies include high query rates, long subdomain labels, non-standard record types, and responses from unauthorized resolvers. Windows Event ID 5156 logs allowed network connections via the Windows Filtering Platform. Event ID 4624 combined with unusual source ports or protocols may indicate lateral movement over non-standard channels. SMB-based attacks generate Event IDs 4648, 4771, and 5140. SIEM rules should correlate DNS query spikes with process creation events to identify DNS tunneling. NetFlow data enriched with IPFIX records helps detect volumetric anomalies and scanning patterns. Passive OS fingerprinting via p0f or Zeek's software framework helps identify unexpected protocol implementations.

Red Team operations frequently target network protocols to achieve initial access, lateral movement, or data exfiltration. ARP spoofing using tools such as Bettercap or Ettercap enables man-in-the-middle positioning on a local segment. DNS poisoning can redirect victims to attacker-controlled infrastructure, bypassing host-based controls entirely. BGP hijacking requires access to a misconfigured or compromised autonomous system but allows large-scale traffic interception. SMB relay attacks, executed with Responder and ntlmrelayx, capture and forward NTLM authentication without cracking credentials. LLMNR and NBT-NS poisoning remain effective in environments where these legacy protocols are not disabled. IPv6 rogue router advertisements via mitm6 can force clients to use attacker-controlled DNS, even in primarily IPv4 networks. Protocol downgrade attacks on TLS leverage weak cipher suite negotiation or missing HSTS headers. Countermeasures include enforcing SMB signing, deploying DNSSEC and DNS-over-HTTPS, filtering LLMNR and NBT-NS at the host firewall level via GPO, enabling Dynamic ARP Inspection on managed switches, and using 802.1X port-based authentication to restrict network access to authorized devices only.