VPN Encryption Is Not Enough: How DPI, TLS Fingerprinting, and Active Probing Expose Your Traffic

The CyberSec Guru

VPN Encryption Is Not Enough

If you like this post, then please share it:

Buy me A Coffee!

Support The CyberSec Guru’s Mission

🔐 Fuel the cybersecurity crusade by buying me a coffee! Why your support matters: Zero paywalls: Keep the main content 100% free for learners worldwide, Writeup Access: Get complete in-depth writeup with scripts access within 12 hours of machine drop.

“Your coffee keeps the servers running and the knowledge flowing in our fight against cybercrime.”☕ Support My Work

Buy Me a Coffee Button

Let me start with something that took me an embarrassingly long time to fully internalize: a VPN does not make you invisible on the network. It makes your payload unreadable. Those are completely different things and conflating them has led a lot of people including some who really should know better into a false sense of operational security.

The moment your device connects through a VPN tunnel, any competent observer on the other side of that connection can often tell. Not what you’re doing. But that you’re doing something worth hiding. And in certain environments such as oppressive government firewalls, corporate deep packet inspection, state-level traffic analysis, that inference alone is enough to get you blocked, flagged, or worse.

Let me break down exactly how this works.

What Encryption Actually Hides (and What It Doesn’t)

Encryption scrambles the contents of your packets. Anyone intercepting them sees noise. Your credentials, your messages, your DNS queries, your HTTP payloads – all of it is ciphertext to an observer without your keys. That part works.

But encryption does nothing to hide the shape of your traffic. And shape is far more revealing than most people expect.

Encryption Hides Payload, Not Shape
Encryption Hides Payload, Not Shape

Think about what “shape” means here: packet sizes, timing intervals, connection duration, protocol negotiation patterns, the sequence in which packets are sent, the distribution of entropy across your packet stream. None of this is encrypted. It can’t be – the network needs it to route packets correctly. And it is exactly what modern traffic analysis tools study.

This is not a new insight. Statistical traffic analysis has been an active research area since at least the early 2000s. What has changed is the operational deployment. These techniques, once confined to academic papers, now run inside national firewalls at ISP scale.

The Handshake Problem

Before a single byte of your real data crosses the wire, a VPN protocol has to negotiate a tunnel. That negotiation is a fingerprint.

OpenVPN initiates with a TLS handshake that uses a specific cipher suite order, specific extension fields, and a specific ServerHello response format. The default configuration hasn’t changed meaningfully in years. A classifier trained on OpenVPN traffic can identify it in the first two or three packets with high confidence.

WireGuard is even more distinctive. Its Handshake Initiation message is always exactly 148 bytes. It arrives on UDP port 51820 by default. The message includes a static sender index followed by a NOISE_IKpsk2 handshake with an ephemeral Curve25519 key. These structural constants make WireGuard trivially identifiable by any DPI system with a WireGuard signature.

IPsec/IKEv2 uses UDP 500 for IKE negotiation (or 4500 for NAT traversal), and the IKEv2 exchange follows a well-documented structure: IKE_SA_INIT, IKE_AUTH, and CREATE_CHILD_SA, each with recognizable header formats.

VPN Handshake Fingerprint
VPN Handshake Fingerprint

None of these were designed with hiding in mind. They were designed to be correct, interoperable, and fast. Concealment was an afterthought, if it was a thought at all.

How Deep Packet Inspection Works at Scale

DPI gets described in scary, vague terms a lot. Let me be specific about what it actually does.

A DPI system sits inline on the network at an ISP choke point, at a national border gateway, at a corporate egress firewall and it inspects every packet that passes through. “Inspects” does not mean it breaks your encryption. It means it reads the metadata: packet header fields, protocol indicators, connection behavior over time, and the statistical properties of the packet stream.

Deep Packet Inspection at ISP Scale
Deep Packet Inspection at ISP Scale

There are four main detection techniques that consistently work against unobfuscated VPN traffic.

1. Protocol Signature Matching

The simplest form of DPI. Every protocol has structural constants – magic bytes, message types, fixed-size fields in known positions that appear in every session. Snort and Suricata, the two dominant open-source IDS/IPS engines, ship with protocol detection rules for OpenVPN, WireGuard, IPsec, and dozens of circumvention tools. Commercial DPI appliances from vendors like NetScout, Huawei, and Sandvine have similar signatures, often maintained with far more resources.

A WireGuard Handshake Initiation, for instance, starts with a 4-byte message type field set to 0x01000000. That constant, combined with the packet arriving on port 51820 over UDP, is enough. The connection is flagged before the handshake completes.

2. TLS Fingerprinting (JA3/JA4)

Every TLS ClientHello includes a list of supported cipher suites, compression methods, extensions, and elliptic curve preferences. The specific combination and order of these values depends on the TLS library the client is using, the version of that library, and sometimes compile-time options.

JA3 fingerprinting, developed by John Althouse at Salesforce and published in 2017, hashes these ClientHello fields into a 32-character MD5 signature. JA4, its successor, uses a more structured format that’s more resistant to order-randomization countermeasures. Both are now standard in commercial network security products.

Your VPN client has a JA3 fingerprint. If that fingerprint is known to belong to OpenVPN, WireGuard’s TLS-based handshake variants, or any other circumvention tool, it will be caught by any system running signature lookups against a JA3 database. The cipher suites themselves are fine – no one can read your traffic. But the fact that the specific set of cipher suites in that specific order was sent by a specific piece of software is visible in plaintext before encryption negotiation finishes.

TLS Fingerprinting / JA3-JA4 Visual
TLS Fingerprinting / JA3-JA4 Visual

Mullvad VPN ran an interesting experiment a few years back trying to randomize their TLS fingerprint to look more like Firefox. It works until someone maintains an updated signature database, at which point the randomized-but-consistent fingerprint becomes its own identifier.

3. Statistical Traffic Analysis

This is where it gets interesting and the solutions get harder.

Human web browsing has a recognizable statistical profile. You load a page. That’s a burst of requests to a handful of servers, each with a size distribution that looks like HTML, CSS, JS, and media files. Then you read. The connection goes relatively quiet for seconds or tens of seconds. Then you click something and the cycle repeats. This produces an irregular, bursty pattern with recognizable inter-packet timing gaps.

A VPN tunnel carrying that traffic wraps the pattern but doesn’t eliminate it. If you’re browsing through a VPN, the sizes and timing of the encrypted packets at the VPN layer still correlate with the underlying browsing traffic, though with some distortion from padding and batching.

But a lot of VPN traffic isn’t casual browsing. It’s someone tunneling out of a restricted network to access blocked services. That traffic pattern often looks different: sustained connection, high entropy data, consistent throughput for minutes or hours. Modern classifiers flag sustained high-entropy streams because they don’t match any known normal protocol’s behavior. HTTPS traffic bursts. Netflix buffers then streams at a consistent rate. SSH produces bursty asymmetric patterns. Genuinely random-looking, high-entropy data at consistent rates stands out.

Statistical Traffic Analysis
Statistical Traffic Analysis

This is why “just use more encryption” is bad advice. A double-encrypted payload looks even more like random noise at consistent throughput. You’re training the classifier to be more confident, not less.

4. Active Probing

This one surprised a lot of the security community when it became publicly documented.

Passive detection has limits. If a VPN server is well-obfuscated, passive signatures may not fire reliably. So some networks moved from passive observation to active testing: when a connection looks suspicious, the firewall sends crafted probe packets to the suspected server and analyzes the response.

The logic is simple. A real HTTPS server running on port 443 responds to an HTTP GET request with an HTTP response. A VPN server pretending to be an HTTPS server either drops the connection, sends a TCP RST, or responds with garbage. Any of those deviations from normal HTTP server behavior is a positive probe result.

Shadowsocks was largely defeated in China via exactly this mechanism. The initial passive detection rates were manageable. Active probing exposed servers at scale. Shadowsocks servers that responded “incorrectly” to probe packets were blocklisted. The Shadowsocks community spent significant effort on AEAD improvements that made passive detection harder, but it didn’t matter much once the active probing infrastructure was in place.

The IETF has documented this problem in RFC 8546 (“The Wire Image of a Network Protocol”) and related documents discussing “protocol ossification” and “fingerprinting resistance.” Active probing isn’t going away. It’s a cheap, high-yield detection technique.

Why Obfuscation Is the Right Frame (and What It Actually Means)

The correct mental model is: the goal is not to hide that a connection exists (impossible) or to hide what’s inside it (encryption handles that). The goal is to make the connection look indistinguishable from permitted traffic.

Obfuscation vs Detection Arms Race
Obfuscation vs Detection Arms Race

That’s obfuscation, and there are several approaches with meaningfully different levels of sophistication.

XOR Obfuscation (Don’t Bother)

The simplest form: XOR every byte of the payload with a static key. This defeats naïve signature matching that looks for specific byte sequences. It does not defeat anything else. Entropy analysis catches it immediately because XOR with a static key doesn’t change the statistical properties of the data in a meaningful way. It just shifts them. The JA3 fingerprint is unchanged because the TLS handshake happens before the obfuscation layer. DPI systems were updated to see through XOR obfuscation within months of it becoming common.

Some commercial VPNs still advertise “XOR obfuscation” or “Scramble mode.” This is largely theater.

obfs4 and Pluggable Transports

The Tor Project developed the pluggable transports framework specifically to address traffic analysis against Tor. obfs4 (Obfuscation Protocol 4) is currently the main transport used for Tor bridges in restrictive environments.

obfs4 works differently from simple scrambling. It uses a Mask-and-Obfuscate approach where:

  1. A 24-byte random seed is used to key an ECDH handshake, making every session initiation look statistically independent
  2. The Elligator 2 encoding maps Curve25519 public keys to uniform random-looking byte strings, hiding the mathematical structure of the key exchange
  3. Packet sizes and inter-arrival times are randomized using traffic shaping derived from the seed
  4. The entire stream is formatted as uniform random-looking bytes with no distinguishable headers

This makes obfs4 significantly harder to fingerprint passively. There are no structural constants in the handshake, no identifiable packet sizes, and no predictable timing. However, active probing can still work: an obfs4 server that receives a probe packet it can’t authenticate will silently drop it rather than responding, and “connection that drops probes silently” is itself a mild signal. The arms race continues.

VLESS + REALITY (Current Best Practice)

This is currently the most sophisticated obfuscation technique in widespread deployment, developed by the Xray-core project.

VLESS + REALITY
VLESS + REALITY

REALITY works by borrowing the TLS certificate and connection identity of a legitimate target domain – something like www.microsoft.com or a major CDN. The VPN client initiates what looks, to any external observer, like a genuine TLS 1.3 handshake to that domain. The server, if the client provides a valid REALITY authentication token embedded in the handshake, recognizes it as a REALITY client and establishes the VPN tunnel. If the server receives a probe packet without the token, it forwards the connection to the real target domain and acts as a reverse proxy, returning a genuine TLS response from the real site.

The result: passive fingerprinting sees a legitimate TLS connection to a legitimate domain with a real certificate, because it literally is. The certificate is not faked. It’s fetched from the real site in real time. Active probing receives a genuine HTTP response from the real destination server. There is no behavioral anomaly to detect.

This is meaningfully different from traditional domain fronting (which depended on CDNs tolerating the technique, and which has been largely closed off). REALITY doesn’t require CDN cooperation. It works by acting as a transparent proxy to the target domain for unauthenticated traffic.

The limitations: REALITY currently requires careful target domain selection. The target domain needs to be accessible in the country you’re operating in, needs to use TLS 1.3 (REALITY doesn’t support older TLS versions), and shouldn’t attract suspicion if your traffic pattern to it is unusual (connecting to microsoft.com for six hours continuously might raise questions, depending on the context).

Amnezia WireGuard (AmneziaWG)

Standard WireGuard’s handshake constants make it trivially detectable. AmneziaWG patches this by introducing randomized header fields, configurable packet sizes via Junk Packet parameters, and timing jitter. The Junk Packets (S1, S2, H1, H2, H3) are configurable random data prepended to handshake messages, destroying the fixed-size signature of the original WireGuard handshake.

The AmneziaVPN project publishes these parameters openly. The tradeoff is performance – random padding and timing jitter cost throughput. But in environments where the alternative is getting blocked entirely, the tradeoff is worth it.

The Cat-and-Mouse Problem, Realistically Described

I want to be direct about something: none of the above is a permanent solution, and anyone describing it that way is selling you something.

The reason is straightforward. Once a circumvention technique is deployed at scale, it can be studied. obfs4 has been analyzed extensively by researchers, and while its passive fingerprinting resistance is good, behavioral analysis of the servers themselves (latency patterns, response time distributions, server error rates) can identify them probabilistically. REALITY requires public infrastructure – the servers need IP addresses, and IP ranges can be blocklisted even without protocol-level detection. The collateral damage (blocking Microsoft’s entire IP range to block REALITY traffic) may be unacceptable in some contexts but perfectly acceptable in others.

The practical implication: single-protocol obfuscation is brittle. The more durable architecture is infrastructure that supports multiple protocols and switches between them based on what the local network allows. This is what tools like Outline (from Jigsaw) and some commercial VPN providers with “stealth mode” attempt to do.

What nobody has fully solved is automated switching that’s fast enough to be transparent to the user and accurate enough not to thrash between protocols when the network is borderline. That’s an active engineering problem, not a solved one.

What to Evaluate When Choosing a Tool

If you’re working in an environment where VPN traffic is actively hunted – journalism, security research, corporate environments with aggressive DPI, the marketing copy on a VPN’s website is nearly useless. Here’s what actually matters:

Protocol diversity. A tool that only supports OpenVPN and WireGuard will not survive a serious DPI environment. Ask specifically what obfuscation transports are supported. If the answer is “XOR scrambling” or “obfuscation mode” without further detail, treat it as no obfuscation.

Active probe resistance. Ask directly whether their servers respond to unauthenticated connections in a way that mimics legitimate services. REALITY does this correctly. Most tools don’t.

Automatic protocol fallback. When one protocol gets blocked, does the client automatically try another? Does this happen quickly and silently, or does it require manual intervention? For time-sensitive work, silent automatic fallback is important.

Operational transparency. Do they publish test results from deployment in restrictive environments? Mullvad, Tor Project, and a few others do publish fairly detailed technical documentation about what they’ve tested and what works. Most VPN marketing is just claims.

Fingerprint freshness. JA3 databases are updated continuously. A VPN client that was fingerprint-resistant six months ago may have an active signature today. This requires ongoing maintenance from the provider, not a one-time fix.

The gap between a VPN that works without complaint in Western Europe and one that works reliably inside a country running Sandvine or a similar national DPI apparatus is enormous. That gap is not primarily about encryption strength. It’s about traffic analysis resistance, active probe resistance, and operational agility.

Understanding why the gap exists is useful. What you do about it depends on your threat model, which is a different conversation, but one worth having before you’re in a situation that requires it.

Buy me A Coffee!

Support The CyberSec Guru’s Mission

🔐 Fuel the cybersecurity crusade by buying me a coffee! Your contribution powers free tutorials, hands-on labs, and security resources.

Why your support matters:
  • Writeup Access: Get complete writeup access within 12 hours
  • Zero paywalls: Keep the main content 100% free for learners worldwide

Perks for one-time supporters:
☕️ $5: Shoutout in Buy Me a Coffee
🛡️ $8: Fast-track Access to Live Webinars
💻 $10: Vote on future tutorial topics + exclusive AMA access

“Your coffee keeps the servers running and the knowledge flowing in our fight against cybercrime.”☕ Support My Work

Buy Me a Coffee Button

If you like this post, then please share it:

Discover more from The CyberSec Guru

Subscribe to get the latest posts sent to your email!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Discover more from The CyberSec Guru

Subscribe now to keep reading and get access to the full archive.

Continue reading