From wire to SOC alert:
how LYNX processes traffic
Every packet follows a strict, ordered pipeline. No layer is optional, no step can be reordered. The architecture guarantees that evasion-resistant normalisation runs before any detection logic — always.
Packet capture without kernel overhead
On Linux x86-64, DPDK binds directly to the NIC in poll-mode driver configuration. Packets flow from NIC hardware queues into huge-page-backed memory without any kernel copy. On all other platforms, libpcap provides an identical interface.
The ASIC on hardware firewall deployments forwards all normal traffic at wire speed, punting only suspicious traffic to the IDS CPU. IDS cores are pinned via CPU affinity so the OS scheduler, timer interrupts, and RCU callbacks never touch them.
A lock-free SPSC ring buffer using C11 _Atomic operations sits between the capture thread and the IDS dispatcher. The capture thread never waits on any downstream consumer. The dashboard reads live metrics from a shared memory region mapped as read-only — zero overhead on the hot path.
Flood isolation and ML pre-filter
Before any deep inspection, two gates reduce the volume of traffic reaching expensive layers.
The flood separation gate computes Shannon entropy over the 5-tuple source distribution in a 500ms sliding window. Traffic classified as a flood is diverted to a separate token-bucket rate limiter and never enters the IDS queue. This means an attacker cannot trigger graceful degradation mode by flooding — the IDS never sees the flood's volume.
The ML pre-filter runs per-packet inference on a 24-feature header-only vector using XGBoost exported to ONNX Runtime (or an auto-generated C decision tree on MIPS). Packets classified as obviously malicious enter shadow mode — logged but not dropped — until the model's false-positive rate is verified below threshold over a configurable burn-in window.
Canonical stream — evasion eliminated
Every IDS evasion technique that has ever worked exploits ambiguity at the protocol level. IP fragmentation overlap, TCP segment overlap, IPv6 routing header abuse, QUIC tunnelling — all exist to present different byte streams to the IDS and to the endpoint.
LYNX's normalisation gate eliminates every known class of protocol-level evasion before any detection logic runs. IPv4 datagram reassembly with first-wins overlap policy. TCP stream reassembly with overlap logging (overlaps are themselves anomaly signals). IPv6 extension header parsing for all standard types. QUIC/HTTP3/MASQUE stream reassembly from UDP frames.
Everything above this layer sees one canonical, unambiguous byte stream. There is no ambiguity left to exploit.
Flow reconstruction and fingerprinting
Normalised stream segments are handed to the flow engine, which maintains a cuckoo hash table of bidirectional sessions keyed by 5-tuple canonical form. Cuckoo hashing gives O(1) worst-case lookup — critical for the hot path. A hard memory cap with LRU eviction prevents the flow table from growing unbounded regardless of attacker-controlled connection volume.
The fingerprinting layer then derives a composite pseudo-identity for each session without endpoint access. JA4 TLS fingerprinting (replacing JA3S, which is meaningless under ECH) produces three component values — transport, cipher list, server selection — combined into a composite score. HTTP header order and timing characteristics, TCP/IP stack identity (TTL, window size, options order) contribute additional signals. Identity is a weighted composite, not a single hash.
Kill-chain correlation across time
Feature aggregation extracts session-level timing, volume, and destination attributes from metadata only — no payload, no decryption. Inter-arrival time statistics, byte directionality ratio, destination geo-region and ASN rarity, protocol mix — all derived from observable network behaviour.
The behavioural engine maintains a per-entity baseline using time-decay weighted rolling averages and a kill-chain state machine. The state machine correlates recon → initial access → lateral discovery → privilege escalation → exfiltration patterns across a configurable 1-hour to 7-day correlation window. Individual sessions that appear legitimate in isolation are linked when they form a coherent attack sequence.
A baseline poison detector monitors the rate of change of per-entity baseline parameters, flagging entities whose normal model is drifting faster than the configured threshold — a signal for slow adversarial baseline manipulation.
Signal fusion and TinyLlama verdict
Normalisation anomaly scores, fingerprint deviation scores, behavioural outlier magnitudes, and Snort-compatible rule engine hits are fused into a single context-aware risk score using configurable weights (default: 25% normalisation, 30% fingerprint, 35% behavioural, 10% rules).
An uncertainty gate routes the verdict: scores below the lower threshold are suppressed (clear false positives), scores above the upper threshold generate a direct SOC alert, and scores in between are escalated to the TinyLlama async queue.
TinyLlama-1.1B (QLoRA fine-tuned, Q4_K_M quantised, ~700 MB) runs as a dedicated worker thread off the hot path. It receives a structured text prompt built from wire-only flow graph data — no payloads, no decrypted content — and returns THREAT or FALSE_POSITIVE. Queue overflow sends alerts directly to the SOC rather than suppressing them. The system never silently suppresses an uncertain verdict.