July 15 at 2:30 p.m.
Doctoral Programme | Computer Science
Defense | Detection of Encrypted Malware Command and Control Traffic
Student | Carlos António de Sousa Costa Novo
Date: July 15
Time: 2:30 p.m.
Venue: Room FC6 0.29
President:
Manuel Eduardo Carvalho Duarte Correia
Associate Professor
Department of Computer Science, Faculty of Sciences, University of Porto
Arguentes:
Daniele Cono D'Elia
Tenure-track Assistant Professor
Department of Computer, Control and Management Engineering, Sapienza University of Rome (Itália)
Fernando Manuel Valente Ramos
Associate Professor
Department of Computer Engineering, Instituto Superior Técnico, University of Lisbon
Vogais:
Rolando da Silva Martins
Assistant Professor
Department of Computer Science, Faculty of Sciences, University of Porto
Bernardo Luís Fernandes Portela (Orientador)
Assistant Professor
Department of Computer Science, Faculty of Sciences, University of Porto
Abstract:
Society’s ever-growing dependence on digital services for storing and managing information, turns these into increasingly attractive targets for cyberattacks. In response, malware deployments have evolved towards complex infrastructures that can remotely orchestrate and monetize large scale attacks, commonly relying on Command and Control (C2) channels for this purpose. Network-based detection systems have offered practical means to identify compromised machines and mitigate infections, either through signature-based approaches, or by using classification supported by Machine Learning (ML). However, malware has increasingly used Transport Layer Security (TLS) for its C2 traffic. This protocol, designed to legitimately protect communications from eavesdroppers, reduces the visibility for defenders and limits the effectiveness of approaches that depend on handshake metadata for classification or even dataset labeling.
This work explores encrypted C2 traffic detection under evolving protocols, proposing novel techniques for feature extraction based on information that can be externally observed, namely the lengths, directions and types of TLS records exchanged between client and server. By exploiting early-flow patterns that remain available without decryption, we obtain detection results that suggest applicability even to modern protocol versions, specifically TLS 1.3.
In addition to the labeling difficulties, network traffic data is also hard to share due to privacy and legal concerns. To address these, we explore the feasibility of deploying our detection mechanisms in a Federated Learning(FL)setting. Experimental results validate the possibility of training models in mutual collaboration without sharing private data. At the same time, they also demonstrate the impact of an adversarial attack by a malicious participant that intentionally degrades modelperformance. Ourworkillustrateshowrobust aggregation strategies can be used to mitigate poisoning attacks in this setting.
This work advances the understanding of ML-based encrypted C2 traffic detectors, feature extraction methodologies, and the need for considering evolving protocols, adaptive threats and collaboration challenges. We present new datasets, classifiers, and feature extraction techniques, and introduce a test setup and baseline results for assessing FL related threats.