Event Monitoring Enabling Responses to Anomalous Live Disturbances


EMERALD Overview

The EMERALD (Event Monitoring Enabling Responses to Anomalous Live Disturbances) environment is a distributed scalable tool suite for tracking malicious activity through and across large networks. EMERALD introduces a highly distributed, building-block approach to network surveillance, attack isolation, and automated response. It combines models from research in distributed high-volume event correlation methodologies with over a decade of intrusion detection research and engineering experience. The approach is novel in its use of highly distributed, independently tunable, surveillance and response monitors that are deployable polymorphically at various abstract layers in a large network. These monitors contribute to a streamlined event-analysis system that combines signature analysis with probabilistic inference to provide localized realtime protection of the most widely used network services on the Internet. The EMERALD project represents  a comprehensive attempt to develop an architecture that inherits well-developed analytical techniques for detecting intrusions, and casts them in a framework that is highly reusable, interoperable, and scalable in large network infrastructures. 

EMERALD presently includes several eXpert components that use PBEST rule based inference. These analyze network and host based data and comprise the most complete knowledge base in the field. EMERALD also includes eBayes components that use Bayesian inference.


The EMERALD eXpert (pronounced E-expert) is a highly targetable signature-analysis engine based on the expert system shell P-BEST. Under EMERALD's eXpert architecture (see summary viewgraph), event-stream-specific rule sets are encapsulated within resource objects that are then instantiated with an EMERALD monitor, and which can then be distributed to an appropriate observation point in the computing environment. This enables a spectrum of configurations from lightweight distributed eXpert signature engines to heavy-duty centralized host-layer eXpert engines, such as those constructed for use in eXpert's predecessors,NIDES (Next-Generation Intrusion Detection Expert System), and MIDAS (Multics Intrusion Detection Alerting System). In a given environment, P-BEST-based eXperts may be independently distributed to analyze the activity of multiple network services (e.g., FTP, SMTP, HTTP) or network elements (e.g., a router or firewall). As each EMERALD eXpert is deployed to its target, it is instantiated with an appropriate resource object (e.g., an FTP resource object for FTP monitoring), while the eXpert code base remains independent of the analysis target. For more information about the eXpert inference engine design, capabilities, and language, see http://www.sdl.sri.com/emerald/pbest-sp99-cr.pdf

Back to top


The EMERALD eBayes component extends our earlier work in anomaly detection by encoding probabilistic models of normal, attack, and anomalous behavior modes. Since inference is probabilistic, the approach retains the generalization potential of our earlier work in anomaly detection, while including sensitivity and specificity approaching signature models. The component can operate in an adaptive mode where the encoded models adjust for the observed traffic. Unlike anomaly detection, it is much harder to "train" this system to accept attack traffic as normal -- such traffic merely reinforces the existing attack models.

We presently have two prototypes: eBayes TCP (see summary viewgraph), which detects important classes of attacks visible in the TCP header data, and eBayes host, which dynamically detects degradation of provided services, whether due to malicious or non-malicious causes. Both use the same high performance Bayes inference class library developed specifically for the high throughput and low false alarm requirements of EMERALD. The system regularly runs live on our own TCP gateway. The eBayes TCP component is further described in the paper by Valdes and Skinner, Adaptive, Model-based Monitoring for Cyber Attack Detection.

Back to top

ASIC: Assessing Strategic Intrusions Using CIDF

In joint work with ISI/USC, we are leading the development of analysis methods to correlate intrusion reports, discern large-scale patterns of attack, and infer the intent of the adversary. We will develop methods for cooperative problem solving among heterogeneous intrusion detectors. We will use and extend the Common Intrusion Detection Framework (CIDF) as a basis for communicating information among the independent detectors. Further extensions to CIDF will allow us to incorporate the outputs from tools such as network managers and vulnerability scanners. Extensions to CIDF will also allow for the communication of the results of the correlation and assessment function itself, allowing for early warning and triggering response activities.

Features of our approach are that

  • The strategic intrusion assessment function is distributed throughout the system of systems, making it highly survivable and not dependent on the availability of specific detectors or event generators.
  • The strategic intrusion assessment can be carried out in a bottom-up and naturally scalable way, with each correlation engine operating individually and opportunistically discovering and exchanging information with other correlation engines that have relevant information.
  • The strategic intrusion assessment function can make use of heterogeneous "assessors" using many different methods of performing the assessment.
  • The use of CIDF allows us to make maximum use of the network of heterogeneous detectors and event generators.
  • The use of CIDF allows us to interoperate with automated response components and use the results of their response activities as further feedback to the assessment function.

Back to top


  Project Description Conceptual Overview Publications
  Research Opportunities Component Releases IDS Research
  System Design Lab SRI International Contact