Google Patentsopen access

Anomaly detection based on ensemble machine learning model — Splunk Inc. (US12438891B1)

ABSTRACT

Abstract

A security platform employs a variety techniques and mechanisms to detect security related anomalies and threats in a computer network environment. The security platform is “big data” driven and employs machine learning to perform security analytics. The security platform performs user/entity behavioral analytics (UEBA) to detect the security related anomalies and threats, regardless of whether such anomalies/threats were previously known. The security platform can include both real-time and batch paths/modes for detecting anomalies and threats. By visually presenting analytical results scored with risk ratings and supporting evidence, the security platform enables network security administrators to respond to a detected anomaly or threat, and to take action promptly.

Description

This application is a continuation of U.S. patent application Ser. No. 16/503,181 filed on Jul. 3, 2019, and titled “Anomaly Detection Based on Communication Between Entities Over a Network,” which is a continuation of U.S. patent application Ser. No. 14/929,183 filed on Oct. 30, 2015, issued as U.S. Pat. No. 10,389,738, and titled “Malware Communications Detection,” which claims the benefit of U.S. provisional patent application No. 62/212,541 filed on Aug. 31, 2015, and titled “Network Security System,” each of which is incorporated by reference herein in its entirety.

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

FIELD OF THE INVENTION

At least one embodiment of the present disclosure pertains to distributed data processing systems, and more particularly, to intelligence generation and activity discovery from events in a distributed data processing system.

BACKGROUND

Activity detection, both friendly and malicious, has long been a priority for computer network administrators. In known public and private computer networks, users employ devices such as desktop computers, laptop computers, tablets, smart phones, browsers, etc. to interact with others through computers and servers that are coupled to the network. Digital data, typically in the form of data packets, are passed along the network by interconnected network devices.

Unfortunately, however, malicious activities can cause harm to the network's software or hardware, or its users. Malicious activities may include unauthorized access or subsequent unpermitted use of network resources and data. Network administrators seek to detect such activities, for example, by searching for patterns of behavior that are abnormal or otherwise vary from the expected use pattern of a particular entity, such as an organization or subset thereof, individual user, IP address, node or group of nodes in the network, etc.

Security appliances are used in known systems to provide network security. The appliance approach involves installing security appliances (which are typically servers or computers configured for providing security) at one or more locations in the network. Once installed, the appliance monitors traffic that traverses the network. Functions provided by the appliance may include malware detection, intrusion detection, unauthorized access or unauthorized use of data, among others. Unfortunately, security appliances cannot easily be scaled to handle temporary or permanent increases in network traffic. Increased network traffic often requires a security vendor to perform an appliance swap or an equally time-consuming appliance upgrade. Appliances also tend to have only limited network visibility because they are typically configured to monitor data traversing the link on which a respective appliance is installed only. Such an appliance will be unaware of activities occurring on other network segments monitored by other appliances and thus cannot use the additional context information pertaining to activities occurring on other network segments to detect a cleverly-designed piece of malware that may be difficult to detect from purely localized information.

Installed software products, rather than security hardware appliances, provide another approach to security for data networks. These products, such as anti-virus or anti-malware software, typically are installed on terminal devices (e.g., desktop and laptop computers, tablets, or smart phones). Data traversing the network between the terminal device is monitored by the installed products to detect malware in either inbound or outbound data. Unfortunately, installed software products also do not perform well in terms of scalability or network visibility. Installed products tend to be disposed locally on the terminal devices and thus also tend to have fairly localized views of the data on the network. They also tend to be installed on hardware that cannot be upgraded easily.

BRIEF DESCRIPTION OF THE DRAWINGS

Certain embodiments of the present disclosure are illustrated, by way of example, in the figures of the accompanying drawings, in which like references indicate similar elements.

FIG. 1 shows a general environment in which a security platform, which is an example application of a data processing system introduced here, may be implemented.

FIG. 2 shows an example of functional layers of a security platform.

FIG. 3 shows a high-level view of an example of the processing within the security platform.

FIG. 4 shows an example of the architecture of the security platform.

FIG. 5 shows an example implementation of the real-time processing path in greater detail.

FIG. 6 shows an example representation of the process of building adaptive behavioral baselines and evaluating against such baselines to support the detection of anomalies.

FIGS. 7 A and 7 B collectively show a table listing example types of machine data that can be found in different environments.

FIG. 8 shows an example implementation of the data intake and preparation stage of the security platform.

FIG. 9 A shows raw event data received by the data intake and preparation stage

FIG. 9 B shows an event-specific relationship graph based on the event shown in FIG. 9 A .

FIG. 10 shows an example implementation of an identity resolution technique based on the information in the events.

FIG. 11 shows a diagram of a technique for providing a uniform access interface (“event view”) at the data intake and preparation stage for an event processing engine (e.g., at a subsequent stage) to obtain relevant information from various kinds of machine data.

FIG. 12 shows a table of example uniform access interfaces (“event views”) that can be implemented in the data intake and preparation stage.

FIGS. 13 A and 13 B collectively show a table including example anomalies that can be identified by machine learning models and/or other types of processing entities, and various example uniform access interfaces and fields that can be used by the models/entities to receive relevant information about the events for performing further analytics.

FIG. 14 shows a sessionization technique that can be implemented at the data intake and preparation stage.

FIG. 15 is a block diagram of a machine learning-based complex event processing (CEP) engine.

FIG. 16 is a block diagram illustrating an architectural framework of a machine learning model.

FIG. 17 is a block diagram illustrating an example of the model type definition.

FIG. 18 is a block diagram illustrating an example system architecture for implementing the distributed computation system.

FIG. 19 is a flow diagram illustrating a method to execute a model preparation process thread.

FIG. 20 is a flow diagram illustrating a method to execute a model training process thread.

FIG. 21 is a flow diagram illustrating a method to execute a model deliberation process thread.

FIG. 22 shows a diagram of an example architecture of the security platform in which sharing of model state between real-time and batch processing paths can be implemented.

FIG. 23 is flow diagram illustrating at a high level, the processing hierarchy of detecting anomalies, identifying threat indicators, and identifying threats.

FIG. 24 is flow diagram illustrating in more detail an example process for detecting anomalies, identifying threat indicators, and identifying threats to network security.

FIG. 25 is a flow diagram describing an example process for detecting anomalies.

FIG. 26 is a flow diagram describing an example process for identifying threat indicators.

FIG. 27 shows a first use case for identifying threat indicators based on entity associations with detected anomalies.

FIG. 28 shows a second use case for identifying threat indicators based on entity associations with detected anomalies.

FIG. 29 illustrates a use case for identifying threat indicators based on duration of detected anomalous behavior.

FIG. 30 illustrates a use case for identifying threat indicators based on local and global rarity analysis.

FIG. 31 A illustrates identification of a threat indicator according to a first use case based on combining the outputs from different anomaly models.

FIG. 31 B illustrates nonidentification of a threat indicator according to the first use case based on combing the outputs from different anomaly models.

FIG. 32 A illustrates identification of a threat indicator according to a second use case based on combining the outputs from different anomaly models.

FIG. 32 B illustrates nonidentification of a threat indicator according to the second use case based on combing the outputs from different anomaly models.

FIG. 33 illustrates a use case for identifying threat indicators by enriching the anomaly data using data from external sources.

FIG. 34 is a flow diagram describing an example process for identifying threats to network security based on threat indicators.

FIG. 35 illustrates an example process of combining and storing event-specific relationship graphs into a composite relationship graph.

FIG. 36 illustrates an example of a composite relationship graph.

FIG. 37 illustrates an example of how a composite relationship graph can be stored as separate projections.

FIG. 38 illustrates a sample process of combining event-specific relationship graphs into a composite relationship graph and detecting a security threat based on the composite relationship graph.

FIG. 39 A is an illustrative home screen in a GUI of a system for monitoring potential computer network compromise, in accordance with various embodiments of the disclosure.

FIG. 39 B is an illustrative screen in the GUI of FIG. 39 A , depicting an expanded view of a “Views” tab selector, which enables a GUI user to select between viewing screens identifying instances of potential network compromise and viewing screens identifying entities associated with the instances of potential network compromise, in accordance with various embodiments of the disclosure.

FIG. 40 A is an illustrative view in the GUI of FIG. 39 A of a “Threats Review” screen for reviewing identified threats, in accordance with various embodiments of the disclosure.

FIG. 40 B is an illustrative view of an expanded “Threat Types” selector for filtering the “Threats Review” screen of FIG. 40 A to display only a selected type of threat, in accordance with various embodiments of the disclosure.

FIG. 40 C is an illustrative view of an expanded “Actions” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 D is an illustrative view of a “Threat Relations” diagram and a “Kill Chain” diagram, which are generated upon clicking the “Details” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 E is an illustrative view of a “Threat Anomalies Timeline,” “Threat Anomalies Trend,” and “Threat Anomalies” listing, which are generated upon clicking the “Details” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 F is an illustrative view of a “Device Locations” map, which is generated upon clicking the “Details” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 G is an illustrative view of a text bubble generated upon hovering a cursor over a device location in the “Device Locations” map of FIG. 40 F , in accordance with various embodiments of the disclosure.

FIG. 40 H i

This application is a continuation of U.S. patent application Ser. No. 16/503,181 filed on Jul. 3, 2019, and titled “Anomaly Detection Based on Communication Between Entities Over a Network,” which is a continuation of U.S. patent application Ser. No. 14/929,183 filed on Oct. 30, 2015, issued as U.S. Pat. No. 10,389,738, and titled “Malware Communications Detection,” which claims the benefit of U.S. provisional patent application No. 62/212,541 filed on Aug. 31, 2015, and titled “Network Security System,” each of which is incorporated by reference herein in its entirety.

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

FIELD OF THE INVENTION

At least one embodiment of the present disclosure pertains to distributed data processing systems, and more particularly, to intelligence generation and activity discovery from events in a distributed data processing system.

BACKGROUND

Activity detection, both friendly and malicious, has long been a priority for computer network administrators. In known public and private computer networks, users employ devices such as desktop computers, laptop computers, tablets, smart phones, browsers, etc. to interact with others through computers and servers that are coupled to the network. Digital data, typically in the form of data packets, are passed along the network by interconnected network devices.

Unfortunately, however, malicious activities can cause harm to the network's software or hardware, or its users. Malicious activities may include unauthorized access or subsequent unpermitted use of network resources and data. Network administrators seek to detect such activities, for example, by searching for patterns of behavior that are abnormal or otherwise vary from the expected use pattern of a particular entity, such as an organization or subset thereof, individual user, IP address, node or group of nodes in the network, etc.

Security appliances are used in known systems to provide network security. The appliance approach involves installing security appliances (which are typically servers or computers configured for providing security) at one or more locations in the network. Once installed, the appliance monitors traffic that traverses the network. Functions provided by the appliance may include malware detection, intrusion detection, unauthorized access or unauthorized use of data, among others. Unfortunately, security appliances cannot easily be scaled to handle temporary or permanent increases in network traffic. Increased network traffic often requires a security vendor to perform an appliance swap or an equally time-consuming appliance upgrade. Appliances also tend to have only limited network visibility because they are typically configured to monitor data traversing the link on which a respective appliance is installed only. Such an appliance will be unaware of activities occurring on other network segments monitored by other appliances and thus cannot use the additional context information pertaining to activities occurring on other network segments to detect a cleverly-designed piece of malware that may be difficult to detect from purely localized information.

Installed software products, rather than security hardware appliances, provide another approach to security for data networks. These products, such as anti-virus or anti-malware software, typically are installed on terminal devices (e.g., desktop and laptop computers, tablets, or smart phones). Data traversing the network between the terminal device is monitored by the installed products to detect malware in either inbound or outbound data. Unfortunately, installed software products also do not perform well in terms of scalability or network visibility. Installed products tend to be disposed locally on the terminal devices and thus also tend to have fairly localized views of the data on the network. They also tend to be installed on hardware that cannot be upgraded easily.

BRIEF DESCRIPTION OF THE DRAWINGS

Certain embodiments of the present disclosure are illustrated, by way of example, in the figures of the accompanying drawings, in which like references indicate similar elements.

FIG. 1 shows a general environment in which a security platform, which is an example application of a data processing system introduced here, may be implemented.

FIG. 2 shows an example of functional layers of a security platform.

FIG. 3 shows a high-level view of an example of the processing within the security platform.

FIG. 4 shows an example of the architecture of the security platform.

FIG. 5 shows an example implementation of the real-time processing path in greater detail.

FIG. 6 shows an example representation of the process of building adaptive behavioral baselines and evaluating against such baselines to support the detection of anomalies.

FIGS. 7 A and 7 B collectively show a table listing example types of machine data that can be found in different environments.

FIG. 8 shows an example implementation of the data intake and preparation stage of the security platform.

FIG. 9 A shows raw event data received by the data intake and preparation stage

FIG. 9 B shows an event-specific relationship graph based on the event shown in FIG. 9 A .

FIG. 10 shows an example implementation of an identity resolution technique based on the information in the events.

FIG. 11 shows a diagram of a technique for providing a uniform access interface (“event view”) at the data intake and preparation stage for an event processing engine (e.g., at a subsequent stage) to obtain relevant information from various kinds of machine data.

FIG. 12 shows a table of example uniform access interfaces (“event views”) that can be implemented in the data intake and preparation stage.

FIGS. 13 A and 13 B collectively show a table including example anomalies that can be identified by machine learning models and/or other types of processing entities, and various example uniform access interfaces and fields that can be used by the models/entities to receive relevant information about the events for performing further analytics.

FIG. 14 shows a sessionization technique that can be implemented at the data intake and preparation stage.

FIG. 15 is a block diagram of a machine learning-based complex event processing (CEP) engine.

FIG. 16 is a block diagram illustrating an architectural framework of a machine learning model.

FIG. 17 is a block diagram illustrating an example of the model type definition.

FIG. 18 is a block diagram illustrating an example system architecture for implementing the distributed computation system.

FIG. 19 is a flow diagram illustrating a method to execute a model preparation process thread.

FIG. 20 is a flow diagram illustrating a method to execute a model training process thread.

FIG. 21 is a flow diagram illustrating a method to execute a model deliberation process thread.

FIG. 22 shows a diagram of an example architecture of the security platform in which sharing of model state between real-time and batch processing paths can be implemented.

FIG. 23 is flow diagram illustrating at a high level, the processing hierarchy of detecting anomalies, identifying threat indicators, and identifying threats.

FIG. 24 is flow diagram illustrating in more detail an example process for detecting anomalies, identifying threat indicators, and identifying threats to network security.

FIG. 25 is a flow diagram describing an example process for detecting anomalies.

FIG. 26 is a flow diagram describing an example process for identifying threat indicators.

FIG. 27 shows a first use case for identifying threat indicators based on entity associations with detected anomalies.

FIG. 28 shows a second use case for identifying threat indicators based on entity associations with detected anomalies.

FIG. 29 illustrates a use case for identifying threat indicators based on duration of detected anomalous behavior.

FIG. 30 illustrates a use case for identifying threat indicators based on local and global rarity analysis.

FIG. 31 A illustrates identification of a threat indicator according to a first use case based on combining the outputs from different anomaly models.

FIG. 31 B illustrates nonidentification of a threat indicator according to the first use case based on combing the outputs from different anomaly models.

FIG. 32 A illustrates identification of a threat indicator according to a second use case based on combining the outputs from different anomaly models.

FIG. 32 B illustrates nonidentification of a threat indicator according to the second use case based on combing the outputs from different anomaly models.

FIG. 33 illustrates a use case for identifying threat indicators by enriching the anomaly data using data from external sources.

FIG. 34 is a flow diagram describing an example process for identifying threats to network security based on threat indicators.

FIG. 35 illustrates an example process of combining and storing event-specific relationship graphs into a composite relationship graph.

FIG. 36 illustrates an example of a composite relationship graph.

FIG. 37 illustrates an example of how a composite relationship graph can be stored as separate projections.

FIG. 38 illustrates a sample process of combining event-specific relationship graphs into a composite relationship graph and detecting a security threat based on the composite relationship graph.

FIG. 39 A is an illustrative home screen in a GUI of a system for monitoring potential computer network compromise, in accordance with various embodiments of the disclosure.

FIG. 39 B is an illustrative screen in the GUI of FIG. 39 A , depicting an expanded view of a “Views” tab selector, which enables a GUI user to select between viewing screens identifying instances of potential network compromise and viewing screens identifying entities associated with the instances of potential network compromise, in accordance with various embodiments of the disclosure.

FIG. 40 A is an illustrative view in the GUI of FIG. 39 A of a “Threats Review” screen for reviewing identified threats, in accordance with various embodiments of the disclosure.

FIG. 40 B is an illustrative view of an expanded “Threat Types” selector for filtering the “Threats Review” screen of FIG. 40 A to display only a selected type of threat, in accordance with various embodiments of the disclosure.

FIG. 40 C is an illustrative view of an expanded “Actions” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 D is an illustrative view of a “Threat Relations” diagram and a “Kill Chain” diagram, which are generated upon clicking the “Details” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 E is an illustrative view of a “Threat Anomalies Timeline,” “Threat Anomalies Trend,” and “Threat Anomalies” listing, which are generated upon clicking the “Details” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 F is an illustrative view of a “Device Locations” map, which is generated upon clicking the “Details” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 40 G is an illustrative view of a text bubble generated upon hovering a cursor over a device location in the “Device Locations” map of FIG. 40 F , in accordance with various embodiments of the disclosure.

FIG. 40 H is an illustrative view of a text bubble generated upon hovering a cursor over a line drawn between devices in the “Device Locations” map of FIG. 40 F , in accordance with various embodiments of the disclosure.

FIG. 41 is an illustrative view of a “User Facts” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 42 is an illustrative view of an “Anomalies Table” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 43 is an illustrative view of an “Anomaly Details” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 44 A is an illustrative view of an expanded “Watchlists” tab in the “Threats Review” screen of FIG. 40 A , in accordance with various embodiments of the disclosure.

FIG. 44 B is an illustrative view of an “Anomaly Details” screen in the GUI of FIG. 39 A , including a “Watchlists” designation, in accordance with various embodiments of the disclosure.

FIG. 45 A is an illustrative view of a “Threats Table” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 45 B is an illustrative view of a “Threats Trend” as a line chart.

FIG. 45 C is an illustrative view of a “Threats Trend” as a column chart.

FIG. 45 D is an illustrative view of a text bubble generated upon hovering a data point in the “Threats Trend” diagram of FIG. 45 A , in accordance with various embodiments of the disclosure.

FIG. 45 E is an illustrative view of a “Threats Details” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 46 A is an illustrative view of an “Anomalies Table” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 46 B is an illustrative view of the “Anomalies Trend” and the “Anomalies” listing in the “Anomalies Table” screen of FIG. 46 A , depicting a selection of a data point in the “Anomalies Trend” diagram.

FIG. 46 C is an illustrative view of a portion of an “Anomalies Details” screen for a selected anomaly in the GUI of FIG. 39 A , including a “Users” section, “Anomaly Relations” section, and “Triggering Event” section, in accordance with various embodiments of the disclosure.

FIG. 46 D is an illustrative view of a portion of an “Anomalies Details” screen for a selected anomaly in the GUI of FIG. 39 A , including an “Anomalous Activity Sequence” diagram, in accordance with various embodiments of the disclosure.

FIG. 46 E is an illustrative view of a portion of an “Anomalies Details” screen for a selected anomaly in the GUI of FIG. 39 A , including an “User Activities Baseline” diagram, “Compare Unusual Activity with the Accounts Profile” graphic, “Additional Information” graphic, and “Anomaly Graph” diagram, in accordance with various embodiments of the disclosure.

FIG. 46 F is an illustrative view of a portion of an “Anomalies Details” screen for an anomaly of the type “Machine Generated Beacon,” including a “Users,” “Devices,” and “Domains” sections, and an “Anomaly Relations” box.

FIG. 47 A is an illustrative view of a “Users Table” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 47 B is an illustrative view of a “User Facts” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 47 C is another illustrative view of a “User Facts” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 47 D is an illustrative view of a “User Threats” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 47 E is an illustrative view of a “User Anomalies” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 47 F is an additional illustrative view of a “User Anomalies” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 47 G is an illustrative view of a “Watchlist” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 48 A is an illustrative view of a “Devices Table” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 48 B is an illustrative view of a “Device Facts” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 48 C is an illustrative view of a “Device Anomalies” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 49 A is an illustrative view of an “Apps Table” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 49 B is an illustrative view of an “App Facts” screen in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 50 A is an illustrative view of a Geomap icon in a Threats Table view in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 50 B is an illustrative view of a Geomap in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 50 C is an illustrative view of a text bubble for a threat in a Geomap in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 50 D is an illustrative view of a text bubble for anomalies in a Geomap in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 51 is an illustrative view of an “Analytics Dashboard” in the GUI of FIG. 39 A , in accordance with various embodiments of the disclosure.

FIG. 52 shows an example of a probabilistic suffix tree based model receiving a particular sequence (e.g., during training).

FIG. 53 shows an example timeline illustrating the training of a probabilistic suffix tree based model, the establishment of a baseline prediction profile, and the activation of a particular model version.

FIG. 54 A shows an example of how a normal behavioral sequence may be represented in a probabilistic suffix tree based model.

FIG. 54 B shows an example of how an unusual behavioral sequence may be discovered in a probabilistic suffix tree based model.

FIG. 55 A shows an example user interface for displaying discovered anomalies.

FIG. 55 B shows an example user interface displaying additional information about the user.

FIG. 56 shows an example user interface displaying results from comparing the marginal of the events for the anomalous window and the entire data for the user (e.g., baseline).

FIG. 57 shows an example overview of all the unusual sequence as a timeline.

FIG. 58 shows an example user interface for an administrator to interact with each timeline event.

FIG. 59 is a flowchart showing an example of an automated process for cluster identification from a graph.

FIG. 60 A is a flowchart showing an example of an automated process for cluster identification from a bipartite graph.

FIG. 60 B shows an example of minimizing the L1-norm values to find optimal positions for the normal nodes.

FIG. 61 A shows an example use case corresponding to the cluster identification process for a generic graph, at a first stage of processing.

FIG. 61 B shows an example use case corresponding to the cluster identification process for a generic graph, at a second stage of processing.

FIG. 62 illustrates an example use case corresponding to the cluster identification process for a bipartite graph.

FIG. 63 is a block diagram illustrating a machine learning model that detects lateral movement in a computer network.

FIG. 64 illustrates an example of a bipartite graph showing events of users logging into network devices.

FIG. 65 illustrates an example of a process of assigning similarity scores to network devices.

FIG. 66 A shows an example of a bipartite graph having network devices that have many shared users and tend to have close similarity scores.

FIG. 66 B shows an example of a bipartite graph having network devices that have multiple shared exclusive users.

FIG. 66 C shows an example of a bipartite graph having network devices that have one shared user.

FIG. 66 D shows an example of a bipartite graph having network devices that are associated with similar groups of users.

FIG. 67 illustrates an example of a graph data structure for detecting security threats based on a detected out-of-profile anomaly.

FIG. 68 is a high-level conceptual diagram of an example network traffic scenario for detecting anomalies indicative of malware on a computer network.

FIG. 69 is a high level flow diagram illustrating an example process for detecting an anomaly indicative of malware based on network traffic.

FIG. 70 shows two tables illustrating application of lexical analysis to a character-based entity identifier.

FIG. 71 shows an example incident response output configured for display to a user.

FIG. 72 shows an example graph of outgoing traffic from a network device.

FIG. 73 lists an example of a set of parameters that can be considered for distinguishing between machine-generated traffic and user-generated traffic.

FIG. 74 is a block diagram of an environment in which a system for detecting anomalies in machine-generated traffic can be implemented.

FIG. 75 is an example of a memory cache storing beacon types that are identified as likely to be anomalous.

FIG. 76 is a flow diagram of a process for determining whether outgoing traffic from a device is anomalous.

FIG. 77 is a flow diagram of a process for forming a group of connection requests from outgoing traffic of a device.

FIG. 78 is a flow diagram of a process for determining whether a particular group of connection requests in the outgoing traffic of a device is user-generated traffic or machine generated traffic.

FIG. 79 is a flow diagram of a process for determining whether machine-generated traffic is anomalous.

FIG. 80 is a block diagram of an environment in which a system for detecting anomalies based on rarity scores of features can be implemented.

FIG. 81 shows a table of example features and/or feature pairs to be considered for determining whether an example event is anomalous, consistent with various embodiments.

FIG. 82 shows a table listing example thresholds and/or parameters of a rarity criterion for various example events that can be used for determining whether an event is anomalous.

FIG. 83 is a flow diagram of an example process for determining an anomaly based on a rarity score for a particular value of a feature.

FIG. 84 is a flow diagram of an example process for determining a rarity score for a particular value of a feature.

FIG. 85 is a block diagram of a computing device that may be used to implement the techniques introduced here.

DETAILED DESCRIPTION

References in this description to “an embodiment,” “one embodiment,” or the like, mean that the particular feature, function, structure or characteristic being described is included in at least one embodiment of the present disclosure. Occurrences of such phrases in this specification do not necessarily all refer to the same embodiment. On the other hand, the embodiments referred to also are not necessarily mutually exclusive.

In the following description, the example of a security platform is used, for illustrative purposes only, to explain various techniques that can be implemented by the data processing system. Note, however, that the techniques introduced here are not limited in applicability to security applications, security information and event management (SIEM) applications, or to any other particular kind of application. For example, at least some of the techniques introduced here can be used for automated fraud detection and other purposes, based on machine data. Additionally, the techniques introduced here are not limited to use with security-related anomaly and threat detection; rather, the techniques can be employed with essentially any suitable behavioral analysis (e.g., fraud detection or environmental monitoring) based on machine data. In general, “machine data” can include performance data, diagnostic information and/or any of various other types of data indicative of performance or operation of equipment (e.g., an action such as upload, delete, or log-in) in a computing system, as described further below. In general, “machine data” as used herein includes timestamped event data, as discussed further below. Examples of components that may generate machine data from which events can be derived include: web servers, application servers, databases, firewalls, routers, operating systems, and software applications that execute on computer systems, mobile devices, sensors, Internet of Things (IoT) devices, etc. The data generated by such data sources can include, for example, server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, etc., which are indicative of performance or operation of a computing system in an information technology environment.

In today's enterprises, attacks by users with trusted access often go undetected by existing security approaches. Indeed, traditional security products often suffer from several major drawbacks, including the inability to detect unknown threats and insider threats, and the inability to scale and process huge amount of data. Whether access is obtained by using compromised accounts/systems or by leveraging existing privileges to conduct malicious activities, nowadays attackers often do not need to employ additional malware. The patterns of these malicious activities vary dynamically, and attackers can almost always find ways to evade traditional security technologies, such as rules-driven malware detection, malicious file signature comparison, and sandboxing. Also, as the amount of the data increases, using human analysis to perform threat detection becomes increasingly expensive and time prohibitive and such human analysis does not allow the threat to be responded to in a timely and effective manner. Further, security analysts such as network administrators often use a “kill chain” methodology to identify and stop the progression of malicious activities (e.g., from intrusion to lateral movement, and to exfiltration). These analysts need supporting evidence to make educated decisions in the kill chain, but traditional security products generally do not provide the support for such methodology.

Introduced here, therefore, is a data processing and analytics system (and, as a particular example, a security platform) that employs a variety of techniques and mechanisms for anomalous activity detection in a networked environment in ways that are more insightful and scalable than the conventional techniques. As is described in more detail below, the security platform is “big data” driven and employs a number of machine learning mechanisms to perform security analytics. More specifically, the security platform introduced here can perform user behavioral analytics (UBA), or more generally user/entity behavioral analytics (UEBA), to detect the security related anomalies and threats, regardless of whether such anomalies and threats are previously known or unknown. Additionally, by presenting analytical results scored with risk ratings and supporting evidence, the security platform can enable network security administrators or analysts to respond to a detected anomaly or threat, and to take action promptly. The behavioral analytics techniques introduced here enable the security platform to detect advanced, hidden and insider threats. As one aspect of this disclosure, the behavior analytics leverage machine learning data processing procedures and do not require any preexisting knowledge such as known signatures or rules. The security platform can also improve threat detection and targeted response by using a variety of threat indicators. Further, the security platform supplies supporting evidence within context of the kill chain to enable targeted remediation of any detected anomaly or threat.

Moreover, as introduced here, the security platform can increase a security operations center's (SOC) efficiency with a number of rank-ordered lists having events in the context of a kill chain. In some examples, the kill chain can be linked from an anomaly or threat summary to the supporting information that is gathered over time. All generated security-related information can then be sent to a security information and event management (SIEM) application, such as the Splunk® App for Enterprise Security, to further scope, disrupt, contain and/or recover from the attack.

FIG. 1 shows a general environment 10 in which the security platform introduced here can be implemented. The environment 10 may represent a networked computing environment of one or multiple companies or organizations, and can be implemented across multiple geographic regions. One or more elements in the environment 10 are communicatively coupled to each other through a computer communications network, which can include the Internet and one or more wired or wireless networks (e.g., an Internet Protocol (IP)-based local area network (LAN), metropolitan area network (MAN) wide area network (WAN), a Wireless LAN (WLAN) network such as Wireless Fidelity (WiFi), and/or a cellular telecommunications network such as Global System for Mobile Communications (GSM) network, 3G network, or long term evolution (LTE) network). The computing devices shown in the environment 10 can be, for example, a personal computer, a smart phone, a computer server, a notebook computer, or any other form of computing system or device that allows a user to access the information within the environment 10 . Each of the aforementioned computer systems can include one or more distinct physical computers and/or other processing devices which, in the case of multiple devices, can be connected to each other through one or more wired and/or wireless networks.

The security platform can detect anomalies and threats produced by a user, a device, or an application, for example, regardless of whether the entity that causes the anomalies or threats is from outside or inside the organization's network. The security analytics techniques that can be adopted by the security platform include behavioral analytics that enable organizations of any size or skillset to detect and respond to unknown threats. Some specific examples that behavioral analytics can be based on include machine learning, behavior modeling, peer group analysis, classification, statistical models, and graph analysis. As introduced in more detail below, these analyses can utilize, for example, Markovian processing flows, inference and grouping processes, and risk scoring mechanisms to develop user and entity profiles in order to compare and contrast activities, which ultimately allow the platform to detect and expose anomalies and threats. Also, as mentioned above, the security platform can include a graphical user interface (GUI) that can create visualizations of the detected anomalies and threats within an organization, and optionally, map the threats across an attack kill-chain in a visual way, which the security analysts in the organization can quickly and easily assimilate.

The security platform can be deployed at any of various locations in a network environment. In the case of a private network (e.g., a corporate intranet), at least part of the security platform can be implemented at a strategic location (e.g., a router or a gateway coupled to an administrator's computer console) that can monitor and/or control the network traffic within the private intranet. In the case of cloud-based application where an organization may rely on Internet-based computer servers for data storage and data processing, at least part of the security platform can be implemented at, for example, the cloud-based servers. Additionally or alternatively, the security platform can be implemented in a private network but nonetheless receive/monitor events that occur on the cloud-based servers. In some embodiments, the security platform can monitor a hybrid of both intranet and cloud-based network traffic. More details on ways to deploy the security platform and its detailed functionality are discussed below.

By addressing the entire lifecycle of known and unknown advanced security threats, and by providing a platform to detect, respond to, and automate actions, the security platform introduced here provides a comprehensive solution to the security-related issues in a modern network environment.

I. Security Platform System Overview

The security platform introduced here is capable of handling large volumes of data, particularly machine data, from multiple data sources. These data sources may have different data formats and may provide data at very high data rates (e.g., gigabytes of data per second or more). In some embodiments, incoming data is processed using machine learning/data science techniques to extract knowledge from large volumes of data that are structured or unstructured. In a general sense, data science is a continuation and expansion of the field of data mining and predictive analytics, also known as knowledge discovery and data mining (KDD).

The security platform may be cloud-based and may employ big data techniques to process a vast quantity of high data rate information in a highly scalable manner. In certain embodiments, the security platform may be hosted in the cloud and provided as a service. In certain embodiments, the security platform is provided as a platform-as-a-service (PaaS). PaaS is a category of cloud computing services enabling customers to develop, run and manage Web applications without the complexity of building and maintaining the infrastructure typically associated with developing and launching such applications. PaaS can be delivered in at least two ways, namely: (i) as a public cloud service from a provider, wherein the consumer controls software deployment and configuration settings and the provider provides the networks, servers, storage devices and other services to host the consumer's application, or (ii) as software installed in private data centers or public infrastructure and managed by internal information technology (IT) departments.

Machine learning is employed in certain embodiments to make it unnecessary to know in advance what activity constitutes a security threat or a security threat signature. For example, a security threat may be discovered from the event data as the events occur even though that threat has not been seen before and no signature for that threat existed previously.

In various embodiments discussed herein, security threats are examples of a type of activity to be detected. It should be understood, however, that the security platform and techniques introduced here can be applied to detect any type of unusual or anomalous activity involving data access, data transfer, network access, and network use regardless of whether security is implicated or not.

In this description the term “event data” refers to machine data related to activity on a network with respect to an entity of focus, such as one or more users, one or more network nodes, one or more network segments, one or more applications, etc.). In certain embodiments, incoming event data from various data sources is evaluated in two separate data paths: (i) a real-time processing path and (ii) a batch processing path. Preferably, the evaluation of event data in these two data paths occurs concurrently. The real-time processing path is configured to continuously monitor and analyze the incoming event data (e.g., in the form of an unbounded data stream) to uncover anomalies and threats. To operate in real-time, the evaluation is performed primarily or exclusively on event data pertaining to current events contemporaneously with the data being generated by and/or received from the data source(s). In certain embodiments, the real-time processing path excludes historical data (i.e., stored data pertaining to past events) from its evaluation. Alternatively, in an embodiment, the real-time processing path excludes third-party data from the evaluation in the real-time processing path. These example types of data that are excluded from the real-time path can be evaluated in the batch processing path.

In this description the term “event” is sometimes used synonymously with the term “event data” to mean a discrete set of machine data that represents or corresponds to a specific network activity, although “event” can also refer to the underlying activity itself, as will be apparent from context.

Also in this description, an “anomaly” is a detected variation from an expected pattern of behavior on the part of an entity, which variation may or may not constitute a threat. An anomaly represents an event of possible concern, which may be actionable or warrant further investigation. An anomaly is an observable or detectable fact, or data representing such fact. An anomaly or a set of anomalies may be evaluated together and may result in a determination of a threat indicator or a threat. A threat is an interpretation of one or more anomalies and/or threat indicators. Threat indicators and threats are escalations of events of concern. As an example of scale, hundreds of millions of packets of incoming event data from various data sources may be analyzed to yield 100 anomalies, which may be further analyzed to yield 10 threat indicators, which may again be further analyzed to yield one or two threats. This manner of data scaling is one of the reasons the security platform can provide anomaly and threat detection in a real-time manner.

In the context of machine-learning evaluation, historical data and third-party data may be used to create and improve the machine learning models employed to perform the evaluation; however, the amount of such historical data and/or third-party data can be potentially much larger than the real-time data stream. As such, the actual evaluation of the historical data tends to be slower. Consequently, in certain embodiments, the real-time processing path does not use either or both the historical data and third-party data as inputs. In other embodiments, historical and third-party data may be used as inputs but the majority of the data used for evaluation in the real-time processing path still pertains to contemporaneous incoming event data. This is a consequence of the need to process the voluminous incoming event data quickly to obtain actionable threat information to prevent imminent harm.

The anomalies and threats detected by the real-time processing path may be employed to automatically trigger an action, such as stopping the intrusion, shutting down network access, locking out users, preventing information theft or information transfer, shutting down software and or hardware processes, and the like. In certain embodiments, the discovered anomalies and threats may be presented to a network operator (e.g., a network security administrator or analyst) for decision. As an alternative or in addition to automatically taking action based on the discovered anomalies and threats, the decisions by the user (e.g., that the anomalies and threats are correctly diagnosed, or that the discovered anomalies and threats are false positives) can then be provided as feedback data in order to update and improve the models.

In the batch processing path, historical data and third-party data are processed, optionally with the incoming real-time event data, to uncover, for example, more subtle anomalies and threats than the real-time processing path can uncover because of the real-time processing path's responsive time constraints. Batch processing may occur synchronously with real-time processing or in accordance with a predefined schedule.

Historical data represents past events and may include data from different instantiations of the real-time evaluators deployed in different locations in the network. The historical data may span time and geography. In some implementations, only an abridged version of the incoming event data is analyzed in the real-time processing path while a more complete version is stored as historical data. Thus, the historical data may, in one or more of these implementations, include event data that has more attributes than the abridged event data presented to the real-time processing path for evaluation.

As in the real-time data path, anomalies, threat indicators and threats discovered by the batch analyzer may be actionable automatically or may be presented to a human operator for decision on whether to take action. The action taken by the operator to validate or invalidate the conclusions reached by the batch analyzer may serve as a source of feedback to the security platform to improve its evaluation of subsequently processed data.

FIG. 2 illustrates a high level view of an example security platform 102 . In FIG. 2 , a cloud computing infrastructure is shown, represented in part by a virtualization layer 104 . Various cloud computing operating systems or platforms, such as OpenStack™, VMware™, Amazon Web Services™, or Google Cloud™ may be employed in virtualization layer 104 to create public clouds or private clouds. Generally speaking, these cloud computing operating systems and others permit processing and storage to be implemented on top of a set of shared resources. Among its many advantages, cloud computing per

CLAIMS

Claims ( 18 )

What is claimed is:

1. A method comprising:

receiving, by a computer system, event data associated with an entity on a computer network;

analyzing, by the computer system, the event data;

generating, by the computer system, a plurality of feature scores for the entity based on a result of analyzing the event data;

creating, by the computer system, an entity profile uniquely associated with the entity, the entity profile including the plurality of feature scores for the entity;

accessing the entity profile to read the plurality of features scores for the entity;

processing the plurality of feature scores for the entity, accessed from the entity profile, by using a plurality of machine-learning models;

generating a plurality of intermediate anomaly scores for the entity, each based on processing of a respective one of the plurality of feature scores of the entity using a respective one of the plurality of machine-learning models;

processing the plurality of intermediate anomaly scores for the entity according to an ensemble learning model;

generating an anomaly score for the entity based on processing the plurality of intermediate anomaly scores for the entity according to the ensemble learning model; and

detecting an anomaly associated with the entity in response to determining that the anomaly score for the entity satisfies a specified criterion.

2. The method of claim 1 , wherein the detected anomaly is indicative of a malware communication.

3. The method of claim 1 , wherein each of the plurality of feature scores is representative of a quantified evaluation of risk associated with the entity.

4. The method of claim 1 , wherein detecting the anomaly includes assigning the anomaly score based on a weighted combination of the plurality of feature scores.

5. The method of claim 1 , wherein generating a feature score of the plurality of feature scores includes:

processing the event data using a machine learning model, the machine learning model including:

model processing logic defining a process for assigning the feature score based on the event data; and

a model state defining a set of parameters for applying the model processing logic;

wherein the anomaly is detected in response to determining that the anomaly score satisfies the specified criterion.

6. The method of claim 1 , wherein detecting the anomaly includes:

determining a volume of event data associated with a communication between the entity and another entity;

using the ensemble-learning model if the volume of event data is determined to be at or above a threshold volume.

7. The method of claim 1 , wherein the event data is associated with a communication between an internal entity within a computer network and an external entity outside the computer network.

8. The method of claim 1 , wherein the event data includes an identifier associated with the entity, and wherein at least one feature score of the plurality of feature scores is indicative of a level of confidence that the identifier is machine generated.

9. The method of claim 1 , further comprising:

annotating, by the computer system, the detected anomaly with data from an external data source external to the computer network.

10. The method of claim 1 , further comprising:

outputting, by the computer system, via a user interface, an indication of the detected anomaly to a user.

11. The method of claim 1 , wherein the event data is timestamped machine data.

12. The method of claim 1 , wherein the event data include one or more of: domain name system (DNS) generated log data, firewall generated log data, or proxy generated log data.

13. The method of claim 1 , wherein detecting the anomaly includes processing the entity profile using an anomaly model.

14. A system comprising:

a processor; and

a memory having instructions stored therein, execution of which by the processor causes the system to:

receive event data associated with an entity on a computer network;

analyze the event data;

generate a plurality of feature scores for the entity based on a result of analyzing the event data;

creating, by the computer system, an entity profile uniquely associated with the entity, the entity profile including the plurality of feature scores for the entity;

accessing the entity profile to read the plurality of features scores for the entity;

processing the plurality of feature scores for the entity, accessed from the entity profile, by using a plurality of machine-learning models;

generating a plurality of intermediate anomaly scores for the entity, each based on processing of a respective one of the plurality of feature scores for the entity using a respective one of the plurality of machine-learning models;

processing the plurality of intermediate anomaly scores for the entity according to an ensemble-learning model;

generating an anomaly score for the entity based on processing the plurality of intermediate anomaly scores for the entity according to the ensemble-learning model; and

detecting an anomaly associated with the entity in response to determining that the anomaly score for the entity satisfies a specified criterion.

15. The system of claim 14 , wherein the detected anomaly is indicative of a malware communication.

16. The system of claim 14 , wherein each of the plurality of feature scores is representative of a quantified evaluation of risk associated with the particular entity.

17. A non-transitory machine-readable storage medium containing instructions, execution of which by a computer system causes the computer system to perform operations comprising:

receiving event data associated with an entity on a computer network;

analyzing the event data;

generating a plurality of feature scores for the entity based on a result of analyzing the event data;

creating, by the computer system, an entity profile uniquely associated with the entity, the entity profile including the plurality of feature scores for the entity;

accessing the entity profile to read the plurality of features scores for the entity;

processing the plurality of feature scores for the entity, accessed from the entity profile, by using a plurality of machine-learning models;

generating a plurality of intermediate anomaly scores for the entity, each based on processing of a respective one of the plurality of feature scores for the entity using a respective one of the plurality of machine-learning models;

processing the plurality of intermediate anomaly scores for the entity according to an ensemble-learning model;

generating an anomaly score for the entity based on processing the plurality of intermediate anomaly scores for the entity according to the ensemble-learning model; and

detecting an anomaly associated with the entity in response to determining that the anomaly score for the entity satisfies a specified criterion.

18. The non-transitory machine-readable storage medium of claim 17 , such that detecting the anomaly includes assigning the anomaly score based on a weighted combination of the plurality of feature scores.

US17/676,022

2015-08-31

2022-02-18

Anomaly detection based on ensemble machine learning model

Active

2036-01-24

US12438891B1

( en )

Priority Applications (2)

Application Number

Priority Date

Filing Date

Title

US17/676,022

US12438891B1

( en )

2015-08-31

2022-02-18

Anomaly detection based on ensemble machine learning model

US19/350,613

US20260073310A1

( en )

2015-08-31

2025-10-06

Anomaly detection based on ensemble machine learning model

Applications Claiming Priority (4)

Application Number

Priority Date

Filing Date

Title

US201562212541P

2015-08-31

2015-08-31

US14/929,183

US10389738B2

( en )

2015-08-31

2015-10-30

Malware communications detection

US16/503,181

US11258807B2

( en )

2015-08-31

2019-07-03

Anomaly detection based on communication between entities over a network

US17/676,022

US12438891B1

( en )

2015-08-31

2022-02-18

Anomaly detection based on ensemble machine learning model

Related Parent Applications (1)

Application Number

Title

Priority Date

Filing Date

US16/503,181

Continuation

US11258807B2

( en )

2015-08-31

2019-07-03

Anomaly detection based on communication between entities over a network

Related Child Applications (1)

Application Number

Title

Priority Date

Filing Date

US19/350,613

Continuation

US20260073310A1

( en )

2015-08-31

2025-10-06

Anomaly detection based on ensemble machine learning model

Publications (1)

Publication Number

Publication Date

US12438891B1

true

US12438891B1 ( en )

2025-10-07

Family

ID=57400072

Family Applications (52)

Application Number

Title

Priority Date

Filing Date

US14/841,634

Active

2035-11-18

US9699205B2

( en )

2015-08-31

2015-08-31

Network security system

US14/928,985

Active

2036-03-14

US9838410B2

( en )

2015-08-31

2015-10-30

Identity resolution in data intake stage of machine data processing platform

US14/928,471

Active

2036-02-07

US10469508B2

( en )

2015-08-31

2015-10-30

Interactive threat geo-map for monitoring computer network security

US14/929,168

Active

US9516053B1

( en )

2015-08-31

2015-10-30

Network security threat detection by user/user-entity behavioral analysis

US14/929,182

Active

2036-08-24

US10003605B2

( en )

2015-08-31

2015-10-30

Detection of clustering in graphs in network security analysis

US14/929,183

Active

2036-01-30

US10389738B2

( en )

2015-08-31

2015-10-30

Malware communications detection

US14/929,047

Abandoned

US20170063907A1

( en )

2015-08-31

2015-10-30

Multi-Stage Network Security Threat Detection

US14/929,196

Active

2036-06-11

US10015177B2

( en )

2015-08-31

2015-10-30

Lateral movement detection for network security analysis

US14/929,132

Active

2036-06-03

US10063570B2

( en )

2015-08-31

2015-10-30

Probabilistic suffix trees for network security analysis

US14/929,141

Active

2036-05-24

US10158652B2

( en )

2015-08-31

2015-10-30

Sharing model state between real-time and batch paths in network security anomaly detection

US14/929,042

Active

US9667641B2

( en )

2015-08-31

2015-10-30

Complex event processing of computer network data

US14/929,035

Active

US10110617B2

( en )

2015-08-31

2015-10-30

Modular model workflow in a distributed computation system

US14/929,224

Active

US9591010B1

( en )

2015-08-31

2015-10-30

Dual-path distributed architecture for network security analysis

US14/928,918

Active

US10243970B2

( en )

2015-08-31

2015-10-30

Event views in data intake stage of machine data processing platform

US14/928,563

Active

US9609011B2

( en )

2015-08-31

2015-10-30

Interface having selectable, interactive views for evaluating potential network compromise

US14/928,421

Active

2036-05-28

US10193901B2

( en )

2015-08-31

2015-10-30

Interface providing an interactive timeline for evaluating instances of potential network compromise

US14/929,037

Active

2036-04-03

US10419450B2

( en )

2015-08-31

2015-10-30

Detection of anomalies, threat indicators, and threats to network security

US14/928,535

Active

2035-11-21

US10212174B2

( en )

2015-08-31

2015-10-30

Method and system for reviewing identified threats for performing computer security monitoring

US14/929,203

Active

US9596254B1

( en )

2015-08-31

2015-10-30

Event mini-graphs in data intake stage of machine data processing platform

US14/929,204

Active

2036-06-26

US10038707B2

( en )

2015-08-31

2015-10-30

Rarity analysis in network security anomaly/threat detection

US14/928,503

Abandoned

US20170063900A1

( en )

2015-08-31

2015-10-30

Method And System For Monitoring Entity Activity On An Organization's Computer Network

US14/929,187

Active

2038-07-28

US10904270B2

( en )

2015-08-31

2015-10-30

Enterprise security graph

US14/929,184

Active

2035-12-28

US10069849B2

( en )

2015-08-31

2015-10-30

Machine-generated traffic detection (beaconing)

US14/928,451

Active

2036-07-20

US10154047B2

( en )

2015-08-31

2015-10-30

Method and system for generating a kill chain for monitoring computer network security

US15/335,250

Active

US9609009B2

( en )

2015-08-31

2016-10-26

Network security threat detection by user/user-entity behavioral analysis

US15/413,336

Active

US10135848B2

( en )

2015-08-31

2017-01-23

Network security threat detection using shared variable behavior baseline

US15/415,747

Active

US9813435B2

( en )

2015-08-31

2017-01-25

Network security analysis using real-time and batch detection engines

US15/418,546

Active

US10116670B2

( en )

2015-08-31

2017-01-27

Event specific relationship graph generation and application in a machine data processing platform

US15/490,849

Active

US10148677B2

( en )

2015-08-31

2017-04-18

Model training and deployment in complex event processing of computer network data

US15/616,889

Active

US9900332B2

( en )

2015-08-31

2017-06-07

Network security system with real-time and batch paths

US15/800,000

Active

2036-03-23

US10581881B2

( en )

2015-08-31

2017-10-31

Model workflow control in a distributed computation system

US15/800,010

Active

US10291635B2

( en )

2015-08-31

2017-10-31

Identity resolution in data intake of a distributed data processing system

US15/860,049

Active

US10419462B2

( en )

2015-08-31

2018-01-02

Event information access interface in data intake stage of a distributed data processing system

US15/926,961

Active

US10419463B2

( en )

2015-08-31

2018-03-20

Event specific entity relationship discovery in data intake stage of a distributed data processing system

US15/995,073

Active

US10476898B2

( en )

2015-08-31

2018-05-31

Lateral movement detection for network security analysis

US16/016,472

Active

2036-12-26

US11470096B2

( en )

2015-08-31

2018-06-22

Network security anomaly and threat detection using rarity scoring

US16/041,637

Active

US10560468B2

( en )

2015-08-31

2018-07-20

Window-based rarity determination using probabilistic suffix trees for network security analysis

US16/050,368

Active

US10587633B2

( en )

2015-08-31

2018-07-31

Anomaly detection based on connection requests in network traffic

US16/182,469

Active

US10419465B2

( en )

2015-08-31

2018-11-06

Data retrieval in security anomaly detection platform with shared model state between real-time and batch paths

US16/215,350

Active

US10778703B2

( en )

2015-08-31

2018-12-10

Method and system for generating an interactive kill chain view for training a machine learning model for identifying threats

US16/259,999

Active

US10666668B2

( en )

2015-08-31

2019-01-28

Interface providing an interactive trendline for a detected threat to facilitate evaluation for false positives

US16/264,514

Active

US10986106B2

( en )

2015-08-31

2019-01-31

Method and system for generating an entities view with risk-level scoring for performing computer security monitoring

US16/503,181

Active

2036-10-25

US11258807B2

( en )

2015-08-31

2019-07-03

Anomaly detection based on communication between entities over a network

US16/516,471

Active

2036-08-07

US11411966B2

( en )

2015-08-31

2019-07-19

Processing anomaly data to identify threats to network security

US16/532,312

Active

2036-02-01

US11146574B2

( en )

2015-08-31

2019-08-05

Annotation of event data to include access interface identifiers for use by downstream entities in a distributed data processing system

US16/547,235

Active

US10911468B2

( en )

2015-08-31

2019-08-21

Sharing of machine learning model state between batch and real-time processing paths for detection of network security issues

US16/568,106

Active

US10798113B2

( en )

2015-08-31

2019-09-11

Interactive geographic representation of network security threats

US16/581,094

Active

US10911470B2

( en )

2015-08-31

2019-09-24

Detecting anomalies in a computer network based on usage similarity scores

US17/125,130

Active

2036-04-02

US11575693B1

( en )

2015-08-31

2020-12-17

Composite relationship graph for network security

US17/676,022

Active

2036-01-24

US12438891B1

( en )

2015-08-31

2022-02-18

Anomaly detection based on ensemble machine learning model

US17/845,383

Active

US11824646B1

( en )

2015-08-31

2022-06-21

Processing anomaly data to identify network security threats by use of rarity analysis

US19/350,613

Pending

US20260073310A1

( en )

2015-08-31

2025-10-06

Anomaly detection based on ensemble machine learning model

Family Applications Before (49)

Application Number

Title

Priority Date

Filing Date

US14/841,634

Active

2035-11-18

US9699205B2

( en )

2015-08-31

2015-08-31

Network security system

US14/928,985

Active

2036-03-14

US9838410B2

( en )

2015-08-31

2015-10-30

Identity resolution in data intake stage of machine data processing platform

US14/928,471

Active

2036-02-07

US10469508B2

( en )

2015-08-31

2015-10-30

Interactive threat geo-map for monitoring computer network security

US14/929,168

Active

US9516053B1

( en )

2015-08-31

2015-10-30

Network security threat detection by user/user-entity behavioral analysis

US14/929,182

Active

2036-08-24

US10003605B2

( en )

2015-08-31

2015-10-30

Detection of clustering in graphs in network security analysis

US14/929,183

Active

2036-01-30

US10389738B2

( en )

2015-08-31

2015-10-30

Malware communications detection

US14/929,047

Abandoned

US20170063907A1

( en )

2015-08-31

2015-10-30

Multi-Stage Network Security Threat Detection

US14/929,196

Active

2036-06-11

US10015177B2

( en )

2015-08-31

2015-10-30

Lateral movement detection for network security analysis

US14/929,132

Active

2036-06-03

US10063570B2

( en )

2015-08-31

2015-10-30

Probabilistic suffix trees for network security analysis

US14/929,141

Active

2036-05-24

US10158652B2

( en )

2015-08-31

2015-10-30

Sharing model state between real-time and batch paths in network security anomaly detection

US14/929,042

Active

US9667641B2

( en )

2015-08-31

2015-10-30

Complex event processing of computer network data

US14/929,035

Active

US10110617B2

( en )

2015-08-31

2015-10-30

Modular model workflow in a distributed computation system

US14/929,224

Active

US9591010B1

( en )

2015-08-31

2015-10-30

Dual-path distributed architecture for network security analysis

US14/928,918

Active

US10243970B2

( en )

2015-08-31

2015-10-30

Event views in data intake stage of machine data processing platform

US14/928,563

Active

US9609011B2

( en )

2015-08-31

2015-10-30

Interface having selectable, interactive views for evaluating potential network compromise

US14/928,421

Active

2036-05-28

US10193901B2

( en )

2015-08-31

2015-10-30

Interface providing an interactive timeline for evaluating instances of potential network compromise

US14/929,037

Active

2036-04-03

US10419450B2

( en )

2015-08-31

2015-10-30

Detection of anomalies, threat indicators, and threats to network security

US14/928,535

Active

2035-11-21

US10212174B2

( en )

2015-08-31

2015-10-30

Method and system for reviewing identified threats for performing computer security monitoring

US14/929,203

Active

US9596254B1

( en )

2015-08-31

2015-10-30

Event mini-graphs in data intake stage of machine data processing platform

US14/929,204

Active

2036-06-26

US10038707B2

( en )

2015-08-31

2015-10-30

Rarity analysis in network security anomaly/threat detection

US14/928,503

Abandoned

US20170063900A1

( en )

2015-08-31

2015-10-30

Method And System For Monitoring Entity Activity On An Organization's Computer Network

US14/929,187

Active

2038-07-28

US10904270B2

( en )

2015-08-31

2015-10-30

Enterprise security graph

US14/929,184

Active

2035-12-28

US10069849B2

( en )

2015-08-31

2015-10-30

Machine-generated traffic detection (beaconing)

US14/928,451

Active

2036-07-20

US10154047B2

( en )

2015-08-31

2015-10-30

Method and system for generating a kill chain for monitoring computer network security

US15/335,250

Active

US9609009B2

( en )

2015-08-31

2016-10-26

Network security threat detection by user/user-entity behavioral analysis

US15/413,336

Active

US10135848B2

( en )

2015-08-31

2017-01-23

Network security threat detection using shared variable behavior baseline

US15/415,747

Active

US9813435B2

( en )

2015-08-31

2017-01-25

Network security analysis using real-time and batch detection engines

US15/418,546

Active

US10116670B2

( en )

2015-08-31

2017-01-27

Event specific relationship graph generation and application in a machine data processing platform

US15/490,849

Active

US10148677B2

( en )

2015-08-31

2017-04-18

Model training and deployment in complex event processing of computer network data

US15/616,889

Active

US9900332B2

( en )

2015-08-31

2017-06-07

Network security system with real-time and batch paths

US15/800,000

Active

2036-03-23

US10581881B2

( en )

2015-08-31

2017-10-31

Model workflow control in a distributed computation system

US15/800,010

Active

US10291635B2

( en )

2015-08-31

2017-10-31

Identity resolution in data intake of a distributed data processing system

US15/860,049

Active

US10419462B2

( en )

2015-08-31

2018-01-02

Event information access interface in data intake stage of a distributed data processing system

US15/926,961

Active

US10419463B2

( en )

2015-08-31

2018-03-20

Event specific entity relationship discovery in data intake stage of a distributed data processing system

US15/995,073

Active

US10476898B2

( en )

2015-08-31

2018-05-31

Lateral movement detection for network security analysis

US16/016,472

Active

2036-12-26

US11470096B2

( en )

2015-08-31

2018-06-22

Network security anomaly and threat detection using rarity scoring

US16/041,637

Active

US10560468B2

( en )

2015-08-31

2018-07-20

Window-based rarity determination using probabilistic suffix trees for network security analysis

US16/050,368

Active

US10587633B2

( en )

2015-08-31

2018-07-31

Anomaly detection based on connection requests in network traffic

US16/182,469

Active

US10419465B2

( en )

2015-08-31

2018-11-06

Data retrieval in security anomaly detection platform with shared model state between real-time and batch paths

US16/215,350

Active

US10778703B2

( en )

2015-08-31

2018-12-10

Method and system for generating an interactive kill chain view for training a machine learning model for identifying threats

US16/259,999

Active

US10666668B2

( en )

2015-08-31

2019-01-28

Interface providing an interactive trendline for a detected threat to facilitate evaluation for false positives

US16/264,514

Active

US10986106B2

( en )

2015-08-31

2019-01-31

Method and system for generating an entities view with risk-level scoring for performing computer security monitoring

US16/503,181

Active

2036-10-25

US11258807B2

( en )

2015-08-31

2019-07-03

Anomaly detection based on communication between entities over a network

US16/516,471

Active

2036-08-07

US11411966B2

( en )

2015-08-31

2019-07-19

Processing anomaly data to identify threats to network security

US16/532,312

Active

2036-02-01

US11146574B2

( en )

2015-08-31

2019-08-05

Annotation of event data to include access interface identifiers for use by downstream entities in a distributed data processing system

US16/547,235

Active

US10911468B2

( en )

2015-08-31

2019-08-21

Sharing of machine learning model state between batch and real-time processing paths for detection of network security issues

US16/568,106

Active

US10798113B2

( en )

2015-08-31

2019-09-11

Interactive geographic representation of network security threats

US16/581,094

Active

US10911470B2

( en )

2015-08-31

2019-09-24

Detecting anomalies in a computer network based on usage similarity scores

US17/125,130

Active

2036-04-02

US11575693B1

( en )

2015-08-31

2020-12-17

Composite relationship graph for network security

Family Applications After (2)

Application Number

Title

Priority Date

Filing Date

US17/845,383

Active

US11824646B1

( en )

2015-08-31

2022-06-21

Processing anomaly data to identify network security threats by use of rarity analysis

US19/350,613

Pending

US20260073310A1

( en )

2015-08-31

2025-10-06

Anomaly detection based on ensemble machine learning model

Country Status (1)

Country

Link

US

( 52 )

US9699205B2

( en )

Families Citing this family (2457)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US8732004B1

( en )

2004-09-22

2014-05-20

Experian Information Solutions, Inc.

Automated analysis of data to generate prospect notifications based on trigger events

US8510596B1

( en )

2006-02-09

2013-08-13

Virsec Systems, Inc.

System and methods for run time detection and correction of memory corruption

US9690820B1

( en )

2007-09-27

2017-06-27

Experian Information Solutions, Inc.

Database system for triggering event notifications based on updates to database records

WO2009051766A1

( en )

*

2007-10-15

2009-04-23

23Andme, Inc.

Family inheritance

US9990674B1

( en )

2007-12-14

2018-06-05

Consumerinfo.Com, Inc.

Card registry systems and methods

US8312033B1

( en )

2008-06-26

2012-11-13

Experian Marketing Solutions, Inc.

Systems and methods for providing an integrated identifier

US20100131513A1

( en )

2008-10-23

2010-05-27

Lundberg Steven W

Patent mapping

US20100174638A1

( en )

2009-01-06

2010-07-08

ConsumerInfo.com

Report existence monitoring

US10805331B2

( en )

2010-09-24

2020-10-13

BitSight Technologies, Inc.

Information technology security assessment system

US10268731B2

( en )

*

2011-10-03

2019-04-23

Black Hills Ip Holdings, Llc

Patent mapping

US8738516B1

( en )

2011-10-13

2014-05-27

Consumerinfo.Com, Inc.

Debt services candidate locator

US10600002B2

( en )

2016-08-04

2020-03-24

Loom Systems LTD.

Machine learning techniques for providing enriched root causes based on machine-generated data

US10740692B2

( en )

2017-10-17

2020-08-11

Servicenow, Inc.

Machine-learning and deep-learning techniques for predictive ticketing in information technology systems

US11416325B2

( en )

2012-03-13

2022-08-16

Servicenow, Inc.

Machine-learning and deep-learning techniques for predictive ticketing in information technology systems

<tr itemprop="forwardReferencesFamily" i

Related documents

• Generating augmented reality experiences utilizing physical objects to … — Meta Platforms Technologies, Llc (US20260037214A1)#653960
• Virtual reality system using multiple force arrays for a solver — DeepMotion, Inc. (US10657696B2)#653961
• Tracking position of device inside-out for virtual reality interactivity — Sony Interactive Entertainment America Llc (US10535153B2)#653962
• Augmented reality system with spatialized audio tied to user manipulated … — Magic Leap, Inc. (US11231770B2)#653963
• Big data-based autonomous flight drone system and autonomous flight method … — Rgblab Co., Ltd. (US20210390867A1)#653964