CN111466103B - Method and system for generation and adaptation of network baselines - Google Patents

Abstract

Systems, methods, apparatus and computer program products for generation and adaptation of network baselines are provided. A method may include: generating forecast values for one or more network metrics for a future time period; generating a baseline for the network metric(s) using the forecast values and/or historical data; using at least one time series analysis technique to The network metric(s) are evaluated to detect changes in network conditions, and the baseline is adapted to the detected changes in network conditions using historical data, machine learning and/or time series analysis techniques.

Description

Method and system for generation and adaptation of network baselines

technical field

Certain embodiments may generally relate to wired or wireless communication networks, including but not limited to local or wide area networks, such as the Internet. For example, some embodiments may relate to wireless or cellular communication systems in general, such as, but not limited to, Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (UTRAN), Long Term Evolution (LTE) Evolved UTRAN (E-UTRAN), Advanced LTE (LTE-A), LTE-A Pro and/or fifth generation (5G) or new radio (NR) access technologies. For example, various embodiments may relate to the generation and adaptation of network baselines for such communication networks.

Background technique

A Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (UTRAN) refers to a communication network comprising base stations or Node Bs and eg a Radio Network Controller (RNC). UTRAN allows connectivity between user equipment (UE) and the core network. The RNC provides control functions for one or more Node Bs. The RNC and its corresponding Node Bs are called the Radio Network Subsystem (RNS). In the case of E-UTRAN (Evolved UTRAN), the air interface design, protocol architecture and multiple access principles are more novel than those of UTRAN, and there is no RNC and the radio access function is performed by an evolved Node B (eNodeB or eNB) or Many eNBs provide. For example, in the case of Coordinated Multipoint (CoMP) and Dual Connectivity (DC), a single UE connection involves multiple eNBs.

Long Term Evolution (LTE), or E-UTRAN, improves efficiency and services, offers lower costs, and opens up new spectrum opportunities compared to previous generations. In particular, LTE is a 3GPP standard that provides an uplink peak rate of, for example, at least 75 megabits per second (Mbps) per carrier and a downlink peak rate of, for example, at least 300 Mbps per carrier. LTE supports scalable carrier bandwidth from 20 MHz to 1.4 MHz, and supports both Frequency Division Duplex (FDD) and Time Division Duplex (TDD). Carrier aggregation, or the aforementioned dual connectivity, also allows simultaneous operation on multiple component carriers, thus multiplying performance such as data rates per user.

As mentioned above, LTE can also increase spectral efficiency in the network, allowing carriers to provide more data and voice services on a given bandwidth. Therefore, in addition to high-capacity voice support, LTE is also designed to meet the needs of high-speed data and media transmission. Advantages of LTE include, for example, high throughput, low latency, FDD and TDD support in the same platform, improved end-user experience, and simple architecture, which can reduce operating costs.

Certain additional releases of 3GPP LTE (eg, LTE Rel-10, LTE Rel-11 ) target the International Mobile Telecommunications-Advanced (IMT-A) system, which is simply referred to herein as LTE-Advanced (LTE-A) for convenience.

LTE-A is aimed at extending and optimizing the 3GPP LTE radio access technology. The goal of LTE-A is to provide significantly enhanced services at reduced cost through higher data rates and lower latency. LTE-A is a more optimized radio system that can meet the International Telecommunication Union Radio (ITU-R) requirements for IMT-Advanced while maintaining backward compatibility. One of the key features of LTE-A, introduced in LTE Rel-10, is carrier aggregation, which allows increased data rates through the aggregation of two or more LTE carriers. The next releases of 3GPP LTE (eg, LTE Rel-12, LTE Rel-13, LTE Rel-14, LTE Rel-15) aim to further improve dedicated services, reduce latency, and meet requirements close to 5G.

Fifth Generation (5G) or New Radio (NR) wireless systems refer to Next Generation (NG) radio systems and network architectures. 5G is also known as the IMT-2020 system. It is estimated that 5G will provide bit rates of about 10-20Gbit/s or higher. 5G will support at least enhanced mobile broadband (eMBB) and ultra-reliable low-latency communications (URLLC). 5G is also expected to increase network scalability to hundreds of thousands of connections. The signaling technology for 5G is expected to have greater coverage as well as spectral and signaling efficiencies. 5G is expected to deliver extreme broadband and ultra-robust low-latency connections as well as massive networks to support the Internet of Things (IoT). With the growing popularity of IoT and machine-to-machine (M2M) communications, there will be an increasing need for networks that meet the demands of low power consumption, low data rates, and long battery life. Note that in 5G or NR, a Node B or eNB may be referred to as a Next Generation or 5G Node B (gNB).

The evolution of communication systems to 5G will make networks larger and more complex. In addition, 5G networks are expected to support various verticals such as smart grids, smart cities, connected cars, eHealth, etc., which will make communication systems an even more essential and important part of society than current systems. Therefore, ensuring high reliability and availability of such society-critical infrastructure will be of paramount importance. Furthermore, the network must be efficiently operated and managed in order to support the performance requirements required to deliver the expected data rates.

Contents of the invention

One embodiment relates to a method that may include generating a predicted value of at least one network metric for a future time period. The generating may include generating forecast values using at least one of historical data, machine learning, or time series analysis techniques. The method may also include: generating a baseline for at least one network metric using at least one of predicted values or historical data; evaluating at least one network metric using at least one time series analysis technique to detect changes in network conditions, and using historical At least one of data, machine learning, or time series analysis techniques adapts the baseline to detected changes in network conditions.

In one embodiment, the method may further comprise determining variability based on predicted values of at least one network metric over a future time period using at least one time series analysis technique. In some embodiments, the adaptation of the baseline may include periodically determining a trend of at least one network metric and modifying the baseline when a change in trend is detected.

According to some embodiments, the baseline may include an upper limit and a lower limit, and the method may further comprise comparing the currently observed value of at least one network metric with the baseline, and when the currently observed value of the at least one network metric is above the upper limit of the baseline or within the baseline Violations are detected when the lower limit of .

In some embodiments, the method may include storing each past detected violation of the baseline as a record in a knowledge base, where each record in the knowledge base may include the time period when the violation occurred, the duration of the violation, and/or violated to the maximum extent. According to one embodiment, when a violation is detected, the method may include checking the knowledge base to determine whether there is a violation similar to the detected violation stored in the knowledge base. When no similar violations are found in the knowledge base, the method may include generating an alert and creating a record for the detected violation in the knowledge base. When a similar violation is found in the knowledge base, the method may include monitoring at least one network metric that caused the violation to follow the records of the similar violation to confirm that the detected violation does not indicate an anomaly in the network.

According to some embodiments, when an alert is generated, the method may further include: sending the alert to a network administrator; receiving a response indicating whether the violation that triggered the alert represents an anomaly in the network; and updating the knowledge base to store the response in violation records. In one embodiment, the method may further comprise assigning at least one network metric to a cluster according to the alignment between the metrics, and grouping records in the knowledge base based on the cluster.

In one embodiment, the method may include storing the predicted value in memory for use with future predictions of the at least one network metric. In some embodiments, generating the predicted value may include periodically generating the predicted value, and evaluating the at least one network metric may further include continuously evaluating the at least one network metric to detect changes in network conditions.

Another embodiment relates to an apparatus, which may comprise generating means for generating a predicted value of at least one network metric in a future time period. The generating means may include means for generating predicted values using at least one of historical data, machine learning, or time series analysis techniques. The apparatus may also include generating means for generating a baseline for at least one network metric using at least one of predicted values or historical data; for evaluating at least one network metric using at least one time series analysis technique to detect network evaluation means for changes in conditions; and adaptation means for adapting the baseline to the detected changes in network conditions using at least one of historical data, machine learning, or time series analysis techniques.

Another embodiment relates to an apparatus, which may comprise at least one processor and at least one memory comprising computer program code. At least one memory and computer program code may be configured to, with at least one processor, cause the apparatus to at least: generate a predicted value of at least one network metric for a future time period, e.g., using historical data, machine learning, or time series analysis techniques using at least one of predicted values or historical data to generate a baseline for at least one network metric; using at least one time series analysis technique to evaluate at least one network metric to detect changes in network conditions; and using historical At least one of data, machine learning, or time series analysis techniques adapts the baseline to detected changes in network conditions.

Another embodiment relates to a computer program embodied on a non-transitory computer readable medium. The computer program, when executed by a processor, may cause the processor to perform a process comprising: generating a predicted value of at least one network metric for a future time period, for example, using at least one of historical data, machine learning, or time series analysis techniques A; generating a baseline for at least one network metric using at least one of predicted values or historical data; evaluating at least one network metric using at least one time series analysis technique to detect changes in network conditions; and using historical data, machine At least one of learning or time series analysis techniques to adapt the baseline to detected changes in network conditions.

Description of drawings

For a proper understanding of the invention, reference should be made to the accompanying drawings, in which:

Figure 1 shows an example block diagram of a system according to one embodiment;

Figure 2 shows a block diagram of a process according to one embodiment;

Figure 3 shows a diagram depicting an example of baseline violation modeling, according to one embodiment;

Figure 4a shows a flow chart of a method according to one embodiment;

Figure 4b shows a flow chart of a method according to another embodiment; and

Fig. 5 shows a block diagram of an apparatus according to one embodiment.

Detailed ways

It will be readily understood that the components of the present invention, as generally described and illustrated in the drawings herein, may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of embodiments of systems, methods, apparatus, and computer program products for generation and adaptation of network baselines, as represented in the figures and described below, is not intended to limit the scope of the present invention , but instead represent selected embodiments of the invention.

The features, structures or characteristics of the invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, use of the phrase "certain embodiments," "some embodiments," or other similar language throughout this specification refers to the fact that a particular feature, structure, or characteristic described in connection with the embodiments may be included within the scope of the present invention. In at least one embodiment. Thus, appearances of the phrase "in some embodiments," "in some embodiments," "in other embodiments," or other similar language throughout this specification are not necessarily all referring to the same set of embodiments, and all The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Additionally, different functions or steps discussed below may be performed in a different order and/or concurrently with each other, if desired. Furthermore, one or more of the described functions or steps may be optional or combined, if desired. As such, the following description should be considered as illustrative of the principles, teachings and embodiments of this invention, and not in limitation thereof.

Ensuring the necessary levels of reliability, availability, and operational efficiency requires the ability to monitor network systems in real time. This may be necessary to detect or predict when network performance deviates from a set baseline to perform preventive or corrective action. State-of-the-art monitoring applications require network administrators and/or experts to provide or determine a network baseline that can be used as a basis for determining anomalous behavior. A network baseline may refer to, for example, network performance thresholds that may be used as a basis for determining abnormal network behavior. In other words, a network is considered to be behaving abnormally when its performance deviates too far from the baseline.

One of the biggest challenges in building a monitoring solution is the manual process of establishing a baseline for these networks. This process usually involves using historical data, i.e., monitoring the value of each metric (such as bandwidth, throughput, latency, attachment success rate, handover failure rate, CPU or memory usage, error rate, jitter, etc.) over a given time To determine observed maximum and/or minimum values, these values are then used as upper and/or lower limits for the baseline of the metric. However, current methods for determining and setting network baselines are static and not scalable.

The effort required to determine and set such baselines for each monitored metric in a sizable network is enormous. As networks become more complex and require real-time operations, such actions require a significant amount of network administrator time. As a result, the network is typically only baselined at network element installation time and never tuned (i.e. static), or in some cases never baselined for some metrics and instead vendor defaults are used (such as "CPU utilization should not exceed 60%") and must not change throughout the lifetime of the component. If you set the baseline very high, you risk reliability and availability, and if you set the value very low, you risk operational efficiency (in terms of low resource utilization and multiple false alarms) .

When the monitored value for a given metric deviates from a set baseline, the monitoring application typically generates an alert, which is responded to by the network administrator. Many alerts generated by monitoring applications are false alerts because they do not necessarily indicate actual anomalies in system operation. As networks continue to increase in size and complexity to support IoT, network administrators can no longer respond to every alert generated by monitoring applications. This dictates that either the number of monitored devices and/or metrics should be limited, or the baseline must be set wide enough to reduce the number of false positives per metric. However, both options pose a threat to reliability, availability and operational efficiency. Accordingly, embodiments discussed herein provide a mechanism for a monitoring node or application to learn from generated alerts so that over time the number of false alerts can be reduced without negatively impacting reliability and efficiency.

Accordingly, certain embodiments relate to a method and system for automatically generating and adapting network baselines to changes in network conditions and learning from historical baseline violations to optimize the number of alerts and limit network management member's burden. In one embodiment, a method and system for network baseline generation in which data is collected from one or more network elements and metrics is provided. For each metric, the collected data may be used by a machine learning or analytics algorithm to generate a predicted value of the metric for a given future time period. According to one embodiment, time series analysis techniques may be used to determine the variability of a metric over a future time period. The predicted value and variability can then be used to generate an initial baseline (eg, upper and lower bounds) for the metric over a defined future period. In some embodiments, this process may be applied or repeated for all metrics in the network.

As used herein, according to some embodiments, a network element may be any logical or physical telecommunications entity. These may include physical and/or virtual network entities such as radio network controllers, base stations, access points, serving gateways, mobility management entities, routers or switches, and the like. For each element, embodiments may monitor and collect one or more metrics such as bandwidth, throughput, latency, attachment success rate, handoff failure rate, CPU or memory usage, error rate, jitter, and the like. Assuming that the monitored network(s) consists of E elements and M metrics on all elements, some embodiments may obtain the value x ^t _me of the metric mεM of the element eεE at time t. During the time period T, the collected data can be represented by the time series given in the following formula:

According to an embodiment, the upper limit threshold L and the lower limit threshold U of the baseline within the time period T'≥T can be determined according to the following formula:

In one embodiment, for example, T' may start at the end of T and last for one week (168 hours). However, in other embodiments, the time period T' may be one hour, one day, several days or weeks, or any other suitable time period.

Additional embodiments may provide a method and system for periodically adapting the generated baseline(s) (ie, upper and lower thresholds) to changes in network conditions. To this end, for each metric, an embodiment may include periodically determining an overall trend (eg, upward, downward, or flat) of the observed values of the metric over a past period of time. The observed trend of the metric can be determined using one or more time series analysis methods, such as decomposition. In one embodiment, each determined trend may be stored and compared to previously determined trends to determine whether the observed overall trend has changed. According to some embodiments, when a trend change is detected, the baseline generation method discussed above may be used to regenerate the baseline using the most recent metric observations.

Some embodiments may also include learning from baseline violations. For example, an embodiment may also incorporate a method for learning from previously generated alerts to guide future alert generation. According to this embodiment, several similar measures (in terms of magnitude and character of heave and flow) can be clustered. This metric clustering ensures that metrics in the same cluster are treated as one metric when an alert is generated, and thus ensures that only one alert is raised for all metrics in a cluster for alerts that are considered similar.

Additional embodiments may provide a method and system for determining whether a given alert is similar to previous alerts by incorporating a feedback loop from a network administrator or expert to the monitoring system. According to this embodiment, whenever an alert is generated, the expert may provide feedback to the system, for example in the form of "yes" or "no", as to whether a given alert is a true indication of an operational anomaly. By incorporating such feedback into the knowledge base, the monitoring system can learn from all generated alerts such that for a metric in a given cluster, similar alerts are not repeated, and even for the same Alerts with similar properties are raised more than once when flagged as false.

FIG. 1 shows an example block diagram of a system 100 according to one embodiment. As shown in the example of FIG. 1 , system 100 may include monitoring application 110 configured to collect data about one or more metrics from network element 120 . In one embodiment, monitoring application 110 may also be configured to generate and/or send an alert to expert 130 when observed values of the collected data are outside a set baseline. Expert 130 may interact with monitoring application 110 by providing feedback on the effectiveness of generated alerts. The system 100 may also include a readable and writable storage medium 150 for storing processed metric data and/or several computer programs implementing logic of the methods described herein. In one embodiment, system 100 may include at least one computing medium or processor 140 upon which the logic of the methods described herein runs. Monitored element 120, monitoring application 110, computing medium 140, and storage medium 150 may be physical or virtual elements, and may be separate elements (e.g., connected through an application programming interface (API)), or may reside on the same server superior.

Figure 2 shows a block diagram depicting the steps of a process according to one embodiment. In one embodiment, the process steps of FIG. 2 may be performed by system 100 shown in FIG. 1 , including, for example, monitoring application 110 and computing medium (eg, processor) 140 . As shown in FIG. 2 , this process can be done in at least four steps, including data preprocessing 205 , prediction 210 , decomposition 215 and learning augmentation 220 . Embodiments may also include interaction between the four blocks of FIG. 2 , interaction with network monitoring application 110 configured to collect data from network elements and provide it to data preprocessing 205, and optionally, interaction with the Notifications of generated alerts and expert interaction providing feedback on the alerts.

Data preprocessing 205 may include various data preprocessing techniques, such as data aggregation and normalization. Aggregation is facilitated by understanding that monitored metric values are affected by the traffic carried by the network. Since deterministic changes in traffic may be expected over timeframes greater than one hour (based on practical experience and the notion of "busy hour traffic" in telecommunications), aggregation may include averaging the observations to obtain hourly averages. Aggregation also minimizes random variation in metric values. However, this does not exclude other aggregation timescales.

Since some monitored values (and thus averages) may have very large magnitudes, it is desirable to rescale the input data to be within a uniform range. This may be necessary for many analytical and machine learning methods (such as neural networks) that are sensitive to the size of the input data. A non-limiting example of a normalization method involves transforming observations to a result where the mean is approximately 0 and the standard deviation is in the range close to 1. This ensures that the prediction process focuses on structural similarity/dissimilarity rather than magnitude.

The prediction step 210 may take as input sequential observations covering a given time period (e.g., one week) from the preprocessing step for each metric for a given element, and may output for a given future time period (e.g., one week) Predicted observations for the same metric. In other words, forecast 210 can take observations for the past week and can output a forecast for the next week. To this end, forecasting 210 may include a method (to be generalized over a broad range of metrics) that applies a machine learning based method or algorithm. An example for this purpose can utilize the Long Short-Term Memory (LSTM) structure of neural networks. However, any other machine learning method can be used. Any such algorithm or model can be trained by using historical data. The prediction process 210 may be run periodically and iteratively to make predictions for each metric. In one embodiment, the final output of the prediction step 210 may be post-processed to reverse the normalization that may have been performed in the data preprocessing step 205 . For example, in one embodiment, the final output of the prediction step 210 can be post-processed according to the following formula, where y ^t _me is the predicted value:

The decomposition step 215 can serve two goals, including generating network baselines for each metric, and continuously adapting these baselines to changes in network conditions. To achieve these two goals, Decomposition 215 may decompose the observation of a given metric over a given time period into its trend and variability (or noise) using methods such as time series decomposition.

其中r^t _me、s^t _me和n^t _me分别是趋势、季节和噪声分量。趋势r^t _me可以根据以下公式被确定为阶k+1的中心加权移动平均值，其中k＝168(假定观察时段为一个周期)：To convert predictions into network baselines, a decomposition step 215 may determine the observed variability for any given metric. For this, decomposed observed noise components can be used. Because machine learning predictive methods, such as neural networks, are trained to generalize to map inputs to outputs, their predictions often attempt to remove noise. Therefore, noise can be used as a measure of the normal variation between the metric and the predicted value to create upper and lower bounds. The standard deviation of the decomposed noise (σ _me ) can be determined as the maximum permissible deviation above and below the predicted value, resulting in a baseline for network operation. As an example, the observed value of each metric can be decomposed into three additive components such as

where r ^t _me , s ^t _me and n ^t _me are the trend, seasonal and noise components, respectively. The trend r ^t _me can be determined as a center-weighted moving average of order k+1, where k=168 (assuming the observation period is one period) according to the following formula:

where n = k/2, and the weights _cj of all observations are given by 1/k, except the first and last observations, where weights c ^-n and ^cn are given by 1/2k. In one embodiment, the baseline(s) may be updated whenever the predictive model for a given metric is updated based on the adaptation described below. In one embodiment, the predicted lower bound l ^t _me and upper bound u ^t _me of each metric can be determined according to the following formula:

In order to minimize the time required to generate forecasts and thus ensure real-time forecasts, after each forecast the generated forecast model can be stored in memory for use with future forecasts for the same metric. However, to ensure that the predictive model remains accurate as the network evolves (i.e., taking into account business trends or changes in network topology and components), adaptation can involve actively checking whether the observed trend for a given metric has been since the last predictive model was created has changed (eg, from increasing to decreasing, from constant to increasing, etc.). To this end, in one embodiment, the decomposition step 215 may include periodically (eg, once a week) determining the slope of the trend for each metric. A positive value for the slope indicates an increasing trend, while a negative value indicates a decreasing trend. To determine changes in the observed metric's trend, the decomposition step 215 tracks these slopes over time and compares the current slope to the previous slope. A change in trend is detected when the slope changes from positive to negative (or vice versa), triggering the adaptation of the forecasting model to the most recent data for the metric under consideration. In one embodiment, there may be an additional consideration for adapting the predictive model to a change in trend such that if a detected change in trend results in a violation of the currently set baseline, then if it is shown from an expert or its knowledge base that the violation is not caused by an anomaly , the adaptation is performed. This embodiment aims to avoid recreating the model from anomalous observations.

The system can detect violations when the observed values of any metric exceed the set upper and lower baselines. The purpose of the learning augmentation component 220 is to facilitate a mechanism in which experts augment the predictive baseline by providing feedback as alerts occur so that the system can learn from all alerts it makes. Learning enhancement 220 can then use this information to decide whether to issue future alerts. Considering that many metrics in each network may behave similarly (e.g., the CPU utilization of two nodes may have similar magnitudes and peaks and troughs at the same time, because traffic in the network affects them in the same saw), therefore, learning Enhancement 220 may be configured to ensure that alerts raised for a given metric can be used as a basis for learning to make future decisions about "similar" metrics. To achieve these goals, learning augmentation 220 may consist of three subsystems: (1) knowledge base, (2) alert generation, and (3) metric clustering, as described below.

According to one embodiment, a knowledge base (KB) may contain records of all violations that occurred in the past and/or feedback from experts. The information stored in the KB can be used as a basis for making decisions on whether an alert should be generated after a violation is detected. Figure 3 shows an example of baseline violation modeling showing how violations are converted to entries in the KB. Specifically, the example of FIG. 3 depicts a metric with four violations A, B, C, and D. In this example, each violation can be modeled as a triple with parameters α, β, and γ, where α is the time period over which the violation occurred, β is the duration of the violation, and γ is the maximum extent of the violation. This information can be stored in the KB along with the experts' responses to each metric cluster. In one embodiment, prior to generating any alerts, the KB may be checked to confirm that there are no current violations in the KB for the cluster under consideration. According to some embodiments, a violation may be considered if its α value falls within the same time range (e.g., from 0000 hrs to 0600 hrs), β is less than, and γ is less than the corresponding value of an existing entry for the item in the same cluster in the knowledge base. Similar to another violation.

In one embodiment, for each metric, the system may continuously compare the actual observations to its baseline in order to determine whether the observations represent a violation of an operational boundary, i.e., whether the observed value is above the upper boundary 305 of the set baseline or within the set The lower boundary of the baseline is below 310. When a violation is detected, a decision should be made as to whether an alert is to be raised. Therefore, not all violations of established baselines will raise alarms. To make this determination, one embodiment uses KB. For example, when a violation is detected, it is first checked whether the considered metric has an entry in the KB. If no entry is found, an alert is generated and a new entry is created in the KB to populate available values such as metric id and α, and wait until other values such as β and γ become available to update them. When domain experts (eg, network administrators) receive the alert, they are asked to respond with a "yes" or "no" answer as to whether the violation represents an anomaly. On the other hand, if the given metric already has at least one entry in the KB after the violation, the current violation is evaluated against all existing entries in the KB to establish similarity. According to this embodiment, each of the three parameters currently violated may be compared to the corresponding parameter of each existing entry of the metric. As an example, considering that (∝′, β′, γ′) and (∝″, β″, γ″) are the KB parameters and the current observation for each existing entry, respectively, if for any existing entry, ∝′ =∝" and β'≤β" and γ'≤γ", and its exception column is "No", no alarm is generated.

Many elements in a given network may be affected by network dynamics in a similar manner. For example, Virtual Network Functions (VNFs) in a chain of service functions that process packets in a sequential manner may experience fluctuations in the observation of their metrics (such as CPU and bandwidth) occurring nearly simultaneously. This means that the CPU usage of two VNFs hosted in adjacent nodes will be of similar size and have peaks and troughs at similar times. In this case, when one of the nodes violates a set baseline for a given metric (eg, CPU), there is a high probability that violations will also occur in neighboring nodes. For this reason, an embodiment may also include a method of grouping metrics such that alerts are generated based on clusters of metrics rather than individual metrics. According to this embodiment, each metric can be attached to a cluster (a cluster can have one or more metrics), and the rules in the KB are stored based on the cluster identifier (ID) rather than a single metric ID. Thus, when a violation occurs, a search for entries in the KB can be performed for the cluster in which the current metric is contained.

Clustering metrics can serve at least two purposes. First, since KB entries are grouped, it reduces the number of entries that must be kept in a KB. Second, since a given violation is responded to once for a given metric cluster, it reduces the number of false alarms that can be presented to an expert. To cluster metrics, one embodiment is configured to find alignments between observations among all metric observations over a given time period. This can be done using methods such as time series analysis (such as dynamic time warping) or machine learning (such as k-means or Bayesian inference). Using the alignment between the metrics, the value of the metric that provides the similarity between all the metrics is determined. Two or more measures are clustered together if the similarity distance between them is less than a set constant.

Fig. 4a shows an example process flow diagram of a method for generating and adapting a network baseline according to one embodiment. The method of Figure 4a may be performed by a network element or network node, such as a router, switch, access point, base station, radio network controller, Node B, evolved Node B (eNB), 5G Node B (gNB), next generation node B (NG-NB), WLAN access point, mobility management entity (MME), serving gateway (SGW), application server or subscription server, etc.

As shown in the example of FIG. 4a, the method may include, at 400, calculating or generating predicted values of one or more network metrics in a future time period. Generating 400 may include generating forecast values using historical data, machine learning, and/or time series analysis techniques. According to one embodiment, predictive values may be generated periodically. In one embodiment, the method may include storing the predicted values in memory for use with future predictions of the metric(s).

In one embodiment, the method may further include: at 401, generating a baseline(s) of network metrics using predicted values and/or historical data. A baseline may indicate or include upper and/or lower bounds for a network metric(s), eg, as shown in the example of FIG. 3 discussed above. The method may also include, at 402, evaluating the network metric(s) using, for example, at least one time series analysis technique to detect changes in network conditions. In one embodiment, the network metric(s) may be evaluated continuously or periodically. The method may then include, at 403, changing or adapting the baseline to detected changes in network conditions using historical data, machine learning, and/or time series analysis techniques. Adapting 403 of the baseline may include periodically determining a trend of at least one network metric, and modifying the baseline when a change in trend is detected.

In one embodiment, the method may further comprise determining variability from predicted values of the network metric(s) over a future time period using at least one time series analysis technique. According to some embodiments, the method may further comprise: comparing the current observed value of the network metric(s) to a baseline; and when the currently observed value of the network metric(s) is above the upper limit of the baseline or below the lower limit of the baseline , detect violations. In some embodiments, the method may also include storing each past detected violation of the baseline as a record in the KB. As shown in FIG. 3 discussed above, each record in the KB may include a time period during which a violation occurred, a duration of the violation, and/or a maximum degree of the violation.

In one embodiment, when a violation is detected, the method may include examining the KB to determine if there is a violation similar to the detected violation stored in the KB. When no similar violations are found in the KB, the method may include generating an alert and creating a new record in the KB for the detected violation. When a similar violation is found in the KB, the method may include monitoring the network metric(s) that caused the violation to follow the records of the similar violation to confirm that the detected violation does not indicate an anomaly in the network.

According to some embodiments, when an alert is generated, the method may include: sending the alert to a network administrator; receiving a response indicating whether the violation that triggered the alert represents an anomaly in the network; and updating the knowledge base to store the response in the violation in the record. In some embodiments, the method may also include assigning the network metric(s) to a cluster according to the alignment between the metrics, and grouping records in the KB based on the cluster.

Fig. 4b shows an example process flow diagram of a method according to another embodiment. As shown in Figure 4b, the method starts at 404, and at 405 historical data may be used to generate an initial set of, for example, metric clusters, predictive models, and metric baselines, eg, as shown in Figure 4a. These parameters can be used by the system until new metrics must be incorporated or a change in trend is detected. The following steps can be performed for each metric.

Continuing with FIG. 4b, at 410 metrics can be periodically monitored and received data can be pre-processed and stored in a database or memory. At 415, recent observed trends can be determined. At 420, the determined trend can be compared to previous trends to determine if there is a change in trend. If a trend change is detected, the membership of the measure to any cluster is re-evaluated. At 425, the results of this evaluation can result in remaining in the same cluster, joining a different cluster, or starting a new cluster. Additionally, at 425, the latest observations can be used to update the predictive model and baseline of the metric. On the other hand, if the trend has not changed, then at 430 the observation can be compared to the current baseline, and if it is within set boundaries, the system waits to make the next observation. If the monitored value is outside the set boundaries, then at 440 the KB can be checked for the presence of an entry for the cluster to which the metric belongs. If there are no entries, at 450 an entry can be created and an alert generated. If at least one entry is found at 440, the metric is closely monitored for following all such entries in the KB until it is confirmed at 445 that it does not indicate an anomaly, or that it indicates an anomaly, in which case at 450 Generate an alert. Finally, when an alert is generated at 450, the KB can be updated and the expert's response can be incorporated at 455. The process can then end at 460 .

Figure 5 shows an example of an apparatus 10 according to one embodiment. In one embodiment, apparatus 10 may be a node, host or server in or serving a communication network. For example, apparatus 10 may be a router, switch, access point, base station, radio network controller, Node B, evolved Node B (eNB), 5G Node B (gNB), Next Generation Node B (NG-NB), WLAN Access Point, Mobility Management Entity (MME), Serving Gateway (SGW), Application Server or Subscription Server associated with a radio access network such as UMTS network, LTE network or 5G radio access technology. It should be noted that those of ordinary skill in the art should understand that the device 10 may include components or features not shown in FIG. 5 .

As shown in FIG. 5, apparatus 10 may include a processor 12 for processing information and executing instructions or operations. Processor 12 may be any type of general or special purpose processor. Although a single processor 12 is shown in FIG. 5, according to other embodiments, multiple processors may be utilized. In fact, as an example, processor 12 may include one or more of the following: a general purpose computer, a special purpose computer, a microprocessor, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and processors based on multi-core processor architectures.

Processor 12 may perform functions associated with the operation of apparatus 10, which may include, for example, precoding of antenna gain/phase parameters, encoding and decoding of individual bits forming communication messages, formatting of information, and processing of apparatus 10 General control, including processes related to the management of communication resources.

Apparatus 10 may also include or be coupled to memory 14 (internal or external) (which may be coupled to processor 12 ) for storing information and instructions executable by processor 12 . Memory 14 may include volatile memory 24 and/or non-volatile memory 25 . Accordingly, memory 14 may be one or more memories, and may be of any type suitable for the local application environment, and may be implemented using any suitable volatile or non-volatile data storage technology, such as semiconductor-based memory devices , Magnetic storage devices and systems, Optical storage devices and systems, Fixed and removable storage. For example, volatile memory 24 may include random access memory (RAM) such as dynamic or static RAM. Non-volatile memory 25 may include, for example, read-only memory (ROM), flash memory, and/or mechanical disks, such as hard disks or optical disks. Accordingly, memory 14 may comprise any combination of random access memory (RAM), read only memory (ROM), static memory such as a magnetic or optical disk, hard disk drive (HDD), or any other type of non-transitory machine or computer readable medium. The instructions stored in memory 14 may include program instructions or computer program code that, when executed by processor 12 , enable apparatus 10 to perform the tasks described herein.

In some embodiments, device 10 may also include or be coupled to one or more antennas 15 for transmitting signals to and/or receiving signals and/or data from device 10 . Apparatus 10 may also include or be coupled to a transceiver 18 configured to transmit and receive information. Transceiver 18 may include, for example, multiple radio interfaces that may be coupled to antenna(s) 15 . The radio interface may correspond to a variety of radio access technologies, including one or more of the following: GSM, NB-IoT, LTE, 5G, WLAN, Bluetooth, BT-LE, NFC, Radio Frequency Identifier (RFID), Ultra Broadband (UWB), etc. The radio interface may include components for generating symbols for transmission via one or more downlinks and for receiving symbols (e.g., via an uplink), such as filters, converters (e.g., digital-to-analog converters, etc.) , Mapper, Fast Fourier Transform (FFT) module, etc. As such, transceiver 18 may be configured to modulate information onto a carrier waveform for transmission by antenna(s) 15, and to demodulate information received via antenna(s) 15 for further processing by other elements of apparatus 10. Transceiver 18 may be capable of directly transmitting and receiving signals or data.

In one embodiment, memory 14 may store software modules that provide functionality when executed by processor 12 . For example, the module may include an operating system that provides operating system functionality for device 10 . Memory 14 may also store one or more functional modules, such as applications or programs, to provide additional functionality to device 10 . For example, in one embodiment, memory 14 may store a monitoring application, such as monitoring application 110 shown in FIG. 1 . Note that the components of apparatus 10 may be implemented in hardware, or as any suitable combination of hardware and software.

In one embodiment, apparatus 10 may be a network node, a network element, or a server, as described above. According to certain embodiments, apparatus 10 may be controlled by memory 14 and processor 12 to perform functions associated with any of the embodiments described herein. For example, in some embodiments, the device 10 may be controlled by the memory 14 and the processor 12 to execute at least any one of the methods or block diagrams shown in FIGS. 1-4 .

According to an example embodiment, apparatus 10 may be controlled by memory 14 and processor 12 to calculate or generate predicted values of one or more network metrics for a future time period. Predicted values may be calculated, for example, by using historical data, machine learning and/or time series analysis techniques. According to one embodiment, forecast values may be generated periodically and/or may be stored in memory for use with future forecasts of the metric(s).

In one embodiment, apparatus 10 may be controlled by memory 14 and processor 12 to determine or generate a baseline of network metric(s) using predicted values and/or historical data. A baseline may indicate or include upper and/or lower bounds for a network metric(s), eg, as shown in the example of FIG. 3 discussed above. In some embodiments, apparatus 10 may also be controlled by memory 14 and processor 12 to assess or evaluate network metric(s) using, for example, at least one time series analysis technique to detect changes in network conditions. In one embodiment, the network metric(s) may be evaluated continuously or periodically. According to one embodiment, apparatus 10 may be controlled by memory 14 and processor 12 to use historical data, machine learning and/or time series analysis techniques to change or adapt the baseline to detected changes in network conditions. In one embodiment, apparatus 10 may be controlled by memory 14 and processor 12 to change or adapt the baseline by periodically determining a trend of at least one network metric and/or modifying the baseline when a change in trend is detected.

According to one embodiment, apparatus 10 may be controlled by memory 14 and processor 12 to determine variability from predicted values of network metric(s) over a future time period using at least one time series analysis technique. In some embodiments, apparatus 10 may be controlled by memory 14 and processor 12 to compare the currently observed value of the network metric(s) with a baseline, and when the currently observed value of the network metric(s) is at the upper limit of the baseline Above or below the lower limit of the baseline, a violation is detected. In some embodiments, apparatus 10 may be controlled by memory 14 and processor 12 to store each past detected violation of the baseline as a record in the KB. As shown in FIG. 3 discussed above, each record in the KB may include a time period during which a violation occurred, a duration of the violation, and/or a maximum degree of the violation.

In one embodiment, when a violation is detected, apparatus 10 may be controlled by memory 14 and processor 12 to examine the KB to determine if there is a violation similar to the detected violation stored in the KB. As discussed in detail above, a violation can be considered to be related to another KB if the time period over which it occurred falls within the same time range (e.g., from 0000hrs to 0600hrs) and the duration and maximum magnitude are less than the corresponding values of existing entries in the KB. Violation is similar.

When no similar violation is found in the KB, then the apparatus 10 may be controlled by the memory 14 and processor 12 to generate an alarm and create a new record in the KB for the detected violation. When a similar violation is found in the KB, then the apparatus 10 may be controlled by the memory 14 and the processor 12 to monitor the network metric(s) that cause the violation to follow the records of the similar violation to confirm that the detected violation does not indicate a violation in the network. exception.

According to some embodiments, when an alert is generated, device 10 may be controlled by memory 14 and processor 12 to send said alert to a network administrator or expert, receiving from the network administrator/expert an indication whether the violation triggering the alert represents a network , and update the knowledge base to store the response in the violating record. In some embodiments, apparatus 10 may be controlled by memory 14 and processor 12 to assign network metric(s) to clusters according to alignment between the metrics, and to group records in the KB based on the clusters.

Accordingly, embodiments of the present invention provide several technical improvements, enhancements and/or advantages. For example, as a result of certain embodiments, the network can be better monitored in real time. Thus, the reliability, availability and/or operational efficiency of the network and its nodes are ensured and improved. Additionally, certain embodiments may reduce the number of false alarms and limit the burden on network administrators. As such, embodiments of the present invention may improve the performance and throughput of networks and network nodes, including for example access points, base stations/eNBs/gNBs and mobile devices or UEs. Thus, the use of embodiments of the present invention results in improved functionality of the communication network and its nodes.

In some embodiments, the functionality of any method, process, signaling diagram or flowchart described herein may be implemented by software and/or computer program code or code portions stored in a memory or other computer readable or tangible medium , and is executed by the processor.

In some embodiments, an apparatus may be comprised or associated with at least one software application, module, unit or entity configured to perform an arithmetic operation(s), or to be A configuration is a program or portion thereof, including added or updated software routines, executed by at least one operational processor. Programs (also called program products or computer programs, including software routines, applets, and macros) can be stored on any device-readable data storage medium and include program instructions to perform particular tasks.

A computer program product may include one or more computer-executable components that, when the program is run, are configured to perform embodiments. One or more computer-executable components may be at least one software code or a portion thereof. Modifications and configurations required to implement the functionality of the embodiments may be performed as routine(s), which may be implemented as added or updated software routine(s). The software routine(s) may be downloaded into the device.

Software or computer program code, or portions thereof, may be in source code form, object code form, or some intermediate form and may be stored on some carrier, distribution medium or computer-readable medium, which may be any entity or device capable of carrying the program . Such carriers include, for example, recording media, computer memories, read-only memories, electro-optical and/or electrical carrier signals, telecommunication signals and software distribution packages. Depending on the processing power required, a computer program can be executed in a single electronic digital computer or it can be distributed among a plurality of computers. Computer readable media or computer readable storage media may be non-transitory media.

In other embodiments, the functionality may be performed by hardware or circuitry included in a device (e.g., device 10 or device 20), such as through the use of application-specific integrated circuits (ASICs), programmable gate arrays (PGAs), field programmable Gate Array (FPGA), or any other combination of hardware and software. In yet another embodiment, this function may be implemented as a signal, an intangible means that may be carried by an electromagnetic signal downloaded from the Internet or other network.

According to one embodiment, a device such as a node, a device or a corresponding component may be configured as a circuit system, computer or microprocessor (such as a single-chip computer element) or a chipset comprising at least a memory for providing storage capacity for arithmetic operations and an arithmetic processor for performing the arithmetic operation.

Those of ordinary skill in the art will readily appreciate that the invention as described above may be practiced in a different order of steps and/or with different configurations of hardware elements than that disclosed. Therefore, although the present invention has been described based on these preferred embodiments, it will be apparent to those skilled in the art that certain modifications, variations and alternative constructions will be made without departing from the spirit and scope of the present invention. very clear. For determining the limits of the invention, therefore, reference should be made to the appended claims.

Claims (11)

1. A method for communication, comprising:

generating a predicted value of at least one network metric over a future time period, wherein the generating includes at least one of using historical data, machine learning, or time series analysis techniques;

generating a baseline for the at least one network metric using at least one of the predicted value or the historical data, wherein the baseline includes an upper limit and a lower limit;

evaluating the at least one network metric using the time series analysis technique to detect a change in network conditions;

adapting the baseline to the detected change in the network condition using the at least one of historical data, machine learning, or time series analysis techniques;

comparing the current observations of the at least one network metric to the baseline;

detecting a violation when the current observed value of the at least one network metric is above the upper limit of the baseline or below the lower limit of the baseline;

storing each past detected violation of the baseline in a knowledge base as a record, wherein the record includes at least one of a time period during which the violation occurred, a duration of the violation, or a maximum extent of the violation; and

When a violation is detected, a decision is made using the knowledge base as to whether to raise an alarm.

2. An apparatus for communication, comprising:

means for performing the method of claim 1.

3. An apparatus for communication, comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to

Generating a predicted value of at least one network metric over a future time period, wherein the generating includes at least one of using historical data, machine learning, or time series analysis techniques;

generating a baseline for the at least one network metric using at least one of the predicted value or the historical data, wherein the baseline includes an upper limit and a lower limit;

evaluating the at least one network metric using the time series analysis technique to detect a change in network conditions;

adapting the baseline to the detected change in the network condition using the at least one of historical data, machine learning, or time series analysis techniques;

Comparing the current observations of the at least one network metric to the baseline;

detecting a violation when the current observed value of the at least one network metric is above the upper limit of the baseline or below the lower limit of the baseline;

storing each past detected violation of the baseline in a knowledge base as a record, wherein the record includes at least one of a time period during which the violation occurred, a duration of the violation, or a maximum extent of the violation; and

when a violation is detected, a decision is made using the knowledge base as to whether to raise an alarm.

4. The apparatus of claim 3, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to determine variability from the predicted value of the at least one network metric over the future time period using the at least one time series analysis technique.

5. The apparatus according to claim 3 or 4, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to: periodically determining a trend of the at least one network metric, and

When a change in the trend is detected, the baseline is modified.

6. The apparatus of claim 3 or 4, wherein when a violation is detected, the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

checking the knowledge base to determine if there is a violation similar to the detected violation stored in the knowledge base;

when no similar violation is found in the knowledge base, generating an alert and creating a record in the knowledge base for the violation detected; and

when a similar violation is found in the knowledge base, the at least one network metric that causes the violation to follow the record of the similar violation is monitored to confirm that the detected violation does not represent an anomaly in the network.

7. The apparatus of claim 6, wherein when an alert is generated, the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

sending the alert to a network administrator;

receiving a response indicating whether the violation triggering the alarm represents an anomaly in the network; and

The knowledge base is updated to store the response in the record of the violation.

8. The apparatus according to claim 3 or 4, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

assigning the at least one network metric to a cluster according to an alignment between metrics; and

records in the knowledge base are grouped based on the clusters.

9. The apparatus according to claim 3 or 4, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

the predicted values are stored in memory for use with future predictions of the at least one network metric.

10. The apparatus of claim 3 or 4, wherein the predictive value is generated periodically, and wherein the at least one network metric is continuously evaluated to detect a change in the network condition.

11. A non-transitory computer readable medium having stored thereon a computer program, wherein the computer program when executed by a processor causes the processor to perform the steps of a method for communication, comprising:

Generating a predicted value of at least one network metric over a future time period, wherein the generating includes at least one of using historical data, machine learning, or time series analysis techniques;

generating a baseline for the at least one network metric using at least one of the predicted value or the historical data, wherein the baseline includes an upper limit and a lower limit;

evaluating the at least one network metric using the time series analysis technique to detect a change in network conditions;

adapting the baseline to the detected change in the network condition using the at least one of historical data, machine learning, or time series analysis techniques;

comparing the current observations of the at least one network metric to the baseline;

detecting a violation when the current observed value of the at least one network metric is above the upper limit of the baseline or below the lower limit of the baseline;

storing each past detected violation of the baseline in a knowledge base as a record, wherein the record includes at least one of a time period during which the violation occurred, a duration of the violation, or a maximum extent of the violation; and

When a violation is detected, a decision is made using the knowledge base as to whether to raise an alarm.

Images