Skip to content

Orb Policy Reference

The policy model has seven top level sections: name, desciption, tags, backend, schema_version, policy and format.

{
    "name": "",
    "description" : "",
    "tags": {},
    "backend": "",
    "schema_version": "",
    "format": "",
    "policy": {}
}
Policy Sections Default Required
name - ✅
backend - ✅
policy - ✅
description None ❌
tags None ❌
schema_version 1.0 ❌
format json ❌
  • Name

    Policy name must be unique, must start with a letter and contain only letters, numbers, "-" or "_".

Examples

Success

"name": "my_policy"
"name": "MY-policy_2

Failure

"name": "1_policy"
"name": "MY-policy/2

  • Description

    Policy description is a free string. You can describe using spaces, letters, numbers and special characters.

  • Tags

    Tags are intended to facilitate organization and are a dict type.

The tags usage syntax is:

    "tags": {"key1":"value1", "key2":"value2", "key3":"value3"}

  • Backend

    Backend determine to which backend the policy will be attached. Check the available backends here.

  • Schema version

    Each backend supported on Orb must have a policy schema to be validated, parsed and applied. schema_version is the field responsible for allowing different schema versions and backward compatibility. Default should be "1.0".

  • Format

    The format specifies in which format the policy data will be written. The options are json and yaml. json is the default value.

  • Policy

    Each supported backend has specific structures for its policy data. Check the available options below.

Pktvisor policy

The policy data for pktvisor can be written in either YAML or JSON, and has four top level sections: “input”, “handlers”, "config" and “kind”.

input: ..
config: ...
kind: collection
handlers: ...

{
  "input": {       
   },
  "config": {
   },
  "kind": "collection"
   },
  "handlers": {        
}

Input section

The input section specifies what data streams the policy will be using for analysis, in other words, this specifies what data the agent should be listening in on, and is defined at the agent level.

Required fields:
input_type - the type of input. This field will be validated with the type of tap indicated by the tap parameter or by the tap selector . If the types are incompatible, the policy will fail.

tap - the name given to this input in the tap/agent configuration or tap_selector - tags to match existing agent taps. If tap_selector is used, it can be chosen whether taps with any of the tags or with all tags will be attached.

Optional fields:
filter - to specify what data to include from the input
config - how the input will be used

Note

Every configuration set at the input can be reset at the tap level, with the one set on the tap dominant over the one set on the input.

Default input structure

Using specific tap (check the application in a policy here):

input:
  tap: tap_name
  input_type: type_of_input
  filter:
    bpf: ...
  config:
    ...

{
  "input": {
    "tap": "tap_name",
    "input_type": "type_of_input",
    "filter": {
      "bpf": "..."
    },
    "config": "..."
  }
}

or using tap selector matching any (check the application in a policy here):

input:
  tap_selector:
    any:
      - key1: value1
      - key2: value2
  input_type: type_of_input
  filter:
    bpf: ...
  config:
    ...

{
  "input": {
    "tap_selector": {
      "any": [
        {
          "key1": "value1"
        },
        {
          "key2": "value2"
        }
      ]
    },
    "input_type": "type_of_input",
    "filter": {
      "bpf": "..."
    },
    "config": "..."
  }
}

or using tap selector matching all (check the application in a policy here):

input:
  tap_selector:
    all:
      - key1: value1
      - key2: value2
  input_type: type_of_input
  filter:
    bpf: ...
  config:
    ...

{
  "input": {
    "tap_selector": {
      "all": [
        {
          "key1": "value1"
        },
        {
          "key2": "value2"
        }
      ]
    },
    "input_type": "type_of_input",
    "filter": {
      "bpf": "..."
    },
    "config": "..."
  }
}

Config section

There is the possibility of defining settings on the policy level. Currently, the only configuration available is the merge_like_handlers.

Policy Configuration Type Default
merge_like_handlers bool false

merge_like_handlers

When merge_like_handlers config is true, metrics from all handlers of the same type are scraped together. This is useful when the tap_selector is used, as, by default, metrics are generated separately for each tap in the policy and this can be very expensive, depending on the number of taps.

The merge_like_handlers filter usage syntax is:

config:
  merge_like_handlers: true

{
  "config": {
    "merge_like_handlers": true
  }
}

Kind section

What kind of object you want to create The only option for now is "collection".

Handlers section (Analysis)

Handlers are the modules responsible for extracting metrics from inputs. For each handler type, specific configuration, filters and group of metrics can be defined, and there are also configs (abstract configuration) that can be applied to all handlers:

Default handler structure:

handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
    topn_percentile_threshold: 0
  modules:
    tap_name:
      type: ...
      require_version: ...
      config: ...
      filter: ...
      metric_groups:
        enable:
          - ...
          - ....
        disable:
          - .....
          - ......     

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10,
      "topn_percentile_threshold": 0
    },
    "modules": {
      "tap_name": {
        "type": "...",
        "require_version": "...",
        "config": "...",
        "filter": "...",
        "metric_groups": {
          "enable": [
            "...",
            "...."
          ],
          "disable": [
            ".....",
            "......"
          ]
        }
      }
    }
  }
}


To enable any metric group use the syntax:

metric_groups:
  enable:
    - group_to_enable

{
  "metric_groups": {
    "enable": [
      "group_to_enable"
    ]
  }
}

To enable all available metric groups use the syntax:

metric_groups:
  enable:
    - all

{
  "metric_groups": {
    "enable": [
      "all"
    ]
  }
}

In order to disable any metric group use the syntax:

metric_groups:
  disable:
    - group_to_disable

{
  "metric_groups": {
    "disable": [
      "group_to_disable"
    ]
  }
}

To disable all metric groups use the syntax:

metric_groups:
  disable:
    - all

{
  "metric_groups": {
    "disable": [
      "all"
    ]
  }
}
  • Attention: enable is dominant over disable. So if both are passed, the metrics group will be enabled;

Abstract Configurations

There are general configurations, which can be applied to all handlers. These settings can be reset for each module, within the specific module configs. In this case, the configuration inside the module will override the configuration passed in general handler.

Abstract Configuration Type Default
deep_sample_rate int 100 (per second)
num_periods int 5
topn_count int 10
topn_percentile_threshold int 0

deep_sample_rate
Back to Top - Abstract Configurations

deep_sample_rate determines the number of data packets that will be analyzed deeply per second. Some metrics are operationally expensive to generate, such as metrics that require string parsing (qname2, qtype, etc.). For this reason, a maximum number of packets per second to be analyzed is determined. If in one second fewer packages than the maximum amount are transacted, all packages will compose the deep metrics sample, if there are more packages than the established one, the value of the variable will be used. Allowed values are in the range [1,100]. Default value is 100.

Note

If a value less than 1 is passed, the deep_sample_rate will be 1. If the value passed is more than 100, deep_sample_rate will be 100.

The deep_sample_rate usage syntax is:

deep_sample_rate: int

{
  "deep_sample_rate": int
}

num_periods
Back to Top - Abstract Configurations

num_periods determines the amount of minutes of data that will be available on the metrics endpoint. Allowed values are in the range [2,10]. Default value is 5.

The num_periods usage syntax is:

num_periods: int

{
"num_periods": int
}

topn_count
Back to Top - Abstract Configurations

topn_count sets the maximum amount of elements displayed in top metrics. If there is less quantity than the configured value, the composite metrics will have the existing value. But if there are more metrics than the configured value, the variable will be actively limiting. Any positive integer is valid and the default value is 10.

The topn_count usage syntax is:

topn_count: int

{
"topn_count": int
}

topn_percentile_threshold
Back to Top - Abstract Configurations

topn_percentile_threshold sets the threshold of data to be considered based on the percentiles, so allowed values are in the range [0,100]. The default value is 0, that is, all data is considered. If, for example, the value 10 is set, scraped topn metrics will only consider data from the 10th percentile, that is, data between the highest 90%.

The topn_percentile_threshold usage syntax is:

topn_percentile_threshold: int

{
"topn_percentile_threshold": int
}

DNS Analyzer (dns)

Warning

Status: Beta. The metric names and configuration options may still change

Example of policy with input pcap and handler DNS(v2) 
handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
  modules:
    default_dns:
      type: dns
      require_version: "2.0"
      config:
        public_suffix_list: true
        deep_sample_rate: 50
        num_periods: 2
        topn_count: 25
        topn_percentile_threshold: 10
      filter:
        only_rcode: 0
        only_dnssec_response: true
        answer_count: 1
        only_qtype: [1, 2]
        only_qname_suffix: [".google.com", ".orb.live"]
        geoloc_notfound: false
        asn_notfound: false
        dnstap_msg_type: "auth"
      metric_groups:
        enable:
          - cardinality
          - counters
          - quantiles
          - top_rcodes
          - top_qnames
          - top_qtypes
        disable:
          - top_size
          - top_ports
          - xact_times
          - top_ecs
input:
  input_type: pcap
  tap: default_pcap
  filter:
    bpf: udp port 53
  config:
    iface: wlo1
    host_spec: 192.168.1.167/24
    pcap_source: libpcap
    debug: true
config:
    merge_like_handlers: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10
    },
    "modules": {
      "default_dns": {
        "type": "dns",
        "require_version": "2.0",
        "config": {
          "public_suffix_list": true,
          "deep_sample_rate": 50,
          "num_periods": 2,
          "topn_count": 25,
          "topn_percentile_threshold": 10
        },
        "filter": {
          "only_rcode": 0,
          "only_dnssec_response": true,
          "answer_count": 1,
          "only_qtype": [
            1,
            2
          ],
          "only_qname_suffix": [
            ".google.com",
            ".orb.live"
          ],
          "geoloc_notfound": false,
          "asn_notfound": false,
          "dnstap_msg_type": "auth"
        },
        "metric_groups": {
          "enable": [
            "cardinality",
            "counters",
            "quantiles",
            "top_rcodes",
            "top_qnames",
            "top_qtypes"
          ],
          "disable": [
            "top_size",
            "top_ports",
            "xact_times",
            "top_ecs"
          ]
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap": "default_pcap",
    "filter": {
      "bpf": "udp port 53"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.167/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "config": {
    "merge_like_handlers": true
  },
  "kind": "collection"
}


Handler Type: "dns"

Metrics Group - Check the dns metrics belonging to each group

Metric Group Default
top_ecs disabled
top_ports disabled
top_size disabled
xact_times disabled
cardinality enabled
counters enabled
top_qnames enabled
quantiles enabled
top_qtypes enabled
top_rcodes enabled

Filters

Filter Type Input
only_rcode str[] PCAP
exclude_noerror bool PCAP
only_dnssec_response bool PCAP
answer_count int PCAP
only_qtype str[] PCAP
only_qname str[] PCAP
only_qname_suffix str[] PCAP
geoloc_notfound bool PCAP
asn_notfound bool PCAP
only_xact_directions str[] PCAP
dnstap_msg_type str DNSTAP

only_rcode: str[].
Back to DNS-v2 filters list

Input: PCAP

When a DNS server returns a response to a query made, one of the properties of the response is the "response code" (rcode), a code that describes what happened to the query that was made.

Most response codes indicate why the query failed and when the query succeeds, the return is an RCODE:0, whose name is NOERROR.
Supported types are in the table below (if you use any other code that is not in the table below, your policy will fail):

DNS response code Name Description Reference
0 NOERROR No error condition [RFC1035]
1 FORMERR Format error - The name server was unable to interpret the query. [RFC1035]
2 SERVFAIL Server failure - The name server was unable to process this query due to a problem with the name server. [RFC1035]
3 NXDOMAIN Name Error - Meaningful only for responses from an authoritative name server, this code signifies that the domain name referenced in the query does not exist. [RFC1035]
4 NOTIMP Not Implemented - The name server does not support the requested kind of query. [RFC1035]
5 REFUSED The name server refuses to perform the specified operation for policy reasons. For example, a name server may not wish to provide the information to the particular requester, or a name server may not wish to perform a particular operation (e.g., zone) [RFC1035]
6 YXDOMAIN Name that should not exist, does exist [RFC2136]
7 YXRRSET RR set that should not exist, does exist [RFC2136]
8 NXRRSET RR Set that should exist, does not exist [RFC2136]
9 NOTAUTH Server Not Authoritative for zone or Not Authorized [RFC2136]
10 NOTZONE Name not contained in zone [RFC2136]
11 DSOTYPENI DSO-TYPE Not Implemented [RFC8490]
16 BADVERS/BADSIG Bad OPT Version or TSIG Signature Failure [RFC8945]
17 BADKEY Key not recognized [RFC8945]
18 BADTIME Signature out of time window [RFC8945]
19 BADMODE Bad TKEY Mode [RFC2930]
20 BADNAME Duplicate key name [RFC2930]
21 BADALG Algorithm not supported [RFC2930]
22 BADTRUNC Bad Truncation [RFC8945]
23 BADCOOKIE Bad/missing Server Cookie [RFC7873]

The only_rcode filter usage syntax is:

only_rcode:
  - str
  - str

{
  "only_rcode": ["str", "int"]
}

with the int referring to the response code to be filtered, written as string.

Example:
If you want to filter only successful queries responses you should use (note that all that the query will be discarded and the result will be just the responses): 

only_rcode:
  - "NXDOMAIN"
  - "2"

{
"only_rcode": ["NXDOMAIN", "2"]
}

Important information is that only one response code is possible for each handler. So, in order to have multiple filters on the same policy, multiple handlers must be created, each with a rcode type.

exclude_noerror: bool
Back to DNS-v2 filters list

Input: PCAP

You may still want to filter out only responses with any kind of error. For this, there is the exclude_noerror filter, which removes from its results all responses that did not return any type of error. The exclude_noerror filter usage syntax is:

exclude_noerror: true

{
  "exclude_noerror": true
}

Attention: the filter of exclude_noerror is dominant in relation to the filter of only_rcode, that is, if the filter of exclude_noerror is true, even if the filter of only_rcode is set, the results will be composed only by responses without any type of error (all type of errors will be kept).

only_dnssec_response: bool
Back to DNS-v2 filters list

Input: PCAP

When you make a DNS query, the response you get may have a DNSSEC signature, which authenticates that DNS records originate from an authorized sender, thus protecting DNS from falsified information.
To filter only responses signed by an authorized sender, use: The only_dnssec_response filter usage syntax is:

only_dnssec_response: true

{
"only_dnssec_response": true
}


answer_count: int
Back to DNS-v2 filters list

Input: PCAP

One of the properties present in the query message structure is Answer RRs, which is the count of entries in the responses section (RR stands for “resource record”).
The number of answers in the query is always zero, as a query message has only questions and no answers, and when the server sends the answer to that query, the value is set to the amount of entries in the answers section.

The answer_count filter usage syntax is:

answer_count: int

{
  "answer_count": int
}

with the int referring to the desired amount of answer.
Note that any value greater than zero that is defined will exclude queries from the results, since in queries the number of answers is always 0.
As the answers count of queries is 0, whenever the value set for the answer_count is 0, both queries and responses will compose the result.

A special case is the concept of NODATA, which is one of the possible returns to a query made to a DNS server is known as. This happens when the query is successful (so rcode:0), but there is no data as a response, so the number of answers is 0.
In this case, to have in the results only the cases of NODATA, that is, the responses, the filter must be used together with the filter exclude_noerror.

Important information is that only one answer_count is possible for each handler. So, in order to have multiple counts on the same policy, multiple handlers must be created, each with an amount of answers.

only_qtype: str[]
Back to DNS-v2 filters list

Input: PCAP

DNS record types are records that provide important information about a hostname or domain. Supported default types can be seen here.

The only_qtype filter usage syntax is:

only_qtype: 
  - str
  - str

{
  "only_qtype": [
    "str", "str"
  ]
}

If you want to filter only IPV4 record types, for example, you should use: 

only_qtype:
  - "A"

{
  "only_qtype": [
    "A"
  ]
}

or

only_qtype:
  - 1

{
  "only_qtype": [
    1
  ]
}

Multiple types are also supported and both queries and responses that have any of the values in the array will be considered.

only_qtype:
  - 1
  - 2
  - "A"

{
  "only_qtype": [
    1,
    2,
    "A"
  ]
}


only_qname: str[]
Back to DNS-v2 filters list

Input: PCAP

The only_qname filters dns packets based on queries and responses whose names exactly matches the strings present in the array.
The only_qname filter usage syntax is:

only_qname:
  - str
  - str

{
  "only_qname": [
    "str", "str"
  ]
}

Examples:

only_qname:
  - www.google.com
  - .nsone.net

{
  "only_qname": [
    "www.google.com",
    ".nsone.net"
  ]
}


only_qname_suffix: str[]
Back to DNS-v2 filters list

Input: PCAP

The only_qname_suffix filters queries and responses whose endings (suffixes) of the names match the strings present in the array.
The only_qname_suffix filter usage syntax is:

only_qname_suffix:
  - str
  - str

{
  "only_qname_suffix": [
    "str", "str"
  ]
}

Examples:

only_qname_suffix:
  - .google.com

{
  "only_qname_suffix": [
    ".google.com"
  ]
}

or

only_qname_suffix:
  - google.com
  - .nsone.net

{
  "only_qname_suffix": [
    "google.com",
    ".nsone.net"
  ]
}


geoloc_notfound: bool
Back to DNS-v2 filters list

Input: PCAP

Based on ECS (EDNS Client Subnet) information, it is possible to determine the geolocation of where the query is being made. When the Subnet refers to a region found in the standard databases, the city, state and country (approximated) are returned. However, when based on the subnet it is not possible to determine the geolocation, a not found is returned.
The geoloc_notfound filter only keeps responses whose geolocations were not found.
The geoloc_notfound filter usage syntax is:

geoloc_notfound: true

{
  "geoloc_notfound": true
}


asn_notfound: bool
Back to DNS-v2 filters list

Input: PCAP

Based on ECS (EDNS Client Subnet) information, it is possible to determine the ASN (Autonomous System Number). When the IP of the subnet belongs to some not known ASN in the standard databases, a not found is returned.
The asn_notfound filter only keeps responses whose asn were not found.
The asn_notfound filter usage syntax is:

asn_notfound: true

{
  "asn_notfound": true
}


only_xact_directions: str[]
Back to DNS-v2 filters list

Input: PCAP

Filters metrics according to the direction of the transaction. Options are: in, out and unknown.

only_xact_directions:
  - str
  - str
Example: 
only_xact_directions:
- in
- unknown


{
  "only_xact_directions": [
    "str",
    "str"
  ]
}
Example: 
{
  "only_xact_directions": [
    "in",
    "unknown"
  ]
}


dnstap_msg_type: str
Back to DNS-v2 filters list

Input: DNSTAP

With a dnstap protocol it is possible to know the type of message that must be resolved in the request to the server. This filter therefore allows you to filter by response types. Supported message types are: auth, resolver, client, forwarder, stub, tool and update.

The dnstap_msg_type filter usage syntax is:

dnstap_msg_type: str
Example: 
dnstap_msg_type: auth


{
  "dnstap_msg_type": "str"
}
Example: 
{
  "dnstap_msg_type": "auth"
}


Example of policy with input pcap and handler DNS(v1)
handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
  modules:
    default_dns:
      type: dns
      config:
        public_suffix_list: true
        deep_sample_rate: 50
        num_periods: 2
        topn_count: 25
        topn_percentile_threshold: 10
      filter:
        only_rcode: 0
        only_dnssec_response: true
        answer_count: 1
        only_qtype: [1, 2]
        only_qname_suffix: [".google.com", ".orb.live"]
        geoloc_notfound: false
        asn_notfound: false
        dnstap_msg_type: "auth"
      metric_groups:
        enable:
          - top_ecs
          - top_qnames_details
        disable:
          - histograms
          - quantiles
          - cardinality
          - counters
          - dns_transaction
          - top_qnames
          - top_ports
input:
  input_type: pcap
  tap: default_pcap
  filter:
    bpf: udp port 53
  config:
    iface: wlo1
    host_spec: 192.168.1.167/24
    pcap_source: libpcap
    debug: true
config:
    merge_like_handlers: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10
    },
    "modules": {
      "default_dns": {
        "type": "dns",
        "config": {
          "public_suffix_list": true,
          "deep_sample_rate": 50,
          "num_periods": 2,
          "topn_count": 25,
          "topn_percentile_threshold": 10
        },
        "filter": {
          "only_rcode": 0,
          "only_dnssec_response": true,
          "answer_count": 1,
          "only_qtype": [
            1,
            2
          ],
          "only_qname_suffix": [
            ".google.com",
            ".orb.live"
          ],
          "geoloc_notfound": false,
          "asn_notfound": false,
          "dnstap_msg_type": "auth"
        },
        "metric_groups": {
          "enable": [
            "top_ecs",
            "top_qnames_details"
          ],
          "disable": [
            "cardinality",
            "counters",
            "dns_transaction",
            "top_qnames",
            "top_ports"
          ]
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap": "default_pcap",
    "filter": {
      "bpf": "udp port 53"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.167/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "config": {
    "merge_like_handlers": true
  },
  "kind": "collection"
}

Handler Type: "dns"

Metrics Group - Check the dns metrics belonging to each group

Metric Group Default
top_ecs disabled
top_qnames_details disabled
histograms disabled
cardinality enabled
counters enabled
dns_transaction enabled
top_qnames enabled
top_ports enabled
quantiles enabled

Filters

Filter Type Input
only_rcode str[] PCAP
exclude_noerror bool PCAP
only_dnssec_response bool PCAP
answer_count int PCAP
only_qtype str[] PCAP
only_qname str[] PCAP
only_qname_suffix str[] PCAP
geoloc_notfound bool PCAP
asn_notfound bool PCAP
only_queries bool PCAP
only_responses bool PCAP
dnstap_msg_type str DNSTAP

only_rcode: str[].
Back to DNS-v1 filters list

Input: PCAP

When a DNS server returns a response to a query made, one of the properties of the response is the "response code" (rcode), a code that describes what happened to the query that was made.

Most response codes indicate why the query failed and when the query succeeds, the return is an RCODE:0, whose name is NOERROR.
Supported types are in the table below (if you use any other code that is not in the table below, your policy will fail):

DNS response code Name Description Reference
0 NOERROR No error condition [RFC1035]
1 FORMERR Format error - The name server was unable to interpret the query. [RFC1035]
2 SERVFAIL Server failure - The name server was unable to process this query due to a problem with the name server. [RFC1035]
3 NXDOMAIN Name Error - Meaningful only for responses from an authoritative name server, this code signifies that the domain name referenced in the query does not exist. [RFC1035]
4 NOTIMP Not Implemented - The name server does not support the requested kind of query. [RFC1035]
5 REFUSED The name server refuses to perform the specified operation for policy reasons. For example, a name server may not wish to provide the information to the particular requester, or a name server may not wish to perform a particular operation (e.g., zone) [RFC1035]
6 YXDOMAIN Name that should not exist, does exist [RFC2136]
7 YXRRSET RR set that should not exist, does exist [RFC2136]
8 NXRRSET RR Set that should exist, does not exist [RFC2136]
9 NOTAUTH Server Not Authoritative for zone or Not Authorized [RFC2136]
10 NOTZONE Name not contained in zone [RFC2136]
11 DSOTYPENI DSO-TYPE Not Implemented [RFC8490]
16 BADVERS/BADSIG Bad OPT Version or TSIG Signature Failure [RFC8945]
17 BADKEY Key not recognized [RFC8945]
18 BADTIME Signature out of time window [RFC8945]
19 BADMODE Bad TKEY Mode [RFC2930]
20 BADNAME Duplicate key name [RFC2930]
21 BADALG Algorithm not supported [RFC2930]
22 BADTRUNC Bad Truncation [RFC8945]
23 BADCOOKIE Bad/missing Server Cookie [RFC7873]

The only_rcode filter usage syntax is:

only_rcode:
  - str
  - str

{
  "only_rcode": ["str", "int"]
}

with the int referring to the response code to be filtered, written as string.

Example:
If you want to filter only successful queries responses you should use (note that all that the query will be discarded and the result will be just the responses): 

only_rcode:
  - "NXDOMAIN"
  - "2"

{
"only_rcode": ["NXDOMAIN", "2"]
}

Important information is that only one response code is possible for each handler. So, in order to have multiple filters on the same policy, multiple handlers must be created, each with a rcode type.

exclude_noerror: bool
Back to DNS-v1 filters list

Input: PCAP

The exclude_noerror filter removes from its results all responses that did not return any type of error (RCODE=0)

The exclude_noerror filter usage syntax is:

exclude_noerror: true

{
  "exclude_noerror": true
}

exclude_noerror takes precedence over the "only_rcode" filter. If exclude_noerror = True, then the results will be composed only of responses WITH an error, that is, responses with RCODE=0 will not be returned.


**only_dnssec_response:** *bool* <a name="only_dnssec_response_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>

When you make a DNS query, the response you get may have a DNSSEC signature, which authenticates that DNS records originate from an authorized sender, thus protecting DNS from falsified information. <br>
To filter only responses signed by an authorized sender, use:
The `only_dnssec_response` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    only_dnssec_response: true
    ```

=== "JSON"
    ```json
    {
    "only_dnssec_response": true
    }
    ```
<br>
**answer_count:** *int* <a name="answer_count_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>

One of the properties present in the query message structure is `Answer RRs`, which is the count of entries in the responses section (RR stands for “resource record”). <br>
The number of answers in the query is always zero, as a query message has only questions and no answers, and when the server sends the answer to that query, the value is set to the amount of entries in the answers section. <br>

The `answer_count` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    answer_count: int
    ```
=== "JSON"
    ```json
    {
      "answer_count": int
    }
    ```

with the `int` referring to the desired amount of answer. <br>
Note that any value greater than zero that is defined will exclude queries from the results, since in queries the number of answers is always 0. <br>
As the answers count of queries is 0, whenever the value set for the answer_count is 0, both queries and responses will compose the result. <br>


A special case is the concept of `NODATA`, which is one of the possible returns to a query made to a DNS server is known as. This happens when the query is successful (so rcode:0), but there is no data as a response, so the number of answers is 0. <br>
In this case, to have in the results only the cases of `NODATA`, that is, the responses, the filter must be used together with the filter `exclude_noerror`.


Important information is that only one answer_count is possible for each handler. So, in order to have multiple counts on the same policy, multiple handlers must be created, each with an amount of answers.

**only_qtype:** *str[]* <a name="only_qtype_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>

DNS record types are records that provide important information about a hostname or domain. Supported default types can be seen [here](https://github.com/orb-community/pktvisor/blob/develop/libs/visor_dns/dns.h#L30). <br>

The `only_qtype` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    only_qtype: 
      - str
      - str
    ```
=== "JSON"
    ```json
    {
      "only_qtype": [
        "str", "str"
      ]
    }
    ```
If you want to filter only IPV4 record types, for example, you should use: <br>

=== "YAML"
    ```yaml
    only_qtype:
      - "A"
    ```
=== "JSON"
    ```json
    {
      "only_qtype": [
        "A"
      ]
    }
    ```
or

=== "YAML"
    ```yaml
    only_qtype:
      - 1
    ```
=== "JSON"
    ```json
    {
      "only_qtype": [
        1
      ]
    }
    ```
Multiple types are also supported and both queries and responses that have any of the values in the array will be considered.

=== "YAML"
    ```yaml
    only_qtype:
      - 1
      - 2
      - "A"
    ```
=== "JSON"
    ```json
    {
      "only_qtype": [
        1,
        2,
        "A"
      ]
    }
    ```
<br>

**only_qname:** *str[]* <a name="only_qname_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>


The `only_qname` filters dns packets based on queries and responses whose names exactly matches the strings present in the array. <br>
The `only_qname` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    only_qname:
      - str
      - str
    ```
=== "JSON"
    ```json
    {
      "only_qname": [
        "str", "str"
      ]
    }
    ```
Examples:

=== "YAML"
    ```yaml
    only_qname:
      - www.google.com
      - .nsone.net
    ```
=== "JSON"
    ```json
    {
      "only_qname": [
        "www.google.com",
        ".nsone.net"
      ]
    }
    ```
<br>

**only_qname_suffix:** *str[]* <a name="only_qname_suffix_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>


The `only_qname_suffix` filters queries and responses whose endings (suffixes) of the names match the strings present in the array. <br>
The `only_qname_suffix` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    only_qname_suffix:
      - str
      - str
    ```
=== "JSON"
    ```json
    {
      "only_qname_suffix": [
        "str", "str"
      ]
    }
    ```
Examples:

=== "YAML"
    ```yaml
    only_qname_suffix:
      - .google.com
    ```
=== "JSON"
    ```json
    {
      "only_qname_suffix": [
        ".google.com"
      ]
    }
    ```
or

=== "YAML"
    ```yaml
    only_qname_suffix:
      - google.com
      - .nsone.net
    ```
=== "JSON"
    ```json
    {
      "only_qname_suffix": [
        "google.com",
        ".nsone.net"
      ]
    }
    ```
<br>
**geoloc_notfound:** *bool* <a name="geoloc_notfound_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>


Based on ECS (EDNS Client Subnet) information, it is possible to determine the geolocation of where the query is being made. When the Subnet refers to a region found in the standard databases, the city, state and country (approximated) are returned. However, when based on the subnet it is not possible to determine the geolocation, a `not found` is returned. <br>
The `geoloc_notfound` filter only keeps responses whose geolocations were not found. <br>
The `geoloc_notfound` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    geoloc_notfound: true
    ```
=== "JSON"
    ```json
    {
      "geoloc_notfound": true
    }
    ```
<br>
**asn_notfound:** *bool* <a name="asn_notfound_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>


Based on ECS (EDNS Client Subnet) information, it is possible to determine the ASN (Autonomous System Number). When the IP of the subnet belongs to some not known ASN in the standard databases, a `not found` is returned. <br>
The `asn_notfound` filter only keeps responses whose asn were not found. <br>
The `asn_notfound` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    asn_notfound: true
    ```
=== "JSON"
    ```json
    {
      "asn_notfound": true
    }
    ```
<br>


**only_queries:** *bool* <a name="only_queries_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>


The `only_queries` filters out all dns response packets and its usage syntax is:<br>


=== "YAML"
    ```yaml
    only_queries: true
    ```
=== "JSON"
    ```json
    {
      "only_queries": true
    }
    ```
<br>



**only_responses:** *bool* <a name="only_responses_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: PCAP <br>


The `only_responses` filters out all dns queries packets and its usage syntax is: <br>

=== "YAML"
    ```yaml
    only_responses: true
    ```
=== "JSON"
    ```json
    {
      "only_responses": true
    }
    ```
<br>


**dnstap_msg_type:** *str* <a name="dnstap_msg_type_v1"></a><br>
<font size="1">[Back to DNS-v1 filters list](#dns_filters_v1)</font>

Input: DNSTAP <br>

With a dnstap protocol it is possible to know the type of message that must be resolved in the request to the server. This filter therefore allows you to filter by response types.
Supported message types are: `auth`, `resolver`, `client`, `forwarder`, `stub`, `tool` and `update`.

The `dnstap_msg_type` filter usage syntax is:<br>

=== "YAML"
    ```yaml
    dnstap_msg_type: str
    ```
    Example:
    ```yaml
    dnstap_msg_type: auth
    ```
=== "JSON"
    ```json
    {
      "dnstap_msg_type": "str"
    }
    ```
    Example:
    ```json
    {
      "dnstap_msg_type": "auth"
    }
    ```
<br>

Configurations

public_suffix_list
Back to DNS configurations list

Some names to be resolved by a dns server have public suffixes. These suffixes cause metrics to be generated considering non-relevant data.

The example below illustrates the benefit of using this type of configuration. The qnames consider each part of the name to be resolved. When a name has a public suffix, generic information is generated. Note that in the standard configuration, Qname2 and Qname3 are the same for both domains. With the public suffix setting true (which makes the entire public part be considered as a single part), Qname3 already displays relevant information about the name.
The list of suffixes considered public can be accessed here.

Name Qname2 Standard Qname3 Standard Qname2 Public Suffix Qname3 Public Suffix
www.imagine.qname.co.uk co.uk qname.co.uk qname.co.uk imagine.qname.co.uk
other.example.qname.co.uk co.uk qname.co.uk qname.co.uk example.qname.co.uk

The public_suffix_list configuration usage syntax is:

public_suffix_list: true

{
  "public_suffix_list": true
}

recorded_stream
Back to DNS configurations list

This configuration is useful when a pcap_file is used in taps/input configuration. Set it to True when you want to load an offline traffic (from a pcap_file).

The recorded_stream configuration usage syntax is:

recorded_stream: true

{
  "recorded_stream": true
}

xact_ttl_ms OR xact_ttl_secs
Back to DNS configurations list

Both configurations have the same functionality, that is, defines the time to live of transactions, and only change the unit of measurement to be configured. This configuration causes the metrics to be generated for complete transactions (query and response) within the established time limit.

  • xact_ttl_ms: Defines the time to live of transactions in milliseconds.
  • xact_ttl_secs: Defines the time to live of transactions in seconds.

Note that xact_ttl_ms is dominant over xact_ttl_secs, that is, if xact_ttl_ms exists, even if xact_ttl_secs also exists, the value of xact_ttl_ms will be considered.

The xact_ttl_ms or xact_ttl_secs configuration usage syntax is:

xact_ttl_ms: 5000

or

xact_ttl_secs: 5

{
  "xact_ttl_ms": 5000
}

or

{
  "xact_ttl_secs": 5
}

Network (L2-L3) Analyzer (net)

Warning

Status: Beta. The metric names and configuration options may still change

Example of policy with input pcap and handler NET(v2) 
handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
  modules:
    default_net:
      type: net
      require_version: "2.0"
      config:
        deep_sample_rate: 1
        num_periods: 2
        topn_count: 25
      filter:
        geoloc_notfound: true
        asn_notfound: true
        only_geoloc_prefix:
          - BR
          - US/CA
        only_asn_number:
          - 7326
          - 16136
      metric_groups:
        disable:
          - cardinality
          - counters
          - top_geo
          - top_ips
          - quantiles
input:
  input_type: pcap
  tap_selector:
    any:
      - key1: value1
      - key2: value2
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10
    },
    "modules": {
      "default_net": {
        "type": "net",
        "require_version": "2.0",
        "config": {
          "deep_sample_rate": 1,
          "num_periods": 2,
          "topn_count": 25
        },
        "filter": {
          "geoloc_notfound": true,
          "asn_notfound": true,
          "only_geoloc_prefix": [
            "BR",
            "US/CA"
          ],
          "only_asn_number": [
            7326,
            16136
          ]
        },
        "metric_groups": {
          "disable": [
            "cardinality",
            "counters",
            "top_geo",
            "top_ips",
            "quantiles"
          ]
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap_selector": {
      "any": [
        {
          "key1": "value1"
        },
        {
          "key2": "value2"
        }
      ]
    },
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "kind": "collection"
}


Handler Type: "net"

Metrics Group - Check the net metrics belonging to each group

Metric Group Default
cardinality enabled
counters enabled
top_geo enabled
top_ips enabled
quantiles enabled


Filters

Filter Type Input
geoloc_notfound bool PCAP, DNSTAP
asn_notfound bool PCAP, DNSTAP
only_geoloc_prefix str[] PCAP, DNSTAP
only_asn_number str[] PCAP, DNSTAP

geoloc_notfound: bool
Back to net-v2 filters list

Input: PCAP

The source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. When the IPs refer to a region found in the standard databases, the city, state and country (approximated) are returned. However, when it is not possible to determine the IP geolocation, a not found is returned.

The geoloc_notfound filter usage syntax is:

geoloc_notfound: true

{
  "geoloc_notfound": true
}


asn_notfound: bool
Back to net-v2 filters list

Input: PCAP

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). When the IP of the source or destination belongs to some not known ASN in the standard databases, a not found is returned.

The asn_notfound filter usage syntax is:

asn_notfound: true

{
  "asn_notfound": true
}


only_geoloc_prefix: str[]
Back to net-v2 filters list

Input: PCAP

Source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. In this way it is possible to filter the data considering the geolocation using the filter only_geoloc_prefix.

The only_geoloc_prefix filter usage syntax is:

only_geoloc_prefix: 
  - str
  - str
Example: 
only_geoloc_prefix:
  - BR
  - US/CA


{
  "only_geoloc_prefix": [
    "str", "str"
  ]
}
Example: 
{
  "only_geoloc_prefix": [
    "BR",
    "US/CA"
  ]
}


only_asn_number: str[]
Back to net-v2 filters list

Input: PCAP

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). In this way it is possible to filter the data considering a specific ASN using the filter only_asn_number.

The only_asn_number filter usage syntax is:

only_asn_number:
  - str
  - str
Example: 
only_asn_number:
  - 7326
  - 16136


{
  "only_asn_number": [
    "str", "str"
  ]
}
Example: 
{
  "only_asn_number": [
    7326,
    16136
  ]
}

Example of policy with input pcap and handler NET(v1)
handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
  modules:
    default_net:
      type: net
      config:
        deep_sample_rate: 1
        num_periods: 2
        topn_count: 25
      filter:
        geoloc_notfound: true
        asn_notfound: true
        only_geoloc_prefix:
          - BR
          - US/CA
        only_asn_number:
          - 7326
          - 16136
      metric_groups:
        disable:
          - cardinality
          - counters
          - top_geo
          - top_ips
input:
  input_type: pcap
  tap_selector:
    any:
      - key1: value1
      - key2: value2
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10
    },
    "modules": {
      "default_net": {
        "type": "net",
        "config": {
          "deep_sample_rate": 1,
          "num_periods": 2,
          "topn_count": 25
        },
        "filter": {
          "geoloc_notfound": true,
          "asn_notfound": true,
          "only_geoloc_prefix": [
            "BR",
            "US/CA"
          ],
          "only_asn_number": [
            7326,
            16136
          ]
        },
        "metric_groups": {
          "disable": [
            "cardinality",
            "counters",
            "top_geo",
            "top_ips"
          ]
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap_selector": {
      "any": [
        {
          "key1": "value1"
        },
        {
          "key2": "value2"
        }
      ]
    },
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "kind": "collection"
}


Handler Type: "net"

Metrics Group - Check the net metrics belonging to each group

Metric Group Default
cardinality enabled
counters enabled
top_geo enabled
top_ips enabled


Filters

Filter Type Input
geoloc_notfound bool PCAP, DNSTAP
asn_notfound bool PCAP, DNSTAP
only_geoloc_prefix str[] PCAP, DNSTAP
only_asn_number str[] PCAP, DNSTAP

geoloc_notfound: bool
Back to net-v1 filters list

Input: PCAP

The source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. When the IPs refer to a region found in the standard databases, the city, state and country (approximated) are returned. However, when it is not possible to determine the IP geolocation, a not found is returned.

The geoloc_notfound filter usage syntax is:

geoloc_notfound: true

{
  "geoloc_notfound": true
}


asn_notfound: bool
Back to net-v1 filters list

Input: PCAP

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). When the IP of the source or destination belongs to some not known ASN in the standard databases, a not found is returned.

The asn_notfound filter usage syntax is:

asn_notfound: true

{
  "asn_notfound": true
}


only_geoloc_prefix: str[]
Back to net-v1 filters list

Input: PCAP

Source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. In this way it is possible to filter the data using the geolocation using the filter only_geoloc_prefix.
. The filter supports the following strings:

  • Continents: two-character continent code, as follows: AF - Africa AN - Antarctica AS - Asia EU - Europe NA - North America OC - Oceania SA - South America

  • Country: the two-character ISO 3166-1 country code

  • Subdivision: the region-portion of the ISO 3166-2 code for the region

    The only_geoloc_prefix filter usage syntax is:

    only_geoloc_prefix: 
  - str
  - str
    Example: 
    only_geoloc_prefix:
  - BR
  - US/CA


    {
  "only_geoloc_prefix": [
    "str", "str"
  ]
}
    Example: 
    {
  "only_geoloc_prefix": [
    "BR",
    "US/CA"
  ]
}


    only_asn_number: str[]
    Back to net-v1 filters list

    Input: PCAP

    Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). In this way it is possible to filter the data considering a specific ASN using the filter only_asn_number.

    The only_asn_number filter usage syntax is:

    only_asn_number:
  - str
  - str
    Example: 
    only_asn_number:
  - 7326
  - 16136


    {
  "only_asn_number": [
    "str", "str"
  ]
}
    Example: 
    {
  "only_asn_number": [
    7326,
    16136
  ]
}

Configurations

recorded_stream
Back to net configurations list

This configuration is useful when a pcap_file is used in taps/input configuration. Set it to True when you want to load an offline traffic (from a pcap_file).

The recorded_stream configuration usage syntax is:

recorded_stream: true

{
  "recorded_stream": true
}

DHCP Analyzer (dhcp)

Example of policy with input pcap and handler DHCP 
handlers:
  config:
    deep_sample_rate: 100
    num_periods: 8
    topn_count: 10
  modules:
    default_dhcp:
      type: dhcp
      config:
        deep_sample_rate: 1
        num_periods: 8
        topn_count: 25
input:
  input_type: pcap
  tap_selector:
    all:
      - key1: value1
      - key2: value
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 8,
      "topn_count": 10
    },
    "modules": {
      "default_dhcp": {
        "type": "dhcp",
        "config": {
          "deep_sample_rate": 1,
          "num_periods": 8,
          "topn_count": 25
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap_selector": {
      "all": [
        {
          "key1": "value1"
        },
        {
          "key2": "value"
        }
      ]
    },
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "kind": "collection"
}


Handler Type: "dhcp"

Metrics Group - Check dhcp metrics

  • No metrics group available

Filters

  • No filters available.

Configurations

BGP Analyzer (bgp)

Example of policy with input pcap and handler BGP 
handlers:
  config:
    deep_sample_rate: 100
    num_periods: 8
    topn_count: 10
  modules:
    default_bgp:
      type: bgp
      config:
        deep_sample_rate: 1
        num_periods: 8
        topn_count: 25
input:
  input_type: pcap
  tap_selector:
    all:
      - key1: value1
      - key2: value
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
config:
  merge_like_handlers: true        
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 8,
      "topn_count": 10
    },
    "modules": {
      "default_bgp": {
        "type": "bgp",
        "config": {
          "deep_sample_rate": 1,
          "num_periods": 8,
          "topn_count": 25
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap_selector": {
      "all": [
        {
          "key1": "value1"
        },
        {
          "key2": "value"
        }
      ]
    },
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "config": {
    "merge_like_handlers": true
  },
  "kind": "collection"
}


Handler Type: "bgp"

Metrics Group - Check BGP metrics

  • No metrics group available

Filters

  • No filters available.

Configurations

Packet Capture Analyzer (pcap)

Example of policy with input pcap and handler PCAP 
handlers:
  config:
    deep_sample_rate: 100
    num_periods: 8
    topn_count: 10
  modules:
    default_pcap:
      type: pcap
      config:
        deep_sample_rate: 6
        num_periods: 3
        topn_count: 25
input:
  input_type: pcap
  tap: default_pcap
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 8,
      "topn_count": 10
    },
    "modules": {
      "default_pcap": {
        "type": "pcap",
        "config": {
          "deep_sample_rate": 6,
          "num_periods": 3,
          "topn_count": 25
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap": "default_pcap",
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "kind": "collection"
}


Handler Type: "pcap"

Metrics Group - Check pcap metrics

  • No metrics group available.

Filters

  • No filters available.

Configurations

Flow Analyzer (flow) [BETA]

Warning

Status: Beta. The metric names and configuration options may still change

Example of policy with input flow and handler FLOW 
handlers:
    config:
        deep_sample_rate: 95
        num_periods: 6
        topn_count: 8
    modules:
        my_flow:
            type: flow
            config:
                sample_rate_scaling: false
                deep_sample_rate: 85
                num_periods: 5
                topn_count: 7
                first_filter_if_as_label: true
                enrichment: true
                device_map:
                  192.168.3.32:
                    name: Device1
                    description: This is a device map example
                    interfaces:
                      2:
                        name: Prov1
                        description: This is an interface map example
                summarize_ips_by_asn: true
                exclude_unknown_asns_from_summarization: true
                exclude_asns_from_summarization:
                  - 16509
                exclude_ips_from_summarization_flow:
                  - 192.168.3.32/32
            metric_groups:
                enable:
                    - cardinality
                    - counters
                    - top_geo
                    - by_packets
                    - by_bytes
                    - conversations
                    - top_ports
                    - top_ips
                    - top_interfaces
                    - top_ips_ports
                    - top_tos
            filter:
                only_ports:
                    - 10853
                    - 10860-10890
                only_device_interfaces:
                    - 216.239.38.10:
                        - 2
                only_directions: "in"
                only_ips:
                  - 192.168.2.1/24
                  - 192.158.1.38/32
                geoloc_notfound: true
                asn_notfound: true
input:
  input_type: flow
  tap: default_flow
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 95,
      "num_periods": 6,
      "topn_count": 8
    },
    "modules": {
      "my_flow": {
        "type": "flow",
        "config": {
          "sample_rate_scaling": false,
          "deep_sample_rate": 85,
          "num_periods": 5,
          "topn_count": 7,
          "first_filter_if_as_label": true,
          "enrichment": true,
          "device_map": {
            "192.168.3.32": {
              "name": "Device1",
              "description": "This is a device map example",
              "interfaces": {
                "2": {
                  "name": "Prov1",
                  "description": "This is an interface map example"
                }
              }
            }
          },
          "summarize_ips_by_asn": true,
          "exclude_unknown_asns_from_summarization": true,
          "exclude_asns_from_summarization": [
            16509
          ],
          "exclude_ips_from_summarization_flow": [
            "192.168.3.32/32"
          ]
        },
        "metric_groups": {
          "enable": [
            "cardinality",
            "counters",
            "top_geo",
            "by_packets",
            "by_bytes",
            "conversations",
            "top_ports",
            "top_ips",
            "top_interfaces",
            "top_ips_ports",
            "top_tos"
          ]
        },
        "filter": {
          "only_ports": [
            10853,
            "10860-10890"
          ],
          "only_device_interfaces": [
            {
              "216.239.38.10": [
                2
              ]
            }
          ],
          "only_directions": "in",
          "only_ips": [
            "192.168.2.1/24",
            "192.158.1.38/32"
          ],
          "geoloc_notfound": true,
          "asn_notfound": true
        }
      }
    }
  },
  "input": {
    "input_type": "flow",
    "tap": "default_flow"
  },
  "kind": "collection"
}


Handler Type: "flow"

Metrics Group - Check the flow metrics belonging to each group

Metric Group Default
cardinality enabled
counters enabled
by_packets enabled
by_bytes enabled
top_ips enabled
top_ports enabled
top_ips_ports enabled
top_geo disabled
conversations disabled
top_interfaces disabled
top_tos disabled

Filters

Filter Type Input
only_device_interfaces str[] FLOW
only_directions str FLOW
only_ips str[] FLOW
only_ports str[] FLOW
geoloc_notfound bool FLOW
asn_notfound bool FLOW

only_device_interfaces: str[]
Back to flow filters list

Input: FLOW

only_device_interfaces filters data by only retaining flows coming from the specific devices and interfaces defined in this filter.
The difference between only_device_interfaces and only_ips is that only_ips filters based on the IPs observed inside the flows, while only_device_interfaces filters based on the device and interface sending the flows.

The only_device_interfaces filter usage syntax is:

only_device_interfaces:
  - device:
    - interface
Example: 
only_device_interfaces:
  - 216.239.38.10:
    - 2 #port can be passed as int
    - 4-10 #port can be passed as range. Ports from 4 to 10: all ports in this interval will be accepted.
    - "1" #port can be passed as str
  - 192.158.1.38: [9, 4-10]
  - 192.168.2.32:
    - "*" #all ports


{
  "only_device_interfaces": [
    {
      "device": [
        "interface"
      ]
    }
  ]
}
Example: 
{
  "only_device_interfaces": [
    {
      "216.239.38.10": [
        2,
        "4-10",
        "10853"
      ]
    },
    {
      "192.158.1.38": [
        3,
        "10860-10890"
      ]
    },
    {
      "192.168.2.32": [
        "*"
      ]
    }
  ]
}


only_directions: str[]
Back to flow filters list

only_directions filters data by its direction. Options are: "in" and/or "out".

The only_directions filter usage syntax is:

only_directions: [str]
Example: 
only_directions:
    - "in"


{
  "only_directions": [
    "str"
  ]
}
Example: 
{
  "only_directions": [
    "in"
  ]
}


only_ips: str[]
Back to flow filters list

Input: FLOW

To filter data only from certain source OR destination, you can use only_ips filter, for which CIDR (Inter-Domain Routing Classes) ranges are supported.

The only_ips filter usage syntax is:

only_ips:
  - array
Example: 
only_ips:
  - 192.168.1.1/24
  - 192.158.1.38/32


{
  "only_ips": [
    "array"
  ]
}
Example: 
{
  "only_ips": [
    "192.168.2.1/24",
    "192.158.1.38/32"
  ]
}


only_ports: str[]
Back to flow filters list

Input: FLOW

only_ports filter only filters data being sent to or received on one of the selected TCP/UDP ports (or range of ports).

The only_ports filter usage syntax is:

only_ports:
  - array
Example: 
only_ports:
  - 10853 #port can be passed as int
  - "10854" #port can be passed as str
  - 10860-10890 #range from 10860 to 10890. All ports in this interval will be accepted


{
  "only_ports": [
    "array"
  ]
}
Example: 
{
  "only_ports": [
    10853,
    "10854",
    "10860-10890"
  ]
}


geoloc_notfound: bool
Back to flow filters list

Input: FLOW

The source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. When the IPs refer to a region found in the standard databases, the city, state and country (approximated) are returned. However, when it is not possible to determine the IP geolocation, a not found is returned.

The geoloc_notfound filter usage syntax is:

geoloc_notfound: true

{
  "geoloc_notfound": true
}


asn_notfound: bool
Back to flow filters list

Input: FLOW

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). When the IP of the source or destination belongs to some not known ASN in the standard databases, a not found is returned.

The asn_notfound filter usage syntax is:

asn_notfound: true

{
  "asn_notfound": true
}


Configurations

sample_rate_scaling
Back to flow configurations list

By default, flow metrics are generated by an approximation based on sampling the data. 1 packet every N is analyzed and the prediction of the entire population is made from the sample. If you want to see exactly all the exact data, you can disable sample_rate_scaling.

The sample_rate_scaling filter usage syntax is:

sample_rate_scaling: false

{
  "sample_rate_scaling": false
}

first_filter_if_as_label
Back to flow configurations list

This configuration requires the only_interfaces filter to be active (true). If this setting is true, the interfaces will be used as labels for the metrics.

The first_filter_if_as_label filter usage syntax is:

first_filter_if_as_label: true

{
"first_filter_if_as_label": true
}

enrichment
Back to flow configurations list

When true, uses device map settings.

The enrichment configuration usage syntax is:

enrichment: True

{
"enrichment": true
}

device_map
Back to flow configurations list

This configuration allows the user to assign a custom name to devices and interfaces, and the proper functioning of this configuration depends on the enrichment being True.

The device_map configuration usage syntax is:

device_map:
  device_ip:
    name: "str" #name is required
    description: "Optionally set a description"
    interfaces: #Interfaces are optional
        interface:
            name: "str" #required
            description: "Optionally set a description"
Example: 
device_map:
  192.168.2.32:
    name: Cisco
    description: This is a device map example
    interfaces:
      2:
        name: GoogleProv
        description: This is an interface map example


{
  "device_map": {
    "device_ip": {
      "name": "str",
      "description": "Optionally set a description",
      "interfaces": {
        "interface": {
          "name": "str",
          "description": "Optionally set a description"
        }
      }
    }
  }
}
Example: 
{
  "device_map": {
    "192.168.2.32": {
      "name": "Cisco",
      "description": "This is a device map example",
      "interfaces": {
        "2": {
          "name": "GoogleProv",
          "description": "This is an interface map example"
        }
      }
    }
  }
}

Summarization is a useful strategy for visualization, but also for decreasing the cardinality of the data, and two types of summarization are supported: by asn and by subnets, and summarization by ASN is dominant over subnet, i.e. If both configurations are present, only the IPs of an excluded asn or an unknown asn (if exclude_unknown_asns_from_summarization is true) will be summarized by subnet.

summarize_ips_by_asn
Back to flow configurations list

When True, it summarizes data by ASN (Autonomous System Number).

The summarize_ips_by_asn configuration usage syntax is:

summarize_ips_by_asn: true

{
  "summarize_ips_by_asn": true
}

exclude_asns_from_summarization
Back to flow configurations list

This configuration must be used in conjunction with summarize_ips_by_asn, in order to exclude ASNs from summarization. In this case, packets transacted by excluded ASNs will be exposed by IPs.

The exclude_asns_from_summarization configuration usage syntax is:

exclude_asns_from_summarization:
  - 8075
  - 16509

{
  "exclude_asns_from_summarization": [
    8075,
    16509
  ]
}

exclude_unknown_asns_from_summarization
Back to flow configurations list

This configuration must be used in conjunction with summarize_ips_by_asn, in order to expose IPs from packets transacted by unknown ASNs.

The exclude_unknown_asns_from_summarization configuration usage syntax is:

exclude_unknown_asns_from_summarization: true

{
  "exclude_unknown_asns_from_summarization": true
}

subnets_for_summarization
Back to flow configurations list

This configuration allows the summarization of flow data by subnets. Attention: This configuration will only work properly if the summarize_ips_by_asn configuration is not set for the IP, since summarize_ips_by_asn is dominant.

The subnets_for_summarization configuration usage syntax is:

subnets_for_summarization:
  - 192.168.2.1/24

{
  "subnets_for_summarization": [
    "192.168.2.1/24"
  ]
}

Tip

It is possible to define a summary pattern by defining the CIDR only. In this way, all present IPs will be summarized. For this to be done, just pass the default host mask for IPv4 or/and IPv6 and the desired CIDR for grouping. This pattern has less priority than the explicitly defined subnets, so only IPs not belonging to any set subnet will be summarized following the general pattern.

subnets_for_summarization:
  - 0.0.0.0/16
  - ::/64

{
  "subnets_for_summarization": [
    "0.0.0.0/16",
    "::/64"
  ]
}

exclude_ips_from_summarization_flow
Back to flow configurations list

This configuration must be used in conjunction with summarize_ips_by_asn or subnets_for_summarization and will remove the specified IPs from the summarization.

The exclude_ips_from_summarization_flow configuration usage syntax is:

exclude_ips_from_summarization_flow:
  - 192.168.2.1/31

{
  "exclude_ips_from_summarization_flow": [
    "192.168.2.1/31"
  ]
}

recorded_stream
Back to flow configurations list

This configuration is useful when a pcap_file is used in taps/input configuration. Set it to True when you want to load an offline traffic (from a pcap_file).

The recorded_stream configuration usage syntax is:

recorded_stream: true

{
  "recorded_stream": true
}

Netprobe [BETA]

Warning

Status: Beta. The metric names and configuration options may still change

Example of policy with input netprobe and handler NETPROBE 
handlers:
  modules:
    default_netprobe:
      type: netprobe
      metric_groups:
        enable:
          - counters
          - quantiles
          - histograms
      config:
        targets:
          www.google.com:
            target: www.google.com
          orb_community:
            target: orb.community
input:
  input_type: netprobe
  tap: default_netprobe
kind: collection

{
  "handlers": {
    "modules": {
      "default_netprobe": {
        "type": "netprobe",
        "metric_groups": {
          "enable": [
            "counters",
            "quantiles",
            "histograms"
          ]
        },
        "config": {
          "targets": {
            "www.google.com": {
              "target": "www.google.com"
            },
            "orb_community": {
              "target": "orb.community"
            }
          }
        }
      }
    }
  },
  "input": {
    "input_type": "netprobe",
    "tap": "default_netprobe"
  },
  "kind": "collection"
}


Handler Type: "netprobe"

Metrics Group - Check the netprobe metrics belonging to each group

Metric Group Default
quantiles disabled
counters enabled
histograms enabled

Filters

  • No filters available.

Configurations

Config Type Required Default
targets map ✅ -

targets: map
Back to netprobe configurations list

Here, the targets against which the probe will run are defined. For each target is required to specify the target name and the address to be tested.

targets: map
Example: 
targets:
  target_name:
    target: ipv4 address to test
Generic Example: 
targets:
  google:
    target: www.google.com

  • In netprobe policies it makes a lot of sense to use the settings from the input directly in the policy, since the settings are more related to the probe than the device the orb agent is running on. Therefore, it is worth reinforcing here the ability to override all tap settings in the policy. See here the available configurations for netprobe.
    handlers:
      modules:
        default_netprobe:
          type: netprobe
          metric_groups:
            enable:
              - counters
              - quantiles
              - histograms
          config:
            targets:
              www.google.com:
                target: www.google.com
              orb_community:
                target: orb.community
    input:
      input_type: netprobe
      tap: default_netprobe
      config:
        test_type: ping
        interval_msec: 2500
        timeout_msec: 2000
        packets_per_test: 5
        packets_interval_msec: 20
        packet_payload_size: 56
    kind: collection

    {
      "handlers": {
        "modules": {
          "default_netprobe": {
            "type": "netprobe",
            "metric_groups": {
              "enable": [
                "counters",
                "quantiles",
                "histograms"
              ]
            },
            "config": {
              "targets": {
                "www.google.com": {
                  "target": "www.google.com"
                },
                "orb_community": {
                  "target": "orb.community"
                }
              }
            }
          }
        }
      },
      "input": {
        "input_type": "netprobe",
        "tap": "default_netprobe",
        "config": {
          "test_type": "ping",
          "interval_msec": 2500,
          "timeout_msec": 2000,
          "packets_per_test": 5,
          "packets_interval_msec": 20,
          "packet_payload_size": 56
        }
      },
      "kind": "collection"
    }