Advanced Policies¶

An Orb policy for pktvisor can be written in either YAML or JSON, and has four top level sections: “input”, “handlers”, "config" and “kind”.

YAMLJSON

input: ..
config: ...
handlers: ...
kind: collection

{
  "input": {       
  },
  "config": {
},
  "handlers": {        
  },
  "kind": "collection"
}

Input section¶

The input section specifies what data streams the policy will be using for analysis, in other words, this specifies what data the agent should be listening in on, and is defined at the agent level.
3 types of input are supported: pcap, flow and dnstap. For each input type, specific configuration, filters and tags can be defined.

Required fields:
input_type - the type of input. This field will be validated with the type of tap indicated by the tap parameter or by the tap selector . If the types are incompatible, the policy will fail.

tap - the name given to this input in the tap/agent configuration or tap_selector - tags to match existing taps. If tap_selector is used, it can be chosen whether taps with any of the tags or with all tags will be attached.

Optional fields:
filter - to specify what data to include from the input
config - how the input will be used

Every configuration set at the input can be reset at the tap level, with the one set on the tap dominant over the one set on the input.

Default input structure¶

Using specific tap (check the application in a policy here):

YAMLJSON

input:
  tap: tap_name
  input_type: type_of_input
  filter:
    bpf: ...
  config:
    ...

{
  "input": {
    "tap": "tap_name",
    "input_type": "type_of_input",
    "filter": {
      "bpf": "..."
    },
    "config": "..."
  }
}

or using tap selector matching any (check the application in a policy here):

YAMLJSON

input:
  tap_selector:
    any:
      - key1: value1
      - key2: value2
  input_type: type_of_input
  filter:
    bpf: ...
  config:
    ...

{
  "input": {
    "tap_selector": {
      "any": [
        {
          "key1": "value1"
        },
        {
          "key2": "value2"
        }
      ]
    },
    "input_type": "type_of_input",
    "filter": {
      "bpf": "..."
    },
    "config": "..."
  }
}

or using tap selector matching all (check the application in a policy here):

YAMLJSON

input:
  tap_selector:
    all:
      - key1: value1
      - key2: value2
  input_type: type_of_input
  filter:
    bpf: ...
  config:
    ...

{
  "input": {
    "tap_selector": {
      "all": [
        {
          "key1": "value1"
        },
        {
          "key2": "value2"
        }
      ]
    },
    "input_type": "type_of_input",
    "filter": {
      "bpf": "..."
    },
    "config": "..."
  }
}

Packet Capture (pcap)¶

Configurations¶

There are 5 configurations for pcap input: pcap_file, pcap_source, iface, host_spec and debug.

Config	Type
pcap_file	str
pcap_source	str
iface	str
host_spec	str
debug	bool

pcap_file: str
One option of using pktvisor is for reading existing network data files. In this case, the path to the file must be passed. This variable is dominant, so if a file is passed, pktvisor will do the entire process based on the file.

YAMLJSON

 pcap_file: "path/to/file"

{
  "pcap_file": "path/to/file"
}

debug: bool

When true activate debug logs

YAMLJSON

debug: true

{
  "debug": true
}

pcap_source: str

pcap_source specifies the type of library to use. Default: libpcap. Options: libpcap or af_packet (linux).

YAMLJSON

pcap_source: "af_packet"

{
  "pcap_source": "af_packet"
}

iface: str

Name of the interface to bind. If the interface name does not exist, the policy will fail.

YAMLJSON

iface: str

Example:

iface: eth0

{
  "iface": "str"
}

Example:

{
  "iface": "eth0"
}

host_spec: str
The host_spec setting is useful to determine the direction of observed packets, once knowing the host ip, it is possible to determine the data flow direction, ie if they are being sent by the observed host (from host) or received (to host).

YAMLJSON

host_spec: str

Example:

host_spec: "192.168.0.1/24"

{
  "host_spec": "str"
}

Example:

{
  "host_spec": "192.168.0.1/24"
}

Filters¶

There is only one filter referring to the input PCAP: bpf.

bpf: str

filter data based on Berkeley Packet Filters (BPF).

YAMLJSON

bpf: str

Example:

bpf: "port 53"

{
  "bpf": "str"
}

Example:

{
  "bpf": "port 53"
}

Sflow/Netflow (flow)¶

Configurations¶

There are 4 configs for flow inputs: pcap_file, port, bind and flow_type. pcap_file and port+bind are mutually exclusive and one of them must exist.

Config	Type
pcap_file	str
port	int
bind	str
flow_type	str

pcap_file: str

One option of using pktvisor is for reading existing network data files. In this case, the path to the file must be passed. This variable is dominant, so if a file is passed, pktvisor will do the entire process based on the file.

YAMLJSON

 pcap_file: path/to/file

{
  "pcap_file": "path/to/file"
}

port: int and bind: str

The other option for using flow is specifying a port AND an ip to bind (only udp bind is supported). Note that, in this case, both variables must be set.

YAMLJSON

port: int
bind: str

Example:

port: 6343
bind: 192.168.1.1

{
  "port": "int",
  "bind": "str"
}

Example:

{
  "port": 6343,
  "bind": "192.168.1.1"
}

flow_type: str

Default: sflow. options: sflow or netflow (ipfix is supported on netflow).

YAMLJSON

flow_type: str

Example:

flow_type: netflow

{
  "flow_type": "str"
}

Example:

{
  "flow_type": "netflow"
}

Filters¶

There are no specific filters for the FLOW input.

Dnstap¶

Configurations¶

The 3 existing DNSTAP configurations (dnstap_file, socket and tcp) are mutually exclusive, that is, only one can be used in each input and one of them must exist. They are arranged in order of priority.

Config	Type
dnstap_file	str
socket	int
tcp	str

dnstap_file: str

One option of using pktvisor is for reading existing network data files. In this case, the path to the file must be passed. This variable is dominant, so if a file is passed, pktvisor will do the entire process based on the file.

YAMLJSON

dnstap_file: path/to/file

{
  "dnstap_file": "path/to/file"
}

socket: str

Path to socket file containing port and ip to bind

YAMLJSON

socket: path/to/file.sock

{
  "socket": "path/to/file.sock"
}

tcp: str

The other way to inform the ip and port to be monitored is through the 'tcp' configuration. Usage syntax is a string with port:ip (only ipv4 is supported for now).

YAMLJSON

tcp: ip:port

Example:

tcp: 192.168.8.2:235

{
  "tcp": "ip:port"
}

Example:

{
  "tcp": "192.168.8.2:235"
}

Filters¶

only_hosts: str

only_hosts filters data from a specific host.

YAMLJSON

only_hosts: str

Example:

only_hosts: 192.168.1.4/32

{
  "only_hosts": "str"
}

Example:

{
  "only_hosts": "192.168.1.4/32"
}

Configuration section¶

There is the possibility of defining settings on the policy level. Currently, the only configuration available is the merge_like_handlers.

Policy Configuration	Type	Default
`merge_like_handlers`	bool	false

merge_like_handlers

When merge_like_handlers config is true, metrics from all handlers of the same type are scraped together. This is useful when the tap_selector is used, as, by default, metrics are generated separately for each tap in the policy and this can be very expensive, depending on the number of taps.

The merge_like_handlers filter usage syntax is:

YAMLJSON

config:
  merge_like_handlers: true

{
  "config": {
    "merge_like_handlers": true
  }
}

Handlers section (Analysis)¶

Handlers are the modules responsible for extracting metrics from inputs. For each handler type, specific configuration, filters and group of metrics can be defined, and there are also configs (abstract configuration) that can be applied to all handlers:

Abstract Configurations¶

There are general configurations, which can be applied to all handlers. These settings can be reset for each module, within the specific module configs. In this case, the configuration inside the module will override the configuration passed in general handler.

Abstract Configuration	Type	Default
`deep_sample_rate`	int	100 (per second)
`num_periods`	int	5
`topn_count`	int	10
`topn_percentile_threshold`	int	0

deep_sample_rate

deep_sample_rate determines the number of data packets that will be analyzed deeply per second. Some metrics are operationally expensive to generate, such as metrics that require string parsing (qname2, qtype, etc.). For this reason, a maximum number of packets per second to be analyzed is determined. If in one second fewer packages than the maximum amount are transacted, all packages will compose the deep metrics sample, if there are more packages than the established one, the value of the variable will be used. Allowed values are in the range [1,100]. Default value is 100.

Note

If a value less than 1 is passed, the deep_sample_rate will be 1. If the value passed is more than 100, deep_sample_rate will be 100.

The deep_sample_rate usage syntax is:

YAMLJSON

deep_sample_rate: int

{
  "deep_sample_rate": int
}

num_periods

num_periods determines the amount of minutes of data that will be available on the metrics endpoint. Allowed values are in the range [2,10]. Default value is 5.

The num_periods usage syntax is:

YAMLJSON

num_periods: int

{
"num_periods": int
}

topn_count

topn_count sets the maximum amount of elements displayed in top metrics. If there is less quantity than the configured value, the composite metrics will have the existing value. But if there are more metrics than the configured value, the variable will be actively limiting. Any positive integer is valid and the default value is 10.

The topn_count usage syntax is:

YAMLJSON

topn_count: int

{
"topn_count": int
}

topn_percentile_threshold

topn_percentile_threshold sets the threshold of data to be considered based on the percentiles, so allowed values are in the range [0,100]. The default value is 0, that is, all data is considered. If, for example, the value 10 is set, scraped topn metrics will only consider data from the 10th percentile, that is, data between the highest 90%.

The topn_percentile_threshold usage syntax is:

YAMLJSON

topn_percentile_threshold: int

{
"topn_percentile_threshold": int
}

Default handler structure:

YAMLJSON

handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
    topn_percentile_threshold: 0
  modules:
    tap_name:
      type: ...
      config: ...
      filter: ...
      metric_groups:
        enable:
          - ...
          - ....
        disable:
          - .....
          - ......

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10,
      "topn_percentile_threshold": 0
    },
    "modules": {
      "tap_name": {
        "type": "...",
        "config": "...",
        "filter": "...",
        "metric_groups": {
          "enable": [
            "...",
            "...."
          ],
          "disable": [
            ".....",
            "......"
          ]
        }
      }
    }
  }
}

To enable any metric group use the syntax:

YAMLJSON

metric_groups:
  enable:
    - group_to_enable

{
  "metric_groups": {
    "enable": [
      "group_to_enable"
    ]
  }
}

In order to disable any metric group use the syntax:

YAMLJSON

metric_groups:
  disable:
    - group_to_disable

{
  "metric_groups": {
    "disable": [
      "group_to_disable"
    ]
  }
}

Attention: disabling is dominant over enabling. So if both are passed, the metric will be disabled;

DNS Analyzer (dns)¶

Handler Type: "dns"

Metrics Group
¶

Metric Group	Default
`top_ecs`	disabled
`cardinality`	enabled
`counters`	enabled
`dns_transaction`	enabled
`top_qnames`	enabled
`top_ports`	enabled

Configurations¶

public_suffix_list: bool.
Abstract configurations.

public_suffix_list

Some names to be resolved by a dns server have public suffixes. These suffixes cause metrics to be generated considering non-relevant data.

The example below illustrates the benefit of using this type of configuration. The qnames consider each part of the name to be resolved. When a name has a public suffix, generic information is generated. Note that in the standard configuration, Qname2 and Qname3 are the same for both domains. With the public suffix setting true (which makes the entire public part be considered as a single part), Qname3 already displays relevant information about the name.
The list of suffixes considered public can be accessed here.

Name	Qname2 Standard	Qname3 Standard	Qname2 Public Suffix	Qname3 Public Suffix
`www.imagine.qname.co.uk`	co.uk	qname.co.uk	qname.co.uk	imagine.qname.co.uk
`other.example.qname.co.uk`	co.uk	qname.co.uk	qname.co.uk	example.qname.co.uk

The public_suffix_list filter usage syntax is:

YAMLJSON

public_suffix_list: true

{
  "public_suffix_list": true
}

Filters
¶

Filter	Type	Input
`only_rcode`	int	PCAP
`exclude_noerror`	bool	PCAP
`only_dnssec_response`	bool	PCAP
`answer_count`	int	PCAP
`only_qtype`	str[]	PCAP
`only_qname_suffix`	str[]	PCAP
`geoloc_notfound`	bool	PCAP
`asn_notfound`	bool	PCAP
`dnstap_msg_type`	str	DNSTAP

only_rcode: int.

Input: PCAP

When a DNS server returns a response to a query made, one of the properties of the response is the "return code" (rcode), a code that describes what happened to the query that was made.

Most return codes indicate why the query failed and when the query succeeds, the return is an RCODE:0, whose name is NOERROR.
There are several possible return codes for a DNS server response, which you can access here, but currently the policies support 4 return types as filters (if you use any other code that is not in the table below, your policy will fail):

DNS Return Code	DNS Return Message	Description
`0`	NOERROR	DNS Query completed successfully
`2`	SERVFAIL	Server failed to complete the DNS request
`3`	NXDOMAIN	Domain name does not exist
`5`	REFUSED	Function not implemented

The only_rcode filter usage syntax is:

YAMLJSON

only_rcode: int

{
  "only_rcode": int
}

with the int referring to the return code to be filtered.

Example:
If you want to filter only successful queries responses you should use (note that all that the query will be discarded and the result will be just the responses):

YAMLJSON

only_rcode: 0

{
"only_rcode": 0
}

Important information is that only one return code is possible for each handler. So, in order to have multiple filters on the same policy, multiple handlers must be created, each with a rcode type.

exclude_noerror: bool

Input: PCAP

You may still want to filter out only responses with any kind of error. For this, there is the exclude_noerror filter, which removes from its results all responses that did not return any type of error. The exclude_noerror filter usage syntax is:

YAMLJSON

exclude_noerror: true

{
  "exclude_noerror": true
}

Attention: the filter of exclude_noerror is dominant in relation to the filter of only_rcode, that is, if the filter of exclude_noerror is true, even if the filter of only_rcode is set, the results will be composed only by responses without any type of error (all type of errors will be kept).

only_dnssec_response: bool

Input: PCAP

When you make a DNS query, the response you get may have a DNSSEC signature, which authenticates that DNS records originate from an authorized sender, thus protecting DNS from falsified information.
To filter only responses signed by an authorized sender, use: The only_dnssec_response filter usage syntax is:

YAMLJSON

only_dnssec_response: true

{
"only_dnssec_response": true
}

answer_count: int

Input: PCAP

One of the properties present in the query message structure is Answer RRs, which is the count of entries in the responses section (RR stands for “resource record”).
The number of answers in the query is always zero, as a query message has only questions and no answers, and when the server sends the answer to that query, the value is set to the amount of entries in the answers section.

The answer_count filter usage syntax is:

YAMLJSON

answer_count: int

{
  "answer_count": int
}

with the int referring to the desired amount of answer.
Note that any value greater than zero that is defined will exclude queries from the results, since in queries the number of answers is always 0.
As the answers count of queries is 0, whenever the value set for the answer_count is 0, both queries and responses will compose the result.

A special case is the concept of NODATA, which is one of the possible returns to a query made to a DNS server is known as. This happens when the query is successful (so rcode:0), but there is no data as a response, so the number of answers is 0.
In this case, to have in the results only the cases of NODATA, that is, the responses, the filter must be used together with the filter exclude_noerror.

Important information is that only one answer_count is possible for each handler. So, in order to have multiple counts on the same policy, multiple handlers must be created, each with an amount of answers.

only_qtype: str[]

Input: PCAP

DNS record types are records that provide important information about a hostname or domain. Supported default types can be seen here.

The only_qtype filter usage syntax is:

YAMLJSON

only_qtype: 
  - array

{
  "only_qtype": [
    "array"
  ]
}

If you want to filter only IPV4 record types, for example, you should use:

YAMLJSON

only_qtype:
  - "A"

{
  "only_qtype": [
    "A"
  ]
}

or

YAMLJSON

only_qtype:
  - 1

{
  "only_qtype": [
    1
  ]
}

Multiple types are also supported and both queries and responses that have any of the values in the array will be considered.

YAMLJSON

only_qtype:
  - 1
  - 2
  - "A"

{
  "only_qtype": [
    1,
    2,
    "A"
  ]
}

only_qname_suffix: str[]

Input: PCAP

The only_qname_suffix filters queries and responses whose endings (suffixes) of the names match the strings present in the array.
The only_qname_suffix filter usage syntax is:

YAMLJSON

only_qname_suffix:
  - array

{
  "only_qname_suffix": [
    "array"
  ]
}

Examples:

YAMLJSON

only_qname_suffix:
  - .google.com

{
  "only_qname_suffix": [
    ".google.com"
  ]
}

or

YAMLJSON

only_qname_suffix:
  - google.com
  - .nsone.net

{
  "only_qname_suffix": [
    "google.com",
    ".nsone.net"
  ]
}

geoloc_notfound: bool

Input: PCAP

Based on ECS (EDNS Client Subnet) information, it is possible to determine the geolocation of where the query is being made. When the Subnet refers to a region found in the standard databases, the city, state and country (approximated) are returned. However, when based on the subnet it is not possible to determine the geolocation, a not found is returned.
The geoloc_notfound filter only keeps responses whose geolocations were not found.
The geoloc_notfound filter usage syntax is:

YAMLJSON

geoloc_notfound: true

{
  "geoloc_notfound": true
}

asn_notfound: bool

Input: PCAP

Based on ECS (EDNS Client Subnet) information, it is possible to determine the ASN (Autonomous System Number). When the IP of the subnet belongs to some not known ASN in the standard databases, a not found is returned.
The asn_notfound filter only keeps responses whose asn were not found.
The asn_notfound filter usage syntax is:

YAMLJSON

asn_notfound: true

{
  "asn_notfound": true
}

dnstap_msg_type: str

Input: DNSTAP

With a dnstap protocol it is possible to know the type of message that must be resolved in the request to the server. This filter therefore allows you to filter by response types. Supported message types are: auth, resolver, client, forwarder, stub, tool and update.

The dnstap_msg_type filter usage syntax is:

YAMLJSON

dnstap_msg_type: str

Example:

dnstap_msg_type: auth

{
  "dnstap_msg_type": "str"
}

Example:

{
  "dnstap_msg_type": "auth"
}

Examples of DNS policy¶

Example policy pcap DNS:

YAMLJSON

handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
  modules:
    default_dns:
      type: dns
      config:
        public_suffix_list: true
        deep_sample_rate: 50
        num_periods: 2
        topn_count: 25
        topn_percentile_threshold: 10
      filter:
        only_rcode: 0
        only_dnssec_response: true
        answer_count: 1
        only_qtype: [1, 2]
        only_qname_suffix: [".google.com", ".orb.live"]
        geoloc_notfound: false
        asn_notfound: false
        dnstap_msg_type: "auth"
      metric_groups:
        enable:
          - top_ecs
        disable:
          - cardinality
          - counters
          - dns_transaction
          - top_qnames
          - top_ports
input:
  input_type: pcap
  tap: default_pcap
  filter:
    bpf: udp port 53
  config:
    iface: wlo1
    host_spec: 192.168.1.167/24
    pcap_source: libpcap
    debug: true
config:
    merge_like_handlers: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10
    },
    "modules": {
      "default_dns": {
        "type": "dns",
        "config": {
          "public_suffix_list": true,
          "deep_sample_rate": 50,
          "num_periods": 2,
          "topn_count": 25,
          "topn_percentile_threshold": 10
        },
        "filter": {
          "only_rcode": 0,
          "only_dnssec_response": true,
          "answer_count": 1,
          "only_qtype": [
            1,
            2
          ],
          "only_qname_suffix": [
            ".google.com",
            ".orb.live"
          ],
          "geoloc_notfound": false,
          "asn_notfound": false,
          "dnstap_msg_type": "auth"
        },
        "metric_groups": {
          "enable": [
            "top_ecs"
          ],
          "disable": [
            "cardinality",
            "counters",
            "dns_transaction",
            "top_qnames",
            "top_ports"
          ]
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap": "default_pcap",
    "filter": {
      "bpf": "udp port 53"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.167/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "config": {
    "merge_like_handlers": true
  },
  "kind": "collection"
}

Network (L2-L3) Analyzer (net)¶

Handler Type: "net"

Metrics Group
¶

Metric Group	Default
`cardinality`	enabled
`counters`	enabled
`top_geo`	enabled
`top_ips`	enabled

Configurations
¶

Abstract configurations.

Filters
¶

Filter	Type	Input
`geoloc_notfound`	bool	PCAP, DNSTAP
`asn_notfound`	bool	PCAP, DNSTAP
`only_geoloc_prefix`	str[]	PCAP, DNSTAP
`only_asn_number`	str[]	PCAP, DNSTAP

geoloc_notfound: bool

Input: PCAP

The source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. When the IPs refer to a region found in the standard databases, the city, state and country (approximated) are returned. However, when it is not possible to determine the IP geolocation, a not found is returned.

The geoloc_notfound filter usage syntax is:

YAMLJSON

geoloc_notfound: true

{
  "geoloc_notfound": true
}

asn_notfound: bool

Input: PCAP

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). When the IP of the source or destination belongs to some not known ASN in the standard databases, a not found is returned.

The asn_notfound filter usage syntax is:

YAMLJSON

asn_notfound: true

{
  "asn_notfound": true
}

only_geoloc_prefix: str[]

Input: PCAP

Source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. In this way it is possible to filter the data considering the geolocation using the filter only_geoloc_prefix.

The only_geoloc_prefix filter usage syntax is:

YAMLJSON

only_geoloc_prefix: 
  - array

Example:

only_geoloc_prefix:
  - BR
  - US/CA

{
  "only_geoloc_prefix": [
    "array"
  ]
}

Example:

{
  "only_geoloc_prefix": [
    "BR",
    "US/CA"
  ]
}

only_asn_number: str[]

Input: PCAP

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). In this way it is possible to filter the data considering a specific ASN using the filter only_asn_number.

The only_asn_number filter usage syntax is:

YAMLJSON

only_asn_number:
  - array

Example:

only_asn_number:
  - 7326
  - 16136

{
  "only_asn_number": [
    "array"
  ]
}

Example:

{
  "only_asn_number": [
    7326,
    16136
  ]
}

Examples of NET policy¶

Example policy pcap NET :

YAMLJSON

handlers:
  config:
    deep_sample_rate: 100
    num_periods: 5
    topn_count: 10
  modules:
    default_net:
      type: net
      config:
        deep_sample_rate: 1
        num_periods: 2
        topn_count: 25
      filter:
        geoloc_notfound: true
        asn_notfound: true
        only_geoloc_prefix:
          - BR
          - US/CA
        only_asn_number:
          - 7326
          - 16136
      metric_groups:
        disable:
          - cardinality
          - counters
          - top_geo
          - top_ips
input:
  input_type: pcap
  tap_selector:
    any:
      - key1: value1
      - key2: value2
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 5,
      "topn_count": 10
    },
    "modules": {
      "default_net": {
        "type": "net",
        "config": {
          "deep_sample_rate": 1,
          "num_periods": 2,
          "topn_count": 25
        },
        "filter": {
          "geoloc_notfound": true,
          "asn_notfound": true,
          "only_geoloc_prefix": [
            "BR",
            "US/CA"
          ],
          "only_asn_number": [
            7326,
            16136
          ]
        },
        "metric_groups": {
          "disable": [
            "cardinality",
            "counters",
            "top_geo",
            "top_ips"
          ]
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap_selector": {
      "any": [
        {
          "key1": "value1"
        },
        {
          "key2": "value2"
        }
      ]
    },
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "kind": "collection"
}

DHCP Analyzer (dhcp)¶

Handler Type: "dhcp"

Metrics Group¶

No metrics group available

Configurations
¶

Abstract configurations.

Filters¶

No filters available.

Example policy pcap dhcp JSON:

Examples of DHCP policy¶

Example policy pcap DHCP :

YAMLJSON

handlers:
  config:
    deep_sample_rate: 100
    num_periods: 8
    topn_count: 10
  modules:
    default_dhcp:
      type: dhcp
      config:
        deep_sample_rate: 1
        num_periods: 8
        topn_count: 25
input:
  input_type: pcap
  tap_selector:
    all:
      - key1: value1
      - key2: value
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 8,
      "topn_count": 10
    },
    "modules": {
      "default_dhcp": {
        "type": "dhcp",
        "config": {
          "deep_sample_rate": 1,
          "num_periods": 8,
          "topn_count": 25
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap_selector": {
      "all": [
        {
          "key1": "value1"
        },
        {
          "key2": "value"
        }
      ]
    },
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "kind": "collection"
}

BGP Analyzer (bgp)¶

Handler Type: "bgp"

Metrics Group¶

No metrics group available

Configurations
¶

Abstract configurations.

Filters¶

No filters available.

Example policy pcap bgp JSON:

Examples of BGP policy¶

Example policy pcap BGP :

YAMLJSON

handlers:
  config:
    deep_sample_rate: 100
    num_periods: 8
    topn_count: 10
  modules:
    default_bgp:
      type: bgp
      config:
        deep_sample_rate: 1
        num_periods: 8
        topn_count: 25
input:
  input_type: pcap
  tap_selector:
    all:
      - key1: value1
      - key2: value
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
config:
  merge_like_handlers: true        
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 8,
      "topn_count": 10
    },
    "modules": {
      "default_bgp": {
        "type": "bgp",
        "config": {
          "deep_sample_rate": 1,
          "num_periods": 8,
          "topn_count": 25
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap_selector": {
      "all": [
        {
          "key1": "value1"
        },
        {
          "key2": "value"
        }
      ]
    },
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "config": {
    "merge_like_handlers": true
  },
  "kind": "collection"
}

Packet Capture Analyzer (pcap)¶

Handler Type: "pcap"

Metrics Group¶

No metrics group available.

Configurations¶

Abstract configurations.

Filters¶

No filters available.

Examples of PCAP policy¶

Example policy pcap PCAP:

YAMLJSON

handlers:
  config:
    deep_sample_rate: 100
    num_periods: 8
    topn_count: 10
  modules:
    default_pcap:
      type: pcap
      config:
        deep_sample_rate: 6
        num_periods: 3
        topn_count: 25
input:
  input_type: pcap
  tap: default_pcap
  filter:
    bpf: net 192.168.1.0/24
  config:
    iface: wlo1
    host_spec: 192.168.1.0/24
    pcap_source: libpcap
    debug: true
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 100,
      "num_periods": 8,
      "topn_count": 10
    },
    "modules": {
      "default_pcap": {
        "type": "pcap",
        "config": {
          "deep_sample_rate": 6,
          "num_periods": 3,
          "topn_count": 25
        }
      }
    }
  },
  "input": {
    "input_type": "pcap",
    "tap": "default_pcap",
    "filter": {
      "bpf": "net 192.168.1.0/24"
    },
    "config": {
      "iface": "wlo1",
      "host_spec": "192.168.1.0/24",
      "pcap_source": "libpcap",
      "debug": true
    }
  },
  "kind": "collection"
}

Flow Analyzer (flow)¶

Handler Type: "flow"

Metrics Group
¶

Metric Group	Default
`cardinality`	enabled
`counters`	enabled
`top_geo`	enabled
`top_by_packets`	enabled
`top_by_bytes`	enabled

Configurations
¶

sample_rate_scaling: bool
recorded_stream: #todo
Abstract configurations.

sample_rate_scaling

By default, flow metrics are generated by an approximation based on sampling the data. 1 packet every N is analyzed and the prediction of the entire population is made from the sample. If you want to see exactly all the exact data, you can disable sample_rate_scaling.

The sample_rate_scaling filter usage syntax is:

YAMLJSON

sample_rate_scaling: false

{
  "sample_rate_scaling": false
}

Filters
¶

Filter	Type	Input
`only_devices`	str[]	FLOW
`only_ips`	str[]	FLOW
`only_ports`	str[]	FLOW
`only_interfaces`	str[]	FLOW
`geoloc_notfound`	bool	FLOW
`asn_notfound`	bool	FLOW

only_devices: str[]

Input: FLOW

Considering a flow traffic, the same port can be monitoring data from different routers or/and switches. If you want to filter the flow data coming from a specific device, the only_devices filter should be used. CIDR (Inter-Domain Routing Classes) ranges are supported on this filter.
The difference between only_devices and only_ips is that while this one filters the ids of the device that is sending the data, the other filters the ips that are communicating with each other (source or destination).

The only_devices filter usage syntax is:

YAMLJSON

only_devices:
  - array

Example:

only_devices:
  - 216.239.38.10/24
  - 192.158.1.38/32

{
  "only_devices": [
    "array"
  ]
}

Example:

{
  "only_devices": [
    "216.239.38.10/24",
    "192.158.1.38/32"
  ]
}

only_ips: str[]

Input: FLOW

To filter data only from certain source OR destination, you can use only_ips filter, for which CIDR (Inter-Domain Routing Classes) ranges are supported.

The only_ips filter usage syntax is:

YAMLJSON

only_ips:
  - array

Example:

only_ips:
  - 192.168.1.1/24
  - 192.158.1.38/32

{
  "only_ips": [
    "array"
  ]
}

Example:

{
  "only_ips": [
    "192.168.1.1/24",
    "192.158.1.38/32"
  ]
}

only_ports: str[]

Input: FLOW

only_ports filter only filters data being sent to or received on one of the selected IP ports (or range of ports).

The only_ports filter usage syntax is:

YAMLJSON

only_ports:
  - array

Example:

only_devices:
  - 10853 #port can be passed as int
  - "10854" #port can be passed as str
  - 10860-10890 #range from 10860 to 10890. All ports in this interval will be accepted

{
  "only_ports": [
    "array"
  ]
}

Example:

{
  "only_devices": [
    10853,
    "10854",
    "10860-10890"
  ]
}

only_interfaces: str

Input: FLOW

only_interfaces filters data by device ports (not IP ports). In this way, it is possible to filter data referring to a specific type of network connected to the corresponding device port.

The only_interfaces filter usage syntax is:

YAMLJSON

only_interfaces:
  - array

Example:

only_interfaces:
  - 10 #port can be passed as int
  - "11" #port can be passed as str
  - 12-16 #range from 12 to 16. All ports in this interval will be accepted

{
  "only_interfaces": [
    "array"
  ]
}

Example:

{
  "only_interfaces": [
    10,
    "11",
    "12-16"
  ]
}

geoloc_notfound: bool

Input: FLOW

The source and destination IPs are used to determine the geolocation to know where the data is from and where it is going. When the IPs refer to a region found in the standard databases, the city, state and country (approximated) are returned. However, when it is not possible to determine the IP geolocation, a not found is returned.

The geoloc_notfound filter usage syntax is:

YAMLJSON

geoloc_notfound: true

{
  "geoloc_notfound": true
}

asn_notfound: bool

Input: FLOW

Based on source and destination IP, it is possible to determine the ASN (Autonomous System Number). When the IP of the source or destination belongs to some not known ASN in the standard databases, a not found is returned.

The asn_notfound filter usage syntax is:

YAMLJSON

asn_notfound: true

{
  "asn_notfound": true
}

Examples of FLOW policy¶

Example policy input flow handler FLOW:

YAMLJSON

handlers:
    config:
        deep_sample_rate: 95
        num_periods: 6
        topn_count: 8
    modules:
        my_flow:
            type: flow
            config:
                sample_rate_scaling: false
                deep_sample_rate: 85
                num_periods: 5
                topn_count: 7
            metric_groups:
                disable:
                    - cardinality
                    - counters
                    - top_geo
                    - top_by_packets
                    - top_by_bytes
            filter:
                only_devices:
                    - 216.239.38.10/24
                    - 192.158.1.38/32
                only_ports:
                    - 10853
                    - 10860-10890
                only_interfaces:
                    - 10-16
                geoloc_notfound: true
                asn_notfound: true

input:
  input_type: flow
  tap: default_flow
kind: collection

{
  "handlers": {
    "config": {
      "deep_sample_rate": 95,
      "num_periods": 6,
      "topn_count": 8
    },
    "modules": {
      "my_flow": {
        "type": "flow",
        "config": {
          "sample_rate_scaling": false,
          "deep_sample_rate": 85,
          "num_periods": 5,
          "topn_count": 7
        },
        "metric_groups": {
          "disable": [
            "cardinality",
            "counters",
            "top_geo",
            "top_by_packets",
            "top_by_bytes"
          ]
        },
        "filter": {
          "only_devices": [
            "216.239.38.10/24",
            "192.158.1.38/32"
          ],
          "only_ports": [
            10853,
            "10860-10890"
          ],
          "only_interfaces": [
            "10-16"
          ],
          "geoloc_notfound": true,
          "asn_notfound": true
        }
      }
    }
  },
  "input": {
    "input_type": "flow",
    "tap": "default_flow"
  },
  "kind": "collection"
}

Advanced Policies¶

Input section¶

Default input structure¶

Packet Capture (pcap)¶

Configurations¶

Filters¶

Sflow/Netflow (flow)¶

Configurations¶

Filters¶

Dnstap¶

Configurations¶

Filters¶

Configuration section¶

Handlers section (Analysis)¶

Abstract Configurations¶

DNS Analyzer (dns)¶

Metrics Group ¶

Configurations¶

Filters ¶

Examples of DNS policy¶

Network (L2-L3) Analyzer (net)¶

Metrics Group ¶

Configurations ¶

Filters ¶

Examples of NET policy¶

DHCP Analyzer (dhcp)¶

Metrics Group¶

Configurations ¶

Filters¶

Examples of DHCP policy¶

BGP Analyzer (bgp)¶

Metrics Group¶

Configurations ¶

Filters¶

Examples of BGP policy¶

Packet Capture Analyzer (pcap)¶

Metrics Group¶

Configurations¶

Filters¶

Examples of PCAP policy¶

Flow Analyzer (flow)¶

Metrics Group ¶

Configurations ¶

Filters ¶

Examples of FLOW policy¶

Metrics Group
¶

Filters
¶

Metrics Group
¶

Configurations
¶

Filters
¶

Configurations
¶

Configurations
¶

Metrics Group
¶

Configurations
¶

Filters
¶