Kaspersky Unified Monitoring and Analysis Platform

Filtering and searching events

The Events section of the KUMA web interface does not show any data by default. To view events, you need to define an SQL query in the search field and click the Run query button. The SQL query can be entered manually or it can be generated using a query builder.

Data aggregation and grouping is supported in SQL queries.

You can search for events across multiple storages. For example, you can find events to determine where a user account is being blocked or which IP addresses were used to log in to which URLs. Example query for finding a blocked user account:

SELECT * FROM `events` WHERE DestinationUserName = 'username' AND DeviceEventClassID = '4625' LIMIT 250

To search for events in multiple storages, in the drop-down list in the upper-right part of the Events section, select check boxes next to the storages you want to search.

The list displays the following storages:

  • Storages of the Main tenant.
  • Available storages of tenants that satisfy one of the following conditions:
    • The tenant that owns the storage is enabled in the tenant filter and the user has permissions to read events in this tenant.
    • The user has access to the tenant of one of the partitions of the storage and has permissions to read events in this tenant.

      For example, if you have access to the collector tenant, but do not have access to the storage tenant, by default, the inaccessible tenant's storage is not displayed in the list of available storages. If a destination in the storage of a tenant that is not available to you is added to the collector of an available tenant, after an event arrives in the partition of the tenant of the collector, the storage of the unavailable tenant appears in the list of storages in the Events section.

The drop-down list of storages in the upper-right part of the Events section displays the name of the first of the selected storages and the number of selected storages, if there are several. You can hover over the drop-down list to display all of the selected storages. If a space in a storage is deleted, the storage drop-down list shows "deleted<ID of deleted space>", and the events remain searchable for the duration of the TTL.

The tenants selected in the tenant filter affect which storages are displayed in the drop-down list of storages. If you disable tenants whose storages are available to you in the tenant filter, these storages are no longer displayed in the drop-down list of storages. If these storages had been selected in the drop-down list of storages, their check boxes are cleared and events from these storages are not displayed. If only one storage is selected in the drop-down list of storages that is not from the Main tenant, and if in tenant selection you disabled the tenant that owns the selected storage, this storage is not displayed in the list of storages and KUMA automatically changes the selection to one of the storages of the Main tenant.

A simple query to all selected storages is allowed, as in the example above. If at least one of the selected storages is not available for the query, KUMA returns an error.

Limitations for searching events across multiple storages:

  • When querying multiple storages, export to TSV, retroscan, or REST API requests are not available.
  • A SELECT can contain only * and/or names of event fields. Aliases, functions, expressions are not allowed.
  • An ORDER BY clause must also contain only event fields (no functions, constants, expressions, and so on). If a field is not present in the list of fields for the SELECT, such a field is automatically added when sending to a specific cluster. You cannot set an ORDER BY ClusterID.
  • GROUP BY is not available.

Complex queries with grouping and aggregation are allowed for a single selected storage.

You can add filter conditions to an already generated SQL query in the window for viewing statistics, the events table, and the event details area:

  • Changing a query from the Statistics window

    To change the filtering settings in the Statistics window:

    1. Open Statistics details area by using one of the following methods:
      • In the MoreButton drop-down list in the top right corner of the events table select Statistics.
      • In the events table click any value and in the opened context menu select Statistics.

      The Statistics details area appears in the right part of the web interface window.

    2. Open the drop-down list of the relevant parameter and hover your mouse cursor over the necessary value.
    3. Use the plus and minus signs to change the filter settings by doing one of the following:
      • If you want the events selection to include only events with the selected value, click the filter-plus icon.
      • If you want the events selection to exclude all events with the selected value, click the filter-minus icon.

    As a result, the filter settings and the events table will be updated, and the new search query will be displayed in the upper part of the screen.

  • Changing a query from the events table

    To change the filtering settings in the events table:

    1. In the Events section of the KUMA web interface, click any event parameter value in the events table.
    2. In the opened menu, select one of the following options:
      • If you want the table to show only events with the selected value, select Filter by this value.
      • If you want to exclude all events with the selected value from the table, select Exclude from filter.

    As a result, the filter settings and the events table are updated, and the new search query is displayed in the upper part of the screen.

  • Changing a query from the Event details area

    To change the filter settings in the event details area:

    1. In the Events section of the KUMA web interface, click the relevant event.

      The Event details area appears in the right part of the window.

    2. Change the filter settings by using the plus or minus icons next to the relevant settings:
      • If you want the events selection to include only events with the selected value, click the filter-plus icon.
      • If you want the events selection to exclude all events with the selected value, click the filter-minus icon.

    As a result, the filter settings and the events table will be updated, and the new search query will be displayed in the upper part of the screen.

After modifying a query, all query parameters, including the added filter conditions, are transferred to the query builder and the search field.

When you switch to the query builder, the parameters of a query entered manually in the search field are not transferred to the builder, so you will need to create your query again. Also, the query created in the builder does not overwrite the query that was entered into the search string until you click the Apply query button in the builder window.

In the SQL query input field, you can enable the display of control characters.

You can also filter events by time period. Search results can be automatically updated.

The filter configuration can be saved. Existing filter configurations can be deleted.

Filter functions are available for users regardless of their roles.

When accessing certain event fields with IDs, KUMA returns the corresponding names.

For more details on SQL, refer to the ClickHouse documentation. For SQL operators and functions supported in KUMA, see also the KUMA operator usage and supported functions.

In this section

Selecting Storage

Generating an SQL query using a builder

Manually creating an SQL query

Saving query history

Managing saved search queries

Filtering events by period

Grouping events

Displaying names instead of IDs

Presets

Limiting the complexity of queries in alert investigation mode

Saving and selecting events filter configuration

Deleting event filter configurations

Supported ClickHouse functions

Viewing event detail areas

Exporting events

Configuring the table of events

Refreshing events table

Getting events table statistics

Viewing correlation event details

Generating an SQL query using KUMA SQL functions

See also:

About events

Storage

Page top
[Topic 228277]

Selecting Storage

Events that are displayed in the Events section of the KUMA web interface are retrieved from storage (from the ClickHouse cluster). Depending on the demands of your company, you may have more than one Storage. However, you can only receive events from one Storage at a time, so you must specify which one you want to use.

To select the Storage you want to receive events from,

In the Events section of the KUMA web interface, open the drop-down list of storages in the upper-right part of the section and select one or more spaces of one or more storages. You can select all available storage spaces by selecting the check box next to the storage name.

Now events from the selected storage are displayed in the events table. The name of the selected storage is displayed in the drop-down list of storages.

The drop-down list of storages displays only the clusters of tenants available to the user, and the cluster of the main tenant.

See also:

Storage

Page top
[Topic 217994]

Generating an SQL query using a builder

In KUMA, you can use a query builder to generate an SQL query for filtering events.

To generate an SQL query using a builder:

  1. In the Events section of the KUMA web interface, click the parent-category button.

    The filter constructor window opens.

  2. Generate a search query by providing data in the following parameter blocks:

    SELECT—event fields that should be returned. The * value is selected by default, which means that all available event fields must be returned. To make viewing the search results easier, select the necessary fields in the drop-down list. In this case, the data only for the selected fields is displayed in the table. Note that Select * increases the duration of the request execution, but eliminates the need to manually indicate the fields in the request.

    When selecting an event field, you can use the field on the right of the drop-down list to specify an alias for the column of displayed data, and you can use the right-most drop-down list to select the operation to perform on the data: count, max, min, avg, sum.

    If you use aggregation functions in an SQL query, you can customize the display of columns in the event table. After executing the query without *, the columns will match the query. Sorting of events in ascending and descending order, getting statistics is also not possible.

    If the query uses grouping or aggregation functions, instead of one number that was received as a response to the SELECT max(BytesOut) FROM `events` LIMIT 250 query, the radar shows a distribution of events from which this number is obtained.

    When filtering by alert-related events in alert investigation mode, you cannot perform operations on the data of event fields or assign names to the columns of displayed data.

    • FROM—data source. Select the events value.
    • WHERE—conditions for filtering events.

      Conditions and groups of conditions can be added by using the Add condition and Add group buttons. The AND operator value is selected by default in the group of conditions, but you can change the operator by clicking it. Available values: AND, OR, NOT. The structure of conditions and condition groups can be changed by using the DragIcon icon to drag and drop expressions.

      Adding filter conditions:

      1. In the drop-down list on the left, select the event field that you want to use for filtering.
      2. Select the necessary operator from the middle drop-down list. The available operators depend on the type of value of the selected event field.
      3. Enter the value of the condition. Depending on the selected type of field, you may have to manually enter the value, select it from the drop-down list, or select it on the calendar.

      You can delete filter conditions and filter condition groups by clicking cross.

    • GROUP BY—event fields or aliases to be used for grouping the returned data.

      If you use data grouping in an SQL query, custom fields specified in the query are displayed in the list of events. Table columns in a group cannot be customized. In group events, you can customize the display of columns. Sorting of events in ascending and descending order, getting statistics is also not possible.

      When filtering by alert-related events in alert investigation mode, you cannot group the returned data.

    • ORDER BY—columns used as the basis for sorting the returned data. In the drop-down list on the right, you can select the necessary order: DESC—descending, ASC—ascending.
    • LIMIT—number of strings displayed in the table.

      The default value is 250.

      If you are filtering events by user-defined period and the number of strings in the search results exceeds the defined value, you can click the Show next button to display additional strings in the table. This button is not displayed when filtering events by the standard period.

  3. Click Apply query.

    The current SQL query will be overwritten. The generated SQL query is displayed in the search field.

    If you want to reset the builder settings, click the Default query button.

    If you want to close the builder without overwriting the existing query, click the close_sql button in the upper-right part of the query creation window.

  4. Click the Run query button to display the data in the table.

The table will display the search results based on the generated SQL query.

When switching to another section of the web interface, the query generated in the builder is not preserved. If you return to the Events section from another section, the builder will display the default query.

For more details on SQL, refer to the ClickHouse documentation. See also KUMA operator usage and supported functions.

See also:

Manually creating an SQL query

About events

Storage

Page top
[Topic 228337]

Manually creating an SQL query

You can use the search field manually to create SQL queries of any complexity to filter events.

To manually generate an SQL query:

  1. Go to the Events section of the KUMA web interface.

    An input form opens.

  2. Enter your SQL query into the input field. You must use single quotes in your queries.
  3. Click Run query to run the query.

A table of events that satisfy the criteria of your query will be displayed. If necessary, you can filter events by period.

Supported functions and operators

Function

Description

SELECT

Event fields that you want to be returned. The following functions and operators are supported:

  • Aggregation functions: count, avg, max, min, sum.
  • Arithmetic operators: +, -, *, /, <, >, =, !=, >=, <=.

You can combine functions and operators in an SQL query. If you use aggregation functions in an SQL query, you can customize the display of columns in the event table. After executing the query without *, the columns will match the query. Sorting of events in ascending and descending order, getting statistics is also not possible.

If the query uses grouping or aggregation functions, instead of one number that was received as a response to the SELECT max(BytesOut) FROM `events` LIMIT 250 query, the radar shows a distribution of events from which this number is obtained.

FROM

Data source.

When creating an SQL query, you need to specify the events value as the data source.

WHERE

Conditions for filtering events:

  • AND, OR, NOT, =, !=, >, >=, <, <=
  • IN
  • BETWEEN
  • LIKE
  • ILIKE
  • inSubnet
  • match (the re2 syntax of regular expressions is used in SQL queries) You must additionally escape special characters with the backslash \.

GROUP BY

Event fields or aliases to be used for grouping the returned data.

If you use data grouping in an SQL query, custom fields specified in the query are displayed in the list of events. Table columns in a group cannot be customized. In group events, you can customize the display of columns. Sorting of events in ascending and descending order, getting statistics is also not possible.

ORDER BY

Columns by which you want to sort the returned data. Possible values:

  • DESC to sort in descending order
  • ASC to sort in ascending order

OFFSET

The number of rows to skip before displaying the results of the SQL query.

LIMIT

The number of rows that can be displayed in the table. The default value is 250.

If you are filtering events by user-defined period and the number of rows in the search results exceeds the defined value, you can click the Show next button to display additional rows in the table. This button is not displayed when filtering events by the standard period.

Examples of SQL queries

  • SELECT * FROM `events` WHERE Type IN ('Base', 'Audit') ORDER BY Timestamp DESC LIMIT 250

    In the events table, all events with the Base and Audit type are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.

  • SELECT * FROM `events` WHERE BytesIn BETWEEN 1000 AND 2000 ORDER BY Timestamp ASC LIMIT 250

    All events of the events table for which the BytesIn field contains a value of received traffic in the range from 1,000 to 2,000 bytes are sorted by the Timestamp column in ascending order. The number of strings that can be displayed in the table is 250.

  • SELECT * FROM `events` WHERE Message LIKE '%ssh:%' ORDER BY Timestamp DESC LIMIT 250

    In the events table, all events whose Message field contains data corresponding to the defined %ssh:% template in lowercase are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.

  • SELECT * FROM `events` WHERE inSubnet(DeviceAddress, '00.0.0.0/00') ORDER BY Timestamp DESC LIMIT 250

    In the events table, all events for the hosts that are in the 00.0.0.0/00 subnet are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.

  • SELECT * FROM `events` WHERE match(Message, 'ssh.*') ORDER BY Timestamp DESC LIMIT 250

    In the events table, all events whose Message field contains text corresponding to the ssh.* template are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.

  • SELECT max(BytesOut) / 1024 FROM `events`

    Maximum amount of outbound traffic (KB) for the selected time period.

  • SELECT count(ID) AS "Count", SourcePort AS "Port" FROM `events` GROUP BY SourcePort ORDER BY Port ASC LIMIT 250

    Number of events and port number. Events are grouped by port number and sorted by the Port column in ascending order. The number of strings that can be displayed in the table is 250.

    The ID column in the events table is named Count, and the SourcePort column is named Port.

If you want to use a special character in a query, you need to escape this character by placing a backslash (\) character in front of it.

Example:

SELECT * FROM `events` WHERE match(Message, 'ssh:\'connection.*') ORDER BY Timestamp DESC LIMIT 250

In the events table, all events whose Message field contains text corresponding to the ssh: 'connection' template are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.

When creating a normalizer for events, you can choose whether to retain the field values of the raw event. The data is stored in the Extra event field. This field is searched for events by using the LIKE operator.

Example:

SELECT * FROM `events` WHERE DeviceAddress = '00.00.00.000' AND Extra LIKE '%"app":"example"%' ORDER BY Timestamp DESC LIMIT 250

In the events table, all events for hosts with the IP address 00.00.00.000 where the example process is running are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.

If you created an SQL query manually in the search field and then switched to the builder, the SQL query parameters are not transferred to the builder. In this case, you will need to re-create the SQL query in the builder. The SQL query created in the builder does not overwrite the SQL query that was entered into the search string until you click the Apply query button in the builder window. If you created an SQL query in the query builder and then switched to the search field, the query parameters are transferred automatically.

Aliases must not contain spaces.

For more details on SQL, refer to the ClickHouse documentation. See also the supported ClickHouse functions.

See also:

Generating an SQL query using a builder

Limiting the complexity of queries in alert investigation mode

About events

Storage

Page top
[Topic 228356]

Saving query history

In the KUMA web interface, the Events section keeps a personal history of SQL search queries. You can refer to the history of queries and quickly find a query you have used in the past.

The queries are saved in the browser. Changing your browser resets the query history. The history stores the last 32 of your queries. Queries are displayed with formatting. The date and time when the query was executed is displayed.

Only queries with correct syntax are saved in the query history.

To find a search query in history:

  1. In the Events section, place the cursor in the query bar.
  2. Click the icon_request history (SQL query history) button in the toolbar of the query window or press Ctrl+] or Ctrl+[.

    This opens the query history window. The last query you ran is displayed at the top of the list of queries.

  3. Select a search query. To navigate the list of saved queries, press Ctrl+] or Ctrl+[.

    The selected query is pasted to the search bar.

  4. If necessary, run the search query. To do so, press Ctrll+Enter.

When you run a query, the query history window is closed.

You can also close the query history window by pressing the Esc key; this moves the cursor to the query editing window.

You can copy the query to the clipboard by pressing Ctrl+C. This copies the part of the query that is highlighted with the mouse, or the entire query string, limited by the box, if there was no selection.

You can save the query by pressing Ctrl+S, or by clicking the icon_save button (Save query) in the row. This closes the history window and opens the sidebar for saving the query with populated fields.

You can insert a query into the query window without running it by clicking the icon_enter request (Insert query) button, or by pressing Enter.

Page top
[Topic 293580]

Managing saved search queries

In KUMA, in the Events section, you can organize saved SQL queries in a tree of folders for structured storage and quick search of SQL queries. You can edit previously saved queries, rename them, arrange queries in groups (folders) of accessible tenants, and search for previously saved queries in the search bar.

Saved queries follow the tenant access model and are visible to all users that have access to the corresponding tenants.

Saved queries are placed in the Saved queries pane. You can open or close the Saved queries pane using the (bookmark) button on the toolbar of the query window.

The Saved queries navigation pane on the left contains the following:

  • The query search window where you can search by folder name and query name.
  • Tree of folders and queries, the Favorites folder.
  • The query filter that displays the queries you have created (on the My tab) and all queries (on the All tab)

Displaying a query in the query window

To display a query in the query window:

  1. In the Events section, click the (bookmark) button.

    This opens the Saved queries navigation pane.

  2. In the tree of folders in the Saved queries pane, click to select the relevant query.

The text of the selected query is displayed in the query window.

Saving a query to a folder or a tenant

In the Events section, after entering a query in the query window and clicking the Save current query button (floppy disk icon), you can save the SQL query to one of the folders or tenants that you can select.

To save a query to a folder:

  1. In the New query window, in the Filter name field, enter a name for the filter.

    Consider the following when naming the filter:

    • The name can contain Unicode characters.
    • The minimum length of a name is 1 character, and the maximum length is 128 characters.
    • Names must be unique within a tenant.
    • Tab characters, new line characters, and paragraph characters are replaced with space characters.

    When saving the name:

    • Leading and trailing spaces are removed.
    • Multiple spaces between characters are replaced with a single space character.
  2. In the Query field, enter the text of the SQL query.
  3. Select a tenant, a folder from the list of folders created in the tenant that is available to you, or create a new folder by clicking the Add folder button.

    You can also add a folder in the Saved queries window: click the AD_plus button next to the tenant to open the New folder window and add the folder in that window.

  4. Click the Save button.

The query is saved in the selected folder or tenant.

Viewing the text of a query

To view the text of a query:

  1. In the Events section, click the (bookmark) button.

    This opens the Saved queries navigation pane.

  2. In the tree of folders in the Saved queries pane, click the icon_viewing request button next to the relevant query.

This opens a window with the text of the query.

Setting the default query

You can set your default query that the Events section displays when opened.

To set the default query:

  1. In the Events section, click the (bookmark) button.

    This opens the Saved queries navigation pane.

  2. In the tree of folders in the Saved queries pane, select the relevant query.
  3. Click the button and in the displayed menu, select Use by default.

The selected query becomes the default query.

Adding folders and queries to favorites

To add a folder or individual query to favorites:

  1. In the Events section, click the (bookmark) button.

    This opens the Saved queries navigation pane.

  2. In the tree of folders in the Saved queries pane, select the relevant folder or query.
  3. Click the button and in the displayed menu, select Add to favorites.

The selected folder or resource is added to the Favorites folder.

Editing a query

To edit a query:

  1. In the Events section, click the (bookmark) button.

    This opens the Saved queries navigation pane.

  2. In the tree of folders in the Saved queries pane, select the relevant query.
  3. Click the button and in the displayed menu, select Edit.

    This opens the window for editing the saved query.

  4. In the window, edit the query name or text, or the folder in which you want to save the query.
  5. Click the Save button.

The query is updated and saved in the selected folder.

Deleting a query

To delete a query from a folder (if you have sufficient rights):

  1. In the Events section, click the bookmark () icon.

    This opens the Saved queries navigation pane.

  2. In the tree of folders in the Saved queries pane, select the relevant query.
  3. Click the three-dot icon () and in the menu that appears, select Delete.
  4. Confirm deletion in the displayed window.

The query is deleted.

Page top
[Topic 293664]

Filtering events by period

In KUMA, you can specify the time period to display events from.

To filter events by period:

  1. In the Events section of the KUMA web interface in the upper part of the window, open the time period drop-down list in the top right part of the section.
  2. Select a period in one of the following ways:
    • If you want to specify an exact date, in the calendar on the left, select the start and end date of the period and click Apply.

      You can select a date up to and including the current date. The date and time format depends on your browser settings. If the Date from or Date to field has a value and you have not edited the time value manually, when you select a date in the calendar, the Date from field is automatically populated with 00:00:00.000, and the Date to field with 23:59:59.999. If you have manually deleted the value in the Date from or Date to field, when you select a date in the calendar, the field is automatically populated with the current time. After you select a value in one of the fields, the focus switches to the other field. If your Date to is earlier than your Date from, this earlier value is automatically inserted into the Date from field.

    • If you want to specify a relative period, select one of the available periods in the Relative period list on the right.

      The period is calculated relative to the current time.

    • If you want to specify a custom period, edit the value of the Date from and Date to fields.

      You can enter an exact date and time in the DD.MM.YYYY HH:mm:ss.SSS format for the Russian localization and YYYY-MM-DD HH:mm:ss.SSS for the English localization or a period relative to the current time as a formula. You can also combine these methods if necessary. If you do not specify milliseconds when entering the exact date, 000 is substituted automatically. If you have edited the time in the Date from or Date to fields, picking a date in the calendar does not change the time component.

      In the relative period formulas, you can use the now parameter for the current date and time and the interval parameterization language: + (only in the Date to field), -, / (rounding to the nearest), as well as time units: y (year), M (month), w (week), d (day), h (hour), m (minute), s (second). For example, you can specify the period now-5d to get data for the last five days, or now/w to get data from the beginning of the first day of the current week (00:00:00:000 UTC) to the current time (now).

      The Date from field is required, and its value cannot exceed the value of the Date from field, and also cannot be earlier than 1970-01-01 (if specifying an exact date or a relative period). The Date to cannot be earlier than the Date from. If you do not specify a value in the Date from field, now is specified automatically.

    The default setting is 5 minutes (now-5m). The bounds of the period are inclusive: for example, for the Today time range, events are displayed from the beginning (00:00:00:000 UTC) of the current day to the current time (now) inclusive, and for the Yesterday period, events are displayed from the beginning (00:00:00:000 UTC) of the previous day to 00:00:00:000 UTC of the current day.

    KUMA stores time values in UTC, but in the user interface time is converted to the time zone of your browser. This is relevant to the relative periods: Today, Yesterday, This week, and This month. For example, if the time zone in your browser is UTC+3, and you select the Today period, events are displayed for the period from 03:00:00.000 until now, not from 00:00:00.000 until now.

    If, when selecting a relative period, such as Today, Yesterday, This week, or This month, you want to take your time zone into account when displaying events, you need to manually add a time offset in the Date from and Date to fields (if a value other than now is specified) by adding or subtracting the correct number of hours. For example, if your browser's time zone is UTC+3 and you want to display data for Yesterday, you need to change Date from to now-1d/d-3h and Date to to now/d-3h. If you want to display data for the Today period, you only need to change the value in the Date from field to now/d-3h.

    If you need results up to 23:59:59:999 of yesterday, you can use an SQL query with a filter by Timestamp or specify an exact date and time.

  3. If you want to capture the last relative period for which a query was executed as an exact date and time, click the button.

    The relative period is then converted to an exact date and time. Hovering over this button displays the date and time that will be captured. Relative periods (Today, Yesterday, This week, This month) are converted to an exact UTC date and time, other periods are converted to a date and time that takes into account the time zone of the browser.

    If you have executed an SQL query with event grouping, when switching between groups in the query results, the relative time range is automatically converted to an exact date and time.

  4. Click Run query to run the query.

Only events registered during the specified time period are displayed. The period will be displayed in the upper part of the window.

You can also configure the display of events by using the events histogram that is displayed when you click the Histogram icon button in the upper part of the Events section. Events are displayed if you click the relevant data column or select the relevant time period and click the Show events button.

Page top
[Topic 217877]

Grouping events

After getting a list of events, you often need to split the events into groups to localize an information security event. KUMA can group events in a list by one or more fields.

To group events, you no longer need to manually edit the text of the query; instead, you can click a column heading in the Events section and select Add GROUP BY to the query in the context menu. You can select a sequence of multiple fields to group by, and the fields will be automatically added to the query string. Having selected your fields, click Run query. As a result, events are grouped by the specified fields. Found groups are displayed in the Groups section. They can be displayed as a table and as cards. You can toggle between the display modes.

You can exclude a group from the query:

  • In Cards mode, click the - button.
  • In Tables mode, right-click the group and in the context menu, select Exclude group from filter.

As a result, the query is automatically modified and the group is excluded from the query.

If you want to go back to the original query, click Revert to original query.

You can navigate through the groups and view the contents of each group.

You can do a global search in all groups or a local search in events within a selected group.

You can use more complex grouping by adding one or more fields.

You can remove a group from the grouping and in this way, go back one step.

If the grouping query returns many events, only the first 1000 events are displayed. If the query contains SELECT Count(ID), you can click the link with the total number of events in the query result to view all events. If the request does not contain Count(ID), the number of events in the group is not indicated, but you still can click the link and view the total number of events in the group.

Statistics, retrospective check by group, and export to TSV are available.

If you want the grouping result to be independent of time (because events arrive continuously), you can set a fixed relative interval and apply it as an absolute interval so that the events of interest do not drop out of the selection. To fix a relative interval, in the Events section, in the time interval drop-down list, select Apply current range. You can now manage groups within this query.

If you want to arrange the selected events by months, days, minutes, and seconds, you can group events by the Timestamp field. To group events, select a grouping option in the context menu of the Timestamp field in the event table.

If you want to normalize the value of the Timestamp field and display the time values from different sources in the same UTC time scale, select Convert to UTC in the context menu of the Timestamp field in the events table.

Page top
[Topic 276595]

Displaying names instead of IDs

When accessing certain event fields with IDs, KUMA returns the corresponding names rather than IDs. This helps make the information more readable. For example, if you access the TenantID event field (which stores the tenant ID), you get the value of the TenantName event field (which stores the tenant name).

When exporting events, values of both fields are written to the file, the ID as well as the name.

The table below lists the fields that are substituted when accessed:

Requested field

Returned field

TenantID

TenantName

SeriviceID

ServiceName

DeviceAssetID

DeviceAssetName

SourceAssetID

SourceAssetName

DestinationAssetID

DestinationAssetName

SourceAccountID

SourceAccountName

DestinationAccountID

DestinationAccountName

Substitution does not occur if an alias is assigned to the field in the SQL query. Examples:

  • SELECT TenantID FROM `events` LIMIT 250 — in the search result, the name of the tenant is displayed in the TenantID field.
  • SELECT TenantID AS Tenant_name FROM `events` LIMIT 250 — in the search result, the tenant ID will be displayed in the Tenant_name field.
Page top
[Topic 255487]

Presets

You can use

to simplify work with queries if you regularly view data for a specific set of event fields. In the line with the SQL query, you can type Select * and select a saved preset; in that case, the output is limited only to the fields specified in the preset. This method slows down performance but eliminates the need to write a query manually every time.
Presets are saved on the KUMA Core server and are available to all KUMA users of the specified tenant.

To create a preset:

  1. In the Events section, click the icon.
  2. In the window that opens, on the Event field columns tab, select the required fields.

    To simplify your search, you can start typing the field name in the Search area. 

  3. To save the selected fields, click Save current preset.

    The New preset window opens.

  4. In that window, specify the Name of the preset, and in the drop-down list, select the Tenant.
  5. Click Save.

    The preset is created and saved.

To apply a preset:

  1. In the query entry field, enter Select *.
  2. In the Events section of the KUMA web interface, click the icon.
  3. In the opened window, use the Presets tab to select the relevant preset and click the apply_preset button.

    The fields from the selected preset are added to the SQL query field, and the columns are added to the table. No changes are made in Builder.

  4. Click Run query to run the query.

    After the query execution completes, the columns are filled in.

Page top
[Topic 242466]

Limiting the complexity of queries in alert investigation mode

When investigating an alert, the complexity of SQL queries for event filtering is limited if Related to alert is selected in the drop-down list of event sources. If this is the case, only the functions and operators listed below are available for event filtering.

If All events is selected from the drop-down list of event sources, these limitations do not apply.

  • SELECT
    • The * character is used as a wildcard to represent any number of characters.
  • WHERE
    • AND, OR, NOT, =, !=, >, >=, <, <=
    • IN
    • BETWEEN
    • LIKE
    • inSubnet

    Examples:

    • WHERE Type IN ('Base', 'Correlated')
    • WHERE BytesIn BETWEEN 1000 AND 2000
    • WHERE Message LIKE '%ssh:%'
    • WHERE inSubnet(DeviceAddress, '10.0.0.1/24')
  • ORDER BY

    Sorting can be done by column.

  • OFFSET

    Skip the indicated number of lines before printing the query results output.

  • LIMIT

    The default value is 250.

    If you are filtering events by user-defined period and the number of strings in the search results exceeds the defined value, you can click the Show next button to display additional strings in the table. This button is not displayed when filtering events by the standard period.

When filtering by alert-related events in alert investigation mode, you cannot perform operations on the data of event fields or assign names to the columns of displayed data.

Page top
[Topic 230248]

Saving and selecting events filter configuration

In KUMA, you can save a filter configuration and use it in the future. Other users can also use the saved filters if they have the appropriate access rights. When saving a filter, you are saving the configured settings of all the active filters at the same time, including the time-based filter, query builder, and the events table settings. Search queries are saved on the KUMA Core server and are available to all KUMA users of the selected tenant.

To save the current settings of the filter, query, and period:

  1. In the Events section of the KUMA web interface, click the SaveButton icon next to the filter expression.

    This opens the New filter window.

  2. In the window that opens, enter the name of the filter configuration in the Name field. The name can contain up to 128 Unicode characters.
  3. Under Select folder, select the tenant to which you want the created filter to belong and the folder inside the tenant, if necessary.

    To simplify your search, you can start typing the name in the Search area.

  4. If necessary, click Add folder to create a new folder inside the selected tenant.
  5. Click Save.

The filter configuration is now saved.

To select a previously saved filter configuration:

In the Events section of the KUMA web interface, click the icon next to the filter expression and select the relevant filter.

The selected configuration is active, which means that the search field is displaying the search query, and the upper part of the window is showing the configured settings for the period and frequency of updating the search results. Click the Run query button to submit the search query.

To add the filter configuration to favorites, you can hover over the name of the filter configuration, click the icon next to the name of the configuration and click the Add to favorites button.

Page top
[Topic 228358]

Deleting event filter configurations

To delete a previously saved filter configuration:

  1. In the Events section of the KUMA web interface, click the icon next to the filter search query.

    This opens the Saved queries window.

  2. Hover over the configuration that you want to delete and click the icon next to its name.
  3. This opens a menu; in the menu, click the icon.

The filter configuration is now deleted for all KUMA users.

Page top
[Topic 228359]

Supported ClickHouse functions

The following ClickHouse functions are supported in KUMA:

  • Arithmetic functions.
  • Arrays.
  • Comparison functions.
  • Logical functions.
  • Type conversion functions.
  • Date and time functions.
  • String functions.
  • String search functions.
  • Conditional functions: only the regular 'if' operator; the ternary operator is not supported.
  • Mathematical functions.
  • Rounding functions.
  • Functions for splitting and merging strings and arrays.
  • Bit functions.
  • Functions for working with UUIDs.
  • Functions for working with URLs.
  • Functions for working with IP addresses.
  • Functions for working with Nullable arguments.
  • Functions for working with geographic coordinates.

Functions from other sections are not supported.

For more details on SQL, refer to the ClickHouse documentation.

Page top
[Topic 235093]

Viewing event detail areas

To view information about an event:

  1. In the application web interface window, select the Events section.
  2. Search for events by using the query builder or by entering a query in the search field.

    The event table is displayed.

  3. Select the event whose information you want to view.

    The event details window opens.

The Event details area appears in the right part of the web interface window and contains a list of the event's parameters with values. In this area you can:

  • Include the selected field in the search or exclude it from the search by clicking filter-plus or filter-minus next to the setting value.
  • Clicking a file hash in the FileHash field opens a list in which you can select one of the following actions:
  • Open a window containing information about the asset if it is mentioned in the event fields and registered in the application.
  • You can click the link containing the collector name in the Service field to view the settings of the service that registered the event.

    You can also link an event to an alert if the application is in alert investigation mode and open the Correlation event details window if the selected event is a correlation event.

In the Event details area, the name of the described object is shown instead of its ID in the values of the following settings. At the same time, if you change the filtering of events by this setting (for example, by clicking filter-minus to exclude events with a certain setting-value combination from search results), the object's ID, and not its name, is added to the SQL query:

  • TenantID
  • SeriviceID
  • DeviceAssetID
  • SourceAssetID
  • DestinationAssetID
  • SourceAccountID
  • DestinationAccountID
Page top
[Topic 218039]

Exporting events

In KUMA, you can export information about events to a TSV file. The selection of events that will be exported to a TSV file depends on filter settings. The information is exported from the columns that are currently displayed in the events table. The columns in the exported file are populated with the available data even if they did not display in the events table in the KUMA web interface due to the special features of the SQL query.

To export information about events:

  1. In the Events section of the KUMA web interface, click the TSV button in the upper part of the table of events.

    The new export TSV file task is created in the Task manager section.

  2. Find the task you created in the Task manager section.

    When the file is ready to be downloaded, the Status column of the task displays the Completed status and the DoneIcon icon.

  3. Click the task type name and select Upload from the drop-down list.

    The TSV file will be downloaded using your browser's settings. By default, the file name is event-export-<date>_<time>.tsv.

The file is saved based on your web browser's settings.

Page top
[Topic 217871]

Configuring the table of events

Responses to user SQL queries are presented as a table in the Events section. The fields selected in the custom query appear at the end of the table, after the default columns. This table can be updated.

The following columns are displayed in the events table by default:

  • TenantID.
  • Timestamp.
  • Name.
  • DeviceProduct.
  • DeviceVendor.
  • DestinationAddress.
  • DestinationUserName.

In KUMA, you can customize the displayed set of event fields and their display order. The selected configuration can be saved.

When using SQL queries with data grouping and aggregation for filtering events, statistics are not available and the order of displayed columns depends on the specific SQL query.

In the events table, in the event details area, in the alert window, and in the widgets, the names of assets, accounts, and services are displayed instead of the IDs as the values of the SourceAssetID, DestinationAssetID, DeviceAssetID, SourceAccountID, DestinationAccountID, and ServiceID fields. When exporting events to a file, the IDs are saved, but columns with names are added to the file. The IDs are also displayed when you point the mouse over the names of assets, accounts, or services.

Searching for fields with IDs is only possible using IDs.

To configure the fields displayed in the events table:

  1. Click the icon in the upper-right corner of the events table.

    A window for selecting the event fields that should be displayed in the events table will be displayed.

  2. Select the check boxes opposite the fields that you want to view in the table. You can search for relevant fields by using the Search field.

    You can configure the table to display any event field from the KUMA event data model and the extended event schema. The Timestamp and Name parameters are always displayed in the table. Click the Default button to display only default event parameters in the events table.

    When you select a check box, the events table is updated and a new column is added. When a check box is cleared, the column disappears.

    You can also remove columns from the events table by clicking the column title and selecting Hide column from the drop-down list.

  3. If necessary, change the display order of the columns by dragging the column headers in the event tables.
  4. If you want to sort the events by a specific column, click its title and in the drop-down list select one of the available options: Ascending or Descending.

The selected event fields will be displayed as columns in the table of the Events section in the order you specified.

Page top
[Topic 228361]

Refreshing events table

You can update the displayed event selection with the most recent entries by refreshing the web browser page. You can also refresh the events table automatically and set the frequency of updates. Automatic refresh is disabled by default.

To enable automatic refresh,

Select a refresh rate from the drop-down list in the upper-right part of the Events section:

  • 5 seconds
  • 15 seconds
  • 30 seconds
  • 1 minute
  • 5 minutes
  • 15 minutes

The events table now refreshes automatically.

To disable automatic refresh:

In the drop-down list of refresh rates in the upper-right part of the Events section, select No refresh.

Page top
[Topic 217961]

Getting events table statistics

You can get statistics for the current events selection displayed in the events table. The selected events depend on the filter settings.

To obtain statistics:

Select Statistics from the MoreButton drop-down list in the upper-right corner of the events table, or click on any value in the events table and select Statistics from the opened context menu.

The Statistics details area appears with the list of parameters from the current event selection. The numbers near each parameter indicate the number of events with that parameter in the selection. If a parameter is expanded, you can also see its five most frequently occurring values. You can find relevant parameters by using the Search fields field.

In a high availability configuration, for all event fields that contain the FQDN of the Core, the Statistics section displays core instead of the FQDN.

The Statistics window allows you to modify the events filter.

When using SQL queries with data grouping and aggregation for filtering events, statistics are not available.

Page top
[Topic 228360]

Viewing correlation event details

You can view the details of a correlation event in the Correlation event details window.

To view information about a correlation event:

  1. In the Events section of the KUMA web interface, click a correlation event.

    You can use filters to find correlation events by assigning the correlated value to the Type parameter.

    The details area of the selected event will open. If the selected event is a correlation event, the Detailed view button will be displayed at the bottom of the details area.

  2. Click the Detailed view button.

The correlation event window will open. The event name is displayed in the upper left corner of the window.

The Correlation event details section of the correlation event window contains the following data:

  • Correlation event severity—the importance of the correlation event.
  • Correlation rule—the name of the correlation rule that triggered the creation of this correlation event. The rule name is represented as a link that can be used to open the settings of this correlation rule.
  • Correlation rule severity—the importance of the correlation rule that triggered the correlation event.
  • Correlation rule ID—the identifier of the correlation rule that triggered the creation of this correlation event.
  • Tenant—the name of the tenant that owns the correlation event.

The Related events section of the correlation event window contains the table of events related to the correlation event. These are base events that actually triggered the creation of the correlation event. When an event is selected, the details area opens in the right part of the web interface window.

The Find in events link to the right of the section header is used for alert investigation.

The Related endpoints section of the correlation event window contains the table of hosts related to the correlation event. This information comes from the base events related to the correlation event. Clicking the name of the asset opens the Asset details window.

The Related users section of the correlation event window contains the table of users related to the correlation event. This information comes from the base events related to the correlation event.

See also:

About alerts

Correlator

Alert investigation

Page top
[Topic 217946]

Generating an SQL query using KUMA SQL functions

KUMA's SQL functions allow using the attributes of assets and accounts in search queries to filter events, generate reports and widgets (graph type: Table). You can enrich events with data from dictionaries, tables, assets, and accounts using the following sets of functions:

  • The enrich function set. Allows enriching query results with fields of the asset or account, values from a dictionary of the Dictionary type or a dictionary of the Table type. The enrich function set includes the following functions:
    • enrich_assets
    • enrich_accounts
    • enrich_table
    • enrich_dictionary
  • The lookup function set. Allows adding conditions for assets and accounts to the search query. The lookup function set includes the following functions:
    • lookup_assets
    • lookup_accounts
    • lookup_assets_category

The maximum number of events per query is 10,000.

Page top
[Topic 294927]

Using the 'enrich' function set in SQL queries

The enrich function set (see the table below) is used in the SELECT clause of an SQL query to enrich the results.

When using this set of functions, you cannot group events by enriched fields.

Widgets and reports enriched by the functions must be owned by no more than one tenant.

The values of asset fields that are specified in SQL queries are case-sensitive.

Description of the 'enrich' set of functions

Function

Event field

Asset field

Description

enrich_assets

DeviceAssetID

DestinationAssetID

SourceAssetID

 

name

os

osBuild

ipAddress

macAddress

fqdn

source

software

cii

ramSize

kscLastAvBasesUpdate

kscLastInfoUpdate

kscLastUpdate

kscLastSystemStart

kscStatusMask

kscRtProtectionState

kscEncryptionState

kscAntiSpamStatus

kscEmailAvStatus

kscDlpStatus

kscStatusID

kscEdrStatus

lastVisible

aistatus

aiscore

Enriches the query result with asset fields based on the ID specified passed to the function.

For example:

enrich_assets(DeviceAssetID, name)

enrich_accounts

SourceAccountID

DestinationAccountID

name

guid

domain

cn

dn

employee_id

email

mail_nickname

mobile

sid

sam_name

telephone_number

principal_name

sn

type

title

division

department

manager

location

company

street_address

physical_delivery_office_name

uac

Enriches the query result with account fields based on the ID specified passed to the function.

For example:

enrich_account(SourceAccountID, name)

This function is available only for tenants that have LDAP enrichment enabled.

enrich_table

Any

Enriches the query result with values from a dictionary of the Table type based on event fields in the response from the ClickHouse cluster.

For example:

enrich_table('Main/some-table', [DeviceProduct], 'value')

Only one column of the table can be the key, but the record key can be a compound key.

For example:

enrich_table('Main/Dictionary', [BytesOut|BytesIn], 'value2')

The enrich_table function returns only one column of the table. If you need to call more columns, call the function again specifying the relevant fields.

enrich_dictionary

Any

Enriches the query result with values from a dictionary of the Dictionary type based on event fields in the response from the ClickHouse cluster.

For example:

  • enrich_dictionary('Main/some-dict', [SourceAddress]
  • enrich_dictionary('Main/some-dict', [SourceAddress, DeviceAddress])

You can view the path to the dictionary in the Resources → Dictionaries section.

Example SQL queries with an enrich function

You can specify an enrich function when manually creating SQL queries. Example queries with an enrich function

  • All events of the 'events' table with the Name and Message fields, as well as the macAddress and osBuild asset fields to enrich the result with asset data. The number of rows that can be displayed in the table is 250.

    SELECT Name,

    Message,

    enrich_assets(DeviceAssetID, macAddress),

    enrich_assets(DeviceAssetID, osBuild)

    FROM `events`

    WHERE DeviceAssetID != ''

    LIMIT 250

  • All events of the 'events' table with event fields SourceUserName and SourceAddress, as well as values of the mobile field to enrich the result with account data. The number of rows that can be displayed in the table is 10.

    SELECT SourceUserName,

    SourceAddress,

    enrich_accounts(SourceAccountID, mobile) as mobile

    FROM `events`

    WHERE SourceAccountID != ''

    LIMIT 10

  • All events in the events table with DeviceProduct event fields with values from the Main/[OOTB] Linux Auditd record types table. Sorted by Timestamp in descending order. The number of rows that can be displayed in the table is 1000.

    SELECT DeviceProduct,

    enrich_table('Main/[OOTB] Linux. Auditd record types', [DeviceProduct], 'Description') as HTTP

    FROM `events` WHERE Type!=4

    LIMIT 1000

  • All events in the events table with DeviceAction event fields with values from the Main/[OOTB] KEDR dictionary. The number of rows that can be displayed in the table is 250.

    SELECT DeviceAction,

    enrich_dictionary('Main/[OOTB] KEDR. AccountType', [BytesOut]) as accountType

    FROM `events` WHERE Type!=4

    LIMIT 255

Page top
[Topic 295026]

Using the 'lookup' function set in SQL queries

The lookup function set (see the table below) is used in the WHERE clause of an SQL query to add conditions for assets and accounts to the search query.

Description of the 'lookup' set of functions

Function type

Event field

Asset field

Operator

Description

lookup_assets

 

DeviceAssetID

SourceAssetID

DestinationAssetID

 

name

os

osBuild

source

cii

ramSize

kscRtProtectionState

kscEncryptionState

kscAntiSpamStatus

kscEmailAvStatus

kscDlpStatus

kscStatusID

kscEdrStatus

aistatus

aiscore

=

Allows you to add conditions to the query result for assets whose field values match the value specified in the query.

For values that are number, you must specify a string with the required number.

For example:

lookup_assets(DeviceAssetID, osBuild, '=', '19045.000000')

lookup_assets(DeviceAssetID, ramSize, '=', '3241234234')

kscLastAvBasesUpdate

kscLastInfoUpdate

kscLastUpdate

kscLastSystemStart

lastVisible

ramSize

aiscore

<

>

<=

>=

Allows you to add conditions for assets with field values that are:

  • Less than the value specified in the query
  • Greater than the value specified in the query
  • Less than or equal to the value specified in the query
  • Greater than or equal to the value specified in the query

For values that are number, you must specify a string with the required number.

For example:

lookup_assets(DeviceAssetID, aiscore, '>', '50')

Date values can be specified in the following formats:

  • Number of seconds:

    lookup_assets(DeviceAssetID, lastVisible, '>', '1725971837')

  • Number of milliseconds:

    lookup_assets(DeviceAssetID, lastVisible, '<', '1725971837834')

  • Number of days:

    lookup_assets(DeviceAssetID, lastVisible, '<', 'now-15d')

 

name

os

osBuild

fqdn

software

match

imatch

Allows you to add asset conditions to the query result using a string literal or a regular expression.

For example:

  • Query with a string literal and the match operator:

    lookup_assets(DestinationAssetID, name, 'match','test.company.com')

  • Query with a string literal and the imatch operator:

    lookup_assets(DestinationAssetID, name, 'imatch','test.COMPANY.com')

  • Query with a regular expression and the match operator:

    lookup_assets(DestinationAssetID, name, 'match','([.-]?\w+)')

  • Query with a regular expression and the imatch operator:

    lookup_assets(DestinationAssetID, name, 'imatch','([A-Z])')

Using a case-insensitive regular expression (PCRE2) is allowed.

ipAddress

macAddress

fqdn

software

kscStatusMask

in

Allows you to add to the result asset conditions whose values match the value specified in the query.

For example:

lookup_assets(DeviceAssetID, fqdn , 'in', ['test_fqdn1.company.ru', 'test_fqdn2.company.ru'])

ipAddress

insubnet

Allows you to add conditions for assets with IP addresses in the specified subnet. The IP address is specified in CIDR format.

For example:

lookup_assets(DeviceAssetID, ipAddress, 'insubnet', '10.0.0.0/24')

ipAddress

inrange

Allows you to add conditions for assets with IP addresses in the specified range.

The first IP address specified in the query is the start IP address, and the second is the end address.

For example:

lookup_assets(DeviceAssetID, ipAddress, 'inrange', ['192.168.1.1', '192.168.1.184'])

The query filters assets with IP addresses in the range from 192.168.1.1 to 192.168.1.184.

lookup_accounts

 

SourceAccountID

DestinationAccountID

 

name

guid

domain

cn

dn

employee_id

email

mail_nickname

mobile

sid

sam_name

telephone_number

principal_name

sn

title

division

department

manager

location

company

street_address

physical_delivery_office_name

uac

=

Allows you to add conditions to the query result for accounts whose field values match the value specified in the query.

For values that are number, you must specify a string with the required number.

For example:

lookup_accounts(SourceAccountID, employee_id, '=', '15452')

name

guid

domain

cn

dn

employee_id

email

mail_nickname

mobile

sid

sam_name

telephone_number

principal_name

sn

type

title

division

department

manager

location

company

street_address

physical_delivery_office_name

like

Allows you to add conditions to the query result for accounts whose field values match the 'like' expression specified in the query.

For more information about the like operator, please refer to the SQL Language Expressions reference.

lookup_accounts(DestinationAccountID, name, 'like', '_ccount1%')

lookup_assets_category

DeviceAssetID

SourceAssetID

DestinationAssetID

Finds all events that have associated assets in the specified folder.

For example:

lookup_assets_category(DeviceAssetID, ['Main/Business Impact'])

where Main is the name of the tenant and Business Impact is the name of the folder.

Example SQL queries with a lookup function

You can specify a lookup function when manually creating SQL queries. Example queries with a lookup function

  • All events of the 'events' table with the TenantID and DeviceProduct event fields, as well as the fqdn asset field, whose value contains the win10 substring. The number of rows that can be displayed in the table is 250.

    SELECT TenantID, DeviceProduct

    FROM `events`

    WHERE DeviceAssetID != ''

    AND lookup_assets(DeviceAssetID, fqdn, 'match', 'win10')

    LIMIT 250

  • All events in the 'events' table, with the TenantID and DeviceProduct event fields, and with values of the EmployeeID field, whose value is 15452. The number of rows that can be displayed in the table is 10.

    SELECT TenantID, DeviceProduct

    FROM `events`

    WHERE SourceAccountID != ''

    AND lookup_accounts(SourceAccountID, employee_id, '=', '15452')

    LIMIT 10

  • All events of the 'events' table with the TenantID and DeviceProduct event fields, with the folder in which the asset is located. The number of rows that can be displayed in the table is 100.

    SELECT TenantID, DeviceProduct

    FROM `events`

    WHERE DeviceAssetID != ''

    AND lookup_assets_category(DeviceAssetID, ['Main/Business Impact'])

    LIMIT 100

Page top
[Topic 295027]