Skip to content

Commit

Permalink
Merge branch 'master' into kubectl-job
Browse files Browse the repository at this point in the history
  • Loading branch information
Avi-Robusta committed Jan 9, 2025
2 parents e295c8b + 57c13b5 commit b2d7b7c
Show file tree
Hide file tree
Showing 21 changed files with 522 additions and 367 deletions.
18 changes: 0 additions & 18 deletions .github/workflows/test-poetry-lock.yaml

This file was deleted.

9 changes: 9 additions & 0 deletions .github/workflows/test_robusta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,16 @@ name: Test robusta with pytest
on: [push, pull_request, workflow_dispatch]

jobs:
check:
name: Pre-commit checks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: pre-commit/[email protected]

run_tests:
needs: check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
Expand Down
58 changes: 34 additions & 24 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,33 +1,43 @@
repos:
- repo: https://github.com/ambv/black
rev: 22.3.0
hooks:
- id: black
language_version: python3
args: [--config=pyproject.toml]

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.3.0
- repo: https://github.com/python-poetry/poetry
rev: 1.8.5
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: poetry-check
- id: poetry-lock
pass_filenames: false
args:
- --no-update

# - repo: https://github.com/ambv/black
# rev: 22.3.0
# hooks:
# - id: black
# language_version: python3
# args: [--config=pyproject.toml]

- repo: https://github.com/MarcoGorelli/absolufy-imports
rev: v0.3.1
hooks:
- id: absolufy-imports
# - repo: https://github.com/pre-commit/pre-commit-hooks
# rev: v3.3.0
# hooks:
# - id: trailing-whitespace
# - id: end-of-file-fixer

- repo: https://github.com/pycqa/flake8
rev: 5.0.4
hooks:
- id: flake8
args: [--config=.flake8]
# - repo: https://github.com/MarcoGorelli/absolufy-imports
# rev: v0.3.1
# hooks:
# - id: absolufy-imports

- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
args: [--settings-path=pyproject.toml]
# - repo: https://github.com/pycqa/flake8
# rev: 5.0.4
# hooks:
# - id: flake8
# args: [--config=.flake8]

# - repo: https://github.com/pycqa/isort
# rev: 5.12.0
# hooks:
# - id: isort
# args: [--settings-path=pyproject.toml]

# - repo: https://github.com/pre-commit/mirrors-mypy
# rev: v0.790
Expand Down
2 changes: 1 addition & 1 deletion docs/how-it-works/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Alternatively, you can :ref:`integrate an existing Prometheus with Robusta <Inte
Web UI (Optional)
^^^^^^^^^^^^^^^^^^^^^^

The Robusta `SaaS platform <http://home.robusta.dev/ui?from=docs>`_ provides a single pane of glass for all your alerts and clusters.
The Robusta `SaaS platform <http://home.robusta.dev/?from=docs>`_ provides a single pane of glass for all your alerts and clusters.

On commercial plans, the UI is available for self-hosting in your own environment.

Expand Down
5 changes: 1 addition & 4 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,7 @@
notification-routing/configuring-sinks
Routing (Scopes) <notification-routing/routing-with-scopes>
Grouping (Slack Threads) <notification-routing/notification-grouping>
notification-routing/routing-by-namespace
notification-routing/routing-by-type
notification-routing/routing-by-time
notification-routing/routing-exclusion
notification-routing/routing-silencing
notification-routing/notification-routing-examples

Expand Down Expand Up @@ -91,7 +88,7 @@ Better Prometheus Alerts (and more) for Kubernetes
* :ref:`AI Investigation <AI Analysis>` - Kickstart your alert investigations with AI (optional)
* :ref:`Alert Enrichment <Automatically Investigate a Prometheus Alert>` - see pods log and other data alongside your alerts
* :ref:`Self-Healing <Remediate Prometheus Alerts>` - define auto-remediation rules for faster fixes
* :ref:`Advanced Routing <Notification Basics>` by team, namespace, k8s metadata and more
* :ref:`Advanced Routing <Defining Sinks>` by team, namespace, k8s metadata and more
* :ref:`K8s Problem-Detection <Triggers Reference>` - alert on OOMKills or failing Jobs without PromQL
* :ref:`Change Tracking <Track Kubernetes Changes>` - correlate alerts and Kubernetes rollouts
* :ref:`Auto-Resolve <Jira>` - send alerts, resolve them when updated (e.g. in Jira)
Expand Down
81 changes: 65 additions & 16 deletions docs/notification-routing/configuring-sinks.rst
Original file line number Diff line number Diff line change
@@ -1,33 +1,30 @@
.. _sinks-overview:

Notification Basics
Defining Sinks
==========================

Robusta can send notifications to various destinations, called sinks. For a list of all sinks, refer to :ref:`Sinks Reference`.
Robusta can send notifications to various destinations, called sinks.

Defining Sinks
^^^^^^^^^^^^^^^^^^
For a list of all sinks, refer to :ref:`Sinks Reference`.

A Simple Sink Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sinks are defined in Robusta's Helm chart, using the ``sinksConfig`` value.

For example, lets add a :ref:`Microsoft Teams <MS Teams sink>`:
For example, lets add a :ref:`Microsoft Teams Sink <MS Teams sink>`:

.. code-block:: yaml
sinksConfig:
- ms_teams_sink: # sink type
name: my_teams_sink # arbitrary name
webhook_url: <placeholder> # a sink-specific parameter
stop: false # optional (see `Routing Alerts to only one Sink`)
scope: {} # optional routing rules
default: true # optional (see below)
name: my_teams_sink # name that uniquely identifies this sink in Robusta
webhook_url: <placeholder> # the webhook URL for MSTeams - each sink has different parameters like this
Many sinks have unique parameters which can be found under :ref:`Sinks Reference`.
For all options, refer to :ref:`All Sink Options`.

Defining Multiple Sinks
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can define multiple sinks and by default, notifications will be sent to all of them.

If you'd like to selectively send notifications to different sinks, you can define :ref:`routing rules <sink-scope-matching>`.
You can define multiple sinks. By default, notifications will be sent to all of them.

In the following example, we define a :ref:`Slack sink <Slack>` and a :ref:`MS Teams sink <MS Teams>` without any routing rules, so both sinks receive all notifications:

Expand All @@ -42,7 +39,59 @@ In the following example, we define a :ref:`Slack sink <Slack>` and a :ref:`MS T
name: my_teams_sink
webhook_url: <placeholder>
See Also
To selectively send notifications to different sinks, refer to :ref:`routing (scopes) <sink-scope-matching>`.


All Sink Options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Here is an example showing common sink options:

.. code-block:: yaml
sinksConfig:
- slack_sink: # sink type
name: my_sink_name # name that uniquely identifies this sink in Robusta
scope: {} # optional - filter notifications sent to this sink
activity: {} # optional - enable/disable sink according to time of day/week
stop: false # optional - stop notifications from continuing to subsequent sinks
grouping: {} # optional - use grouping to reduce the number of notifications (i.e. group into slack threads)
default: true # optional - disable this sink by default
# sink-specific parameters - e.g. for Slack, some options are shown below
# api_key: xoxb-112...
# slack_channel: general-alerts
Description of each option:

+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+
| Parameter Name | Description | Default | Docs |
+==================+=========================================================+==========================================================+===============================================+
| name | A unique name for this sink in Robusta | - | - |
+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+
| scope | Filters the notifications sent to this sink | *undefined* - all notifications are sent (unless already | |
| | | sent a previou sink that set `stop: true`) | :ref:`Routing (scopes) <sink-scope-matching>` |
+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+
| activity | Controls the hours this sink is active | *undefined* - active all hours and all days of the week | :ref:`Route by Time` |
+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+
| stop | Should notifications continue to subsequent sinks? | false - notification sent to this sink will continue to | :ref:`Stop Further Notifications` |
| | | subsequent sinks | |
+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+
| grouping | Currently only impacts the Slack sink, where it controls| *undefined* (disabled) | :ref:`Grouping <notification-grouping>` |
| | the creation of threads and the grouping of many | | |
| | notifications into one message | | |
+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+
| default | Is this sink enabled by default? When false, this sink | true - this sink is enabled by default | :ref:`Alternative Routing Methods` |
| | only accepts notifications from customPlaybooks which | | |
| | explicitly named this sink (if scope is set, it will | | |
| | still filter those notifications) | | |
+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+
| *sink specific* | Parameters specific to the sink type, like api_key for | - | :ref:`sink-specific docs <Sinks Reference>` |
| *parameters* | Slack and webhook_url for MSTeams | | |
+------------------+---------------------------------------------------------+----------------------------------------------------------+-----------------------------------------------+

Learn More
^^^^^^^^^^^^

🔔 :ref:`All Sinks <Sinks Reference>`
* 🔔 :ref:`All Sinks <Sinks Reference>`
* ↳ :ref:`Routing (scopes) <sink-scope-matching>`
36 changes: 36 additions & 0 deletions docs/notification-routing/implementing-monitoring-shifts.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
Route By Time of Day
####################

In this example we'll route alerts to different support teams based on the time of day:

* Between 12AM and 12PM, #shift-1 will receive alerts.
* The rest of the time, #payment-support-2 will recieve notifications.

.. code-block:: yaml
sinksConfig:
- slack_sink:
name: shift_1_sink
slack_channel: shift-1
api_key: secret-key
activity:
timezone: UTC
intervals:
- days: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun']
hours:
- start: 00:00 # 12 AM
end: 12:00 # 12 PM
- slack_sink:
name: shift_2_sink
slack_channel: shift-2
api_key: secret-key
activity:
timezone: UTC
intervals:
- days: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun']
hours:
- start: 12:00 # 12 PM
end: 23:59 # 11:59 PM
For more information on time-based routing, view :ref:`Route by Time`.
106 changes: 10 additions & 96 deletions docs/notification-routing/notification-routing-examples.rst
Original file line number Diff line number Diff line change
@@ -1,100 +1,14 @@
Common Examples
Routing Cookbook
===================================

Here are examples of common routing patterns that teams use with Robusta. The goal is to send each team only the most relevant alerts.
.. toctree::
:maxdepth: 1
:hidden:

Use Case 1: Route Specific Alerts to Seperate Teams and Slack Channels
**********************************************************************************
In this example we are going to consider two teams, with #frontend and #backend channels. The Frontend team should only receive alerts that are from the Frontend namespace. All the alerts from backend namespace should go to the #backend channel
routing-by-namespace
routing-by-type
implementing-monitoring-shifts
routing-to-multiple-slack-channels
routing-exclusion


.. code-block:: yaml
sinksConfig:
- slack_sink:
name: frontend_sink
slack_channel: frontend-notifications
api_key: secret-key
scope:
include:
- namespace: [frontend]
- slack_sink:
name: backend_sink
slack_channel: backend-notifications
api_key: secret-key
scope:
include:
- namespace: [backend]
For a complete list of filters, view the :ref:`Scopes documentation <sink-scope-matching>`.

Use Case 2: Route Alerts To Different Support Teams Based On Time Of The Day
**********************************************************************************

Let's see how we can route alerts between two teams with different Slack channels. Between 12AM to 12 PM #payments-support-1 should receive alerts. The rest of the time #payment-support-2 should recieve notifications.

.. code-block:: yaml
sinksConfig:
- slack_sink:
name: payments_support_1_sink
slack_channel: payments-support-1
api_key: secret-key
activity:
timezone: UTC
intervals:
- days: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun']
hours:
- start: 00:00 # 12 AM
end: 12:00 # 12 PM
- slack_sink:
name: payments_support_2_sink
slack_channel: payments-support-2
api_key: secret-key
activity:
timezone: UTC
intervals:
- days: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun']
hours:
- start: 12:00 # 12 PM
end: 23:59 # 11:59 PM
For more information on time-based routing, :ref:`view <Route by Time>`

If you want to selectively apply time-based-routing to certain alerts only, then this method can be combined with :ref:`Scopes <sink-scope-matching>`. For example:

.. code-block:: yaml
sinksConfig:
- slack_sink:
name: payments_support_1_sink
slack_channel: payments-support-1
api_key: secret-key
activity:
timezone: UTC
intervals:
- days: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun']
hours:
- start: 00:00 # 12 AM
end: 12:00 # 12 PM
scope:
include:
- namespace: [payment, renewal]
- slack_sink:
name: payments_support_2_sink
slack_channel: payments-support-2
api_key: secret-key
activity:
timezone: UTC
intervals:
- days: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun']
hours:
- start: 12:00 # 12 PM
end: 23:59 # 11:59 PM
scope:
include:
- namespace: [payment, renewal]
In this section you'll find example configurations for common routing patterns.
Loading

0 comments on commit b2d7b7c

Please sign in to comment.