How to send all warnings at once via http_post2

[huangyutongs]

I tried aggregation, and the result was that when there were two messages matching, it was sent twice after waiting for the aggregation time. How to aggregate the contents of the two alerts together and only send them once?
I don't know if I missed something.

this is my message

{
   "@timestamp": "2023-11-23T07:12:11.640Z",
   "message": "Message 11",
   "level": "Error",
   "url": "/api/app/pda-ddd/1",
   "name": {
      "first": "Zachary523",
      "last": "Tong51"
      },
   "fields": {
     "RequestPath": "/"
   }
}

config.yaml

rules_folder: /opt/elastalert/rules

run_every:
  seconds: 20

buffer_time:
  minutes: 15

es_host: es.baidu.cn
es_port: 80
es_username: test
es_password: test

writeback_index: elastalert_status_test

alert_time_limit:
  days: 2

use_ssl: False
verify_certs: False

rules/a.yaml

name: library
type: frequency
index: library*

num_events: 1
timeframe:
  minutes: 15

aggregation:
  seconds: 50

aggregation_key: "level"
top_count_keys: ["level"]
summary_table_fields:
  - level
filter:
  - query:
      query_string:
        query: "level: Error"

realert:
  minutes: 0

alert: post2
http_post2_url: "https://webhook.site/2b487563-0e71-40cf-b756-ddddddd"
attach_related: true
http_post2_payload: |
  {
  "level": "{{level}}",
  "message": "{{message}}"
  }

[jertel]

If the two alerts were for different "level" values then it would make sense that it sent two alerts. If they are for the same "level" value of "Error" then I'm not sure what's going on. You'd need to review the debug logs to better understand what's happening.

[huangyutongs]

Thank you very much for your reply. In fact, I am not very familiar with some of the above parameters. I hope that when there are two or more hit items, their contents can be aggregated to send an alert instead of multiple alerts. In my environment, I only want to receive Error level log alerts and do not want to be alerted. To drown out while not missing some alerts, I have two logs similar to this.

 {
   "@timestamp": "2023-11-23T07:12:11.640Z",
   "message": "Message 11",
   "level": "Error",
   "url": "/api/app/pda-ddd/1",
   "name": {
      "first": "Zachary523",
      "last": "Tong51"
      },
   "fields": {
     "RequestPath": "/"
   }
}
 {
   "@timestamp": "2023-11-23T07:13:11.640Z",
   "message": "Message 12",
   "level": "Error",
   "url": "/api/app/pda-ddd/12",
   "name": {
      "first": "Zachary523",
      "last": "Tong52"
      },
   "fields": {
     "RequestPath": "/"
   }
}

Their levels are all Error. I want elastalert2 to be able to aggregate content and send an alert, which will then be picked up by the webhook server. The result looks like this, or it can look something else.

   {
   "level": "ErrorError",
   "message": "Message 11Message 12"
   }

[jertel]

I suggest reviewing the docs that cover this topic: https://elastalert2.readthedocs.io/en/latest/ruletypes.html#aggregation. Also consider the any rule type if you're num_events property will be 1.

You don't need to specify the aggregation_key property since you don't want to separate the alerts.

Also, your aggregation time of 50 seconds is tiny compared to the frequency window. Consider adjusting either that value or the timeframe so that the aggregations is a multiple of the timeframe.