forked from fail2ban/fail2ban
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathChangeLog
2421 lines (2235 loc) · 126 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
__ _ _ ___ _
/ _|__ _(_) |_ ) |__ __ _ _ _
| _/ _` | | |/ /| '_ \/ _` | ' \
|_| \__,_|_|_/___|_.__/\__,_|_||_|
Fail2Ban: Changelog
===================
ver. 1.0.1-dev-1 (20??/??/??) - development nightly edition
-----------
### Compatibility:
* potential incompatibility by parsing of options of `backend`, `filter` and `action` parameters (if they
are partially incorrect), because fail2ban could throw an error now (doesn't silently bypass it anymore).
* to v.0.11:
- due to change of `actioncheck` behavior (gh-488), some actions can be incompatible as regards
the invariant check, if `actionban` or `actionunban` would not throw an error (exit code
different from 0) in case of unsane environment.
### Fixes
* readline fixed to consider interim new-line character as part of code point in multi-byte logs
(e. g. unicode encoding like utf-16be, utf-16le);
* `action.d/ufw.conf`:
- fixed handling on IPv6 (using prepend, gh-2331, gh-3018)
- application names containing spaces can be used now (gh-656, gh-1532, gh-3018)
* `filter.d/drupal-auth.conf` more strict regex, extended to match "Login attempt failed from" (gh-2742)
### New Features and Enhancements
* `actioncheck` behavior is changed now (gh-488), so invariant check as well as restore or repair
of sane environment (in case of recognized unsane state) would only occur on action errors (e. g.
if ban or unban operations are exiting with other code as 0)
* better recognition of log rotation, better performance by reopen: avoid unnecessary seek to begin of file
(and hash calculation)
* file filter reads only complete lines (ended with new-line) now, so waits for end of line (for its completion)
* `action.d/ufw.conf` (gh-3018):
- new option `add` (default `prepend`), can be supplied as `insert 1` for ufw versions before v.0.36 (gh-2331, gh-3018)
- new options `kill-mode` and `kill` to drop established connections of intruder (see action for details, gh-3018)
* `filter.d/nginx-http-auth.conf` - extended with parameter mode, so additionally to `auth` (or `normal`)
mode `fallback` (or combined as `aggressive`) can find SSL errors while SSL handshaking, gh-2881
ver. 0.11.2 (2020/11/23) - heal-the-world-with-security-tools
-----------
### Compatibility:
* to v.0.10:
- 0.11 is totally compatible to 0.10 (configuration- and API-related stuff), but the database
got some new tables and fields (auto-converted during the first start), so once updated to 0.11, you
have to remove the database /var/lib/fail2ban/fail2ban.sqlite3 (or its different to 0.10 schema)
if you would need to downgrade to 0.10 for some reason.
* to v.0.9:
- Filter (or `failregex`) internal capture-groups:
* If you've your own `failregex` or custom filters using conditional match `(?P=host)`, you should
rewrite the regex like in example below resp. using `(?:(?P=ip4)|(?P=ip6)` instead of `(?P=host)`
(or `(?:(?P=ip4)|(?P=ip6)|(?P=dns))` corresponding your `usedns` and `raw` settings).
Of course you can always define your own capture-group (like below `_cond_ip_`) to do this.
```
testln="1500000000 failure from 192.0.2.1: bad host 192.0.2.1"
fail2ban-regex "$testln" "^\s*failure from (?P<_cond_ip_><HOST>): bad host (?P=_cond_ip_)$"
```
* New internal groups (currently reserved for internal usage):
`ip4`, `ip6`, `dns`, `fid`, `fport`, additionally `user` and another captures in lower case if
mapping from tag `<F-*>` used in failregex (e. g. `user` by `<F-USER>`).
- v.0.10 and 0.11 use more precise date template handling, that can be theoretically incompatible to some
user configurations resp. `datepattern`.
- Since v0.10 fail2ban supports the matching of IPv6 addresses, but not all ban actions are
IPv6-capable now.
### Fixes
* [stability] prevent race condition - no ban if filter (backend) is continuously busy if
too many messages will be found in log, e. g. initial scan of large log-file or journal (gh-2660)
* pyinotify-backend sporadically avoided initial scanning of log-file by start
* python 3.9 compatibility (and Travis CI support)
* restoring a large number (500+ depending on files ulimit) of current bans when using PyPy fixed
* manual ban is written to database, so can be restored by restart (gh-2647)
* `jail.conf`: don't specify `action` directly in jails (use `action_` or `banaction` instead)
* no mails-action added per default anymore (e. g. to allow that `action = %(action_mw)s` should be specified
per jail or in default section in jail.local), closes gh-2357
* ensure we've unique action name per jail (also if parameter `actname` is not set but name deviates from standard name, gh-2686)
* don't use `%(banaction)s` interpolation because it can be complex value (containing `[...]` and/or quotes),
so would bother the action interpolation
* fixed type conversion in config readers (take place after all interpolations get ready), that allows to
specify typed parameters variable (as substitutions) as well as to supply it in other sections or as init parameters.
* `action.d/*-ipset*.conf`: several ipset actions fixed (no timeout per default anymore), so no discrepancy
between ipset and fail2ban (removal from ipset will be managed by fail2ban only, gh-2703)
* `action.d/cloudflare.conf`: fixed `actionunban` (considering new-line chars and optionally real json-parsing
with `jq`, gh-2140, gh-2656)
* `action.d/nftables.conf` (type=multiport only): fixed port range selector, replacing `:` with `-` (gh-2763)
* `action.d/firewallcmd-*.conf` (multiport only): fixed port range selector, replacing `:` with `-` (gh-2821)
* `action.d/bsd-ipfw.conf`: fixed selection of rule-no by large list or initial `lowest_rule_num` (gh-2836)
* `filter.d/common.conf`: avoid substitute of default values in related `lt_*` section, `__prefix_line`
should be interpolated in definition section (inside the filter-config, gh-2650)
* `filter.d/dovecot.conf`:
- add managesieve and submission support (gh-2795);
- accept messages with more verbose logging (gh-2573);
* `filter.d/courier-smtp.conf`: prefregex extended to consider port in log-message (gh-2697)
* `filter.d/traefik-auth.conf`: filter extended with parameter mode (`normal`, `ddos`, `aggressive`) to handle
the match of username differently (gh-2693):
- `normal`: matches 401 with supplied username only
- `ddos`: matches 401 without supplied username only
- `aggressive`: matches 401 and any variant (with and without username)
* `filter.d/sshd.conf`: normalizing of user pattern in all RE's, allowing empty user (gh-2749)
### New Features and Enhancements
* fail2ban-regex:
- speedup formatted output (bypass unneeded stats creation)
- extended with prefregex statistic
- more informative output for `datepattern` (e. g. set from filter) - pattern : description
* parsing of action in jail-configs considers space between action-names as separator also
(previously only new-line was allowed), for example `action = a b` would specify 2 actions `a` and `b`
* new filter and jail for GitLab recognizing failed application logins (gh-2689)
* new filter and jail for Grafana recognizing failed application logins (gh-2855)
* new filter and jail for SoftEtherVPN recognizing failed application logins (gh-2723)
* `filter.d/guacamole.conf` extended with `logging` parameter to follow webapp-logging if it's configured (gh-2631)
* `filter.d/bitwarden.conf` enhanced to support syslog (gh-2778)
* introduced new prefix `{UNB}` for `datepattern` to disable word boundaries in regex;
* datetemplate: improved anchor detection for capturing groups `(^...)`;
* datepattern: improved handling with wrong recognized timestamps (timezones, no datepattern, etc)
as well as some warnings signaling user about invalid pattern or zone (gh-2814):
- filter gets mode in-operation, which gets activated if filter starts processing of new messages;
in this mode a timestamp read from log-line that appeared recently (not an old line), deviating too much
from now (up too 24h), will be considered as now (assuming a timezone issue), so could avoid unexpected
bypass of failure (previously exceeding `findtime`);
- better interaction with non-matching optional datepattern or invalid timestamps;
- implements special datepattern `{NONE}` - allow to find failures totally without date-time in log messages,
whereas filter will use now as timestamp (gh-2802)
* performance optimization of `datepattern` (better search algorithm in datedetector, especially for single template);
* fail2ban-client: extended to unban IP range(s) by subnet (CIDR/mask) or hostname (DNS), gh-2791;
* extended capturing of alternate tags in filter, allowing combine of multiple groups to single tuple token with new tag
prefix `<F-TUPLE_`, that would combine value of `<F-V>` with all value of `<F-TUPLE_V?_n?>` tags (gh-2755)
ver. 0.11.1 (2020/01/11) - this-is-the-way
-----------
### Fixes
* purge database will be executed now (within observer).
* restoring currently banned ip after service restart fixed
(now < timeofban + bantime), ignore old log failures (already banned)
* upgrade database: update new created table `bips` with entries from table `bans` (allows restore
current bans after upgrade from version <= 0.10)
### New Features
* Increment ban time (+ observer) functionality introduced.
* Database functionality extended with bad ips.
* New tags (usable in actions):
- `<bancount>` - ban count of this offender if known as bad (started by 1 for unknown)
- `<bantime>` - current ban-time of the ticket (prolongation can be retarded up to 10 sec.)
* Introduced new action command `actionprolong` to prolong ban-time (e. g. set new timeout if expected);
Several actions (like ipset, etc.) rewritten using net logic with `actionprolong`.
Note: because ban-time is dynamic, it was removed from jail.conf as timeout argument (check jail.local).
### Enhancements
* algorithm of restore current bans after restart changed: update the restored ban-time (and therefore
end of ban) of the ticket with ban-time of jail (as maximum), for all tickets with ban-time greater
(or persistent); not affected if ban-time of the jail is unchanged between stop/start.
* added new setup-option `--without-tests` to skip building and installing of tests files (gh-2287).
* added new command `fail2ban-client get <JAIL> banip ?sep-char|--with-time?` to get the banned ip addresses (gh-1916).
ver. 0.10.5 (2020/01/10) - deserve-more-respect-a-jedis-weapon-must
-----------
Yes, Hrrrm...
### Fixes
* [compatibility] systemd backend: default flags changed to SYSTEM_ONLY(4), fixed in gh-2444 in order to ignore
user session files per default, so could prevent "Too many open files" errors on a lot of user sessions (see gh-2392)
* [grave] fixed parsing of multi-line filters (`maxlines` > 1) together with systemd backend,
now systemd-filter replaces newlines in message from systemd journal with `\n` (otherwise
multi-line parsing may be broken, because removal of matched string from multi-line buffer window
is confused by such extra new-lines, so they are retained and got matched on every followed
message, see gh-2431)
* [stability] prevent race condition - no unban if the bans occur continuously (gh-2410);
now an unban-check will happen not later than 10 tickets get banned regardless there are
still active bans available (precedence of ban over unban-check is 10 now)
* fixed read of included config-files (`.local` overwrites options of `.conf` for config-files
included with before/after)
* `action.d/abuseipdb.conf`: switched to use AbuseIPDB API v2 (gh-2302)
* `action.d/badips.py`: fixed start of banaction on demand (which may be IP-family related), gh-2390
* `action.d/helpers-common.conf`: rewritten grep arguments, now options `-wF` used to match only
whole words and fixed string (not as pattern), gh-2298
* `filter.d/apache-auth.conf`:
- ignore errors from mod_evasive in `normal` mode (mode-controlled now) (gh-2548);
- extended with option `mode` - `normal` (default) and `aggressive`
* `filter.d/sshd.conf`:
- matches `Bad protocol version identification` in `ddos` and `aggressive` modes (gh-2404).
- captures `Disconnecting ...: Change of username or service not allowed` (gh-2239, gh-2279)
- captures `Disconnected from ... [preauth]`, preauth phase only, different handling by `extra`
(with supplied user only) and `ddos`/`aggressive` mode (gh-2115, gh-2239, gh-2279)
* `filter.d/mysqld-auth.conf`:
- MYSQL 8.0.13 compatibility (log-error-verbosity = 3), log-format contains few additional words
enclosed in brackets after "[Note]" (gh-2314)
* `filter.d/sendmail-reject.conf`:
- `mode=extra` now captures port IDs of `TLSMTA` and `MSA` (defaults for ports 465 and 587 on some distros)
* `files/fail2ban.service.in`: fixed systemd-unit template - missing nftables dependency (gh-2313)
* several `action.d/mail*`: fixed usage with multiple log files (ultimate fix for gh-976, gh-2341)
* `filter.d/sendmail-reject.conf`: fixed journal usage for some systems (e. g. CentOS): if only identifier
set to `sm-mta` (no unit `sendmail`) for some messages (gh-2385)
* `filter.d/asterisk.conf`: asterisk can log additional timestamp if logs into systemd-journal
(regex extended with optional part matching this, gh-2383)
* `filter.d/postfix.conf`:
- regexp's accept variable suffix code in status of postfix for precise messages (gh-2442)
- extended with new postfix filter mode `errors` to match "too many errors" (gh-2439),
also included within modes `normal`, `more` (`extra` and `aggressive`), since postfix
parameter `smtpd_hard_error_limit` is default 20 (additionally consider `maxretry`)
* `filter.d/named-refused.conf`:
- support BIND 9.11.0 log format (includes an additional field @0xXXX..., gh-2406);
- `prefregex` extended, more selective now (denied/NOTAUTH suffix moved from failregex, so no catch-all there anymore)
* `filter.d/sendmail-auth.conf`, `filter.d/sendmail-reject.conf` :
- ID in prefix can be longer as 14 characters (gh-2563);
* all filters would accept square brackets around IPv4 addresses also (e. g. monit-filter, gh-2494)
* avoids unhandled exception during flush (gh-2588)
* fixes pass2allow-ftp jail - due to inverted handling, action should prohibit access per default for any IP,
therefore reset start on demand parameter for this action (it will be started immediately by repair);
* auto-detection of IPv6 subsystem availability (important for not on-demand actions or jails, like pass2allow);
### New Features
* new replacement tags for failregex to match subnets in form of IP-addresses with CIDR mask (gh-2559):
- `<CIDR>` - helper regex to match CIDR (simple integer form of net-mask);
- `<SUBNET>` - regex to match sub-net adresses (in form of IP/CIDR, also single IP is matched, so part /CIDR is optional);
* grouped tags (`<ADDR>`, `<HOST>`, `<SUBNET>`) recognize IP addresses enclosed in square brackets
* new failregex-flag tag `<F-MLFGAINED>` for failregex, signaled that the access to service was gained
(ATM used similar to tag `<F-NOFAIL>`, but it does not add the log-line to matches, gh-2279)
* filters: introduced new configuration parameter `logtype` (default `file` for file-backends, and
`journal` for journal-backends, gh-2387); can be also set to `rfc5424` to force filters (which include common.conf)
to use RFC 5424 conform prefix-line per default (gh-2467);
* for better performance and safety the option `logtype` can be also used to
select short prefix-line for file-backends too for all filters using `__prefix_line` (`common.conf`),
if message logged only with `hostname svc[nnnn]` prefix (often the case on several systems):
```ini
[jail]
backend = auto
filter = flt[logtype=short]
```
* `filter.d/common.conf`: differentiate `__prefix_line` for file/journal logtype's (speedup and fix parsing
of systemd-journal);
* `filter.d/traefik-auth.conf`: used to ban hosts, that were failed through traefik
* `filter.d/znc-adminlog.conf`: new filter for ZNC (IRC bouncer); requires the adminlog module to be loaded
### Enhancements
* introduced new options: `dbmaxmatches` (fail2ban.conf) and `maxmatches` (jail.conf) to contol
how many matches per ticket fail2ban can hold in memory and store in database (gh-2402, gh-2118);
* fail2ban.conf: introduced new section `[Thread]` and option `stacksize` to configure default size
of the stack for threads running in fail2ban (gh-2356), it could be set in `fail2ban.local` to
avoid runtime error "can't start new thread" (see gh-969);
* jail-reader extended (amend to gh-1622): actions support multi-line options now (interpolations
containing new-line);
* fail2ban-client: extended to ban/unban multiple tickets (see gh-2351, gh-2349);
Syntax:
- `fail2ban-client set <jain> banip <ip1> ... <ipN>`
- `fail2ban-client set <jain> unbanip [--report-absent] <ip1> ... <ipN>`
* fail2ban-client: extended with new feature which allows to inform fail2ban about single or multiple
attempts (failure) for IP (resp. failure-ID), see gh-2351;
Syntax:
- `fail2ban-client set <jail> attempt <ip> [<failure-message1> ... <failure-messageN>]`
* `action.d/nftables.conf`:
- isolate fail2ban rules into a dedicated table and chain (gh-2254)
- `nftables-allports` supports multiple protocols in single rule now
- combined nftables actions to single action `nftables`:
* `nftables-common` is removed (replaced with single action `nftables` now)
* `nftables-allports` is obsolete, superseded by `nftables[type=allports]`
* `nftables-multiport` is obsolete, superseded by `nftables[type=multiport]`
- allowed multiple protocols in `nftables[type=multiport]` action (single set with multiple rules
in chain), following configuration in jail would replace 3 separate actions, see
https://github.com/fail2ban/fail2ban/pull/2254#issuecomment-534684675
* `action.d/badips.py`: option `loglevel` extended with level of summary message,
following example configuration logging summary with NOTICE and rest with DEBUG log-levels:
`action = badips.py[loglevel="debug, notice"]`
* samplestestcase.py (testSampleRegexsFactory) extended:
- allow coverage of journal logtype;
- new option `fileOptions` to set common filter/test options for whole test-file;
* large enhancement: auto-reban, improved invariant check and conditional operations (gh-2588):
- improves invariant check and repair (avoid unhandled exception, consider family on conditional operations, etc),
prepared for bulk re-ban in repair case (if bulk-ban becomes implemented);
- automatic reban (repeat banning action) after repair/restore sane environment, if already logged ticket causes
new failures (via new action operation `actionreban` or `actionban` if still not defined in action);
* introduces banning epoch for actions and tickets (to distinguish or recognize removed set of the tickets);
* invariant check avoids repair by unban/stop (unless parameter `actionrepair_on_unban` set to `true`);
* better handling for all conditional operations (distinguish families for certain operations like
repair/flush/stop, prepared for other families, e. g. if different handling for subnets expected, etc);
* partially implements gh-980 (more breakdown safe handling);
* closes gh-1680 (better as large-scale banning implementation with on-demand reban by failure,
at least unless a bulk-ban gets implemented);
* fail2ban-regex - several enhancements and fixes:
- improved usage output (don't put a long help if an error occurs);
- new option `--no-check-all` to avoid check of all regex's (first matched only);
- new option `-o`, `--out` to set token only provided in output (disables check-all and outputs only expected data).
ver. 0.10.4 (2018/10/04) - ten-four-on-due-date-ten-four
-----------
### Fixes
* `filter.d/dovecot.conf`:
- failregex enhancement to catch sql password mismatch errors (gh-2153);
- disconnected with "proxy dest auth failed" (gh-2184);
* `filter.d/freeswitch.conf`:
- provide compatibility for log-format from gh-2193:
* extended with new default date-pattern `^(?:%%Y-)?%%m-%%d[ T]%%H:%%M:%%S(?:\.%%f)?` to cover
`YYYY-mm-dd HH:MM::SS.ms` as well as `mm-dd HH:MM::SS.ms` (so year is optional);
* more optional arguments in log-line (so accept [WARN] as well as [WARNING] and optional [SOFIA] hereafter);
- extended with mode parameter, allows to avoid matching of messages like `auth challenge (REGISTER)`
(see gh-2163) (currently `extra` as default to be backwards-compatible), see comments in filter
how to set it to mode `normal`.
* `filter.d/domino-smtp.conf`:
- recognizes failures logged using another format (something like session-id, IP enclosed in square brackets);
- failregex extended to catch connections rejected for policy reasons (gh-2228);
* `action.d/hostsdeny.conf`: fix parameter in config (dynamic parameters stating with '_' are protected
and don't allowed in command-actions), see gh-2114;
* decoding stability fix by wrong encoded characters like utf-8 surrogate pairs, etc (gh-2171):
- fail2ban running in the preferred encoding now (as default encoding also within python 2.x), mostly
`UTF-8` in opposite to `ascii` previously, so minimizes influence of implicit conversions errors;
- actions: avoid possible conversion errors on wrong-chars by replace tags;
- database: improve adapter/converter handlers working on invalid characters in sense of json and/or sqlite-database;
additionally both are exception-safe now, so avoid possible locking of database (closes gh-2137);
- logging in fail2ban is process-wide exception-safe now.
* repaired start-time of initial seek to time (as well as other log-parsing related data),
if parameter `logpath` specified before `findtime`, `backend`, `datepattern`, etc (gh-2173)
* systemd: fixed type error on option `journalflags`: an integer is required (gh-2125);
### New Features
* new option `ignorecache` to improve performance of ignore failure check (using caching of `ignoreip`,
`ignoreself` and `ignorecommand`), see `man jail.conf` for syntax-example;
* `ignorecommand` extended to use actions-similar replacement (capable to interpolate
all possible tags like `<ip-host>`, `<family>`, `<fid>`, `F-USER` etc.)
### Enhancements
* `filter.d/dovecot.conf`: extended with tags F-USER (and alternatives) to collect user-logins (gh-2168)
* since v.0.10.4, fail2ban-client, fail2ban-server and fail2ban-regex will return version without logo info,
additionally option `-V` can be used to get version in normalized machine-readable short format.
ver. 0.10.3 (2018/04/04) - the-time-is-always-right-to-do-what-is-right
-----------
### ver. 0.10.3.1:
* fixed JSON serialization for the set-object within dump into database (gh-2103).
### Fixes
* `filter.d/asterisk.conf`: fixed failregex prefix by log over remote syslog server (gh-2060);
* `filter.d/exim.conf`: failregex extended - SMTP call dropped: too many syntax or protocol errors (gh-2048);
* `filter.d/recidive.conf`: fixed if logging into systemd-journal (SYSLOG) with daemon name in prefix, gh-2069;
* `filter.d/sendmail-auth.conf`, `filter.d/sendmail-reject.conf` :
- fixed failregex, sendmail uses prefix 'IPv6:' logging of IPv6 addresses (gh-2064);
* `filter.d/sshd.conf`:
- failregex got an optional space in order to match new log-format (see gh-2061);
- fixed ddos-mode regex to match refactored message (some versions can contain port now, see gh-2062);
- fixed root login refused regex (optional port before preauth, gh-2080);
- avoid banning of legitimate users when pam_unix used in combination with other password method, so
bypass pam_unix failures if accepted available for this user gh-2070;
- amend to gh-1263 with better handling of multiple attempts (failures for different user-names recognized immediatelly);
- mode `ddos` (and `aggressive`) extended to catch `Connection closed by ... [preauth]`, so in DDOS mode
it counts failure on closing connection within preauth-stage (gh-2085);
* `action.d/abuseipdb.conf`: fixed curl cypher errors and comment quote-issue (gh-2044, gh-2101);
* `action.d/badips.py`: implicit convert IPAddr to str, solves an issue "expected string, IPAddr found" (gh-2059);
* `action.d/hostsdeny.conf`: fixed IPv6 syntax (enclosed in square brackets, gh-2066);
* (Free)BSD ipfw actionban fixed to allow same rule added several times (gh-2054);
### New Features
* several stability and performance optimizations, more effective filter parsing, etc;
* stable runnable within python versions 3.6 (as well as within 3.7-dev);
### Enhancements
* `filter.d/apache-auth.conf`: detection of Apache SNI errors resp. misredirect attempts (gh-2017, gh-2097);
* `filter.d/apache-noscript.conf`: extend failregex to match "Primary script unknown", e. g. from php-fpm (gh-2073);
* date-detector extended with long epoch (`LEPOCH`) to parse milliseconds/microseconds posix-dates (gh-2029);
* possibility to specify own regex-pattern to match epoch date-time, e. g. `^\[{EPOCH}\]` or `^\[{LEPOCH}\]` (gh-2038);
the epoch-pattern similar to `{DATE}` patterns does the capture and cuts out the match of whole pattern from the log-line,
e. g. date-pattern `^\[{LEPOCH}\]\s+:` will match and cut out `[1516469849551000] :` from begin of the log-line.
* badips.py now uses https instead of plain http when requesting badips.com (gh-2057);
* add support for "any" badips.py bancategory, to be able to retrieve IPs from all categories with a desired score (gh-2056);
* Introduced new parameter `padding` for logging within fail2ban-server (default on, excepting SYSLOG):
Usage `logtarget = target[padding=on|off]`
ver. 0.10.2 (2018/01/18) - nothing-burns-like-the-cold
-----------
### Incompatibility list:
* The configuration for jails using banaction `pf` can be incompatible after upgrade, because pf-action uses
anchors now (see `action.d/pf.conf` for more information). If you want use obsolete handling without anchors,
just rewrite it in the `jail.local` by overwrite of `pfctl` parameter, e. g. like `banaction = pf[pfctl="pfctl"]`.
### Fixes
* Fixed logging to systemd-journal: new logtarget value SYSOUT can be used instead of STDOUT, to avoid
write of the time-stamp, if logging to systemd-journal from foreground mode (gh-1876)
* Fixed recognition of the new date-format on mysqld-auth filter (gh-1639)
* jail.conf: port `imap3` replaced with `imap` everywhere, since imap3 is not a standard port and old rarely
(if ever) used and can missing on some systems (e. g. debian stretch), see gh-1942.
* config/paths-common.conf: added missing initial values (and small normalization in config/paths-*.conf)
in order to avoid errors while interpolating (e. g. starting with systemd-backend), see gh-1955.
* `action.d/pf.conf`:
- fixed syntax error in achnor definition (documentation, see gh-1919);
- enclose ports in braces for multiport jails (see gh-1925);
* `action.d/firewallcmd-ipset.conf`: fixed create of set for ipv6 (missing `family inet6`, gh-1990)
* `filter.d/sshd.conf`:
- extended failregex for modes "extra"/"aggressive": now finds all possible (also future)
forms of "no matching (cipher|mac|MAC|compression method|key exchange method|host key type) found",
see "ssherr.c" for all possible SSH_ERR_..._ALG_MATCH errors (gh-1943, gh-1944);
- fixed failregex in order to avoid banning of legitimate users with multiple public keys (gh-2014, gh-1263);
### New Features
* datedetector: extended default date-patterns (allows extra space between the date and time stamps);
introduces 2 new format directives (with corresponding %Ex prefix for more precise parsing):
- %k - one- or two-digit number giving the hour of the day (0-23) on a 24-hour clock,
(corresponds %H, but allows space if not zero-padded).
- %l - one- or two-digit number giving the hour of the day (12-11) on a 12-hour clock,
(corresponds %I, but allows space if not zero-padded).
* `filter.d/exim.conf`: added mode `aggressive` to ban flood resp. DDOS-similar failures (gh-1983);
* New Actions:
- `action.d/nginx-block-map.conf` - in order to ban not IP-related tickets via nginx (session blacklisting in
nginx-location with map-file);
### Enhancements
* jail.conf: extended with new parameter `mode` for the filters supporting it (gh-1988);
* action.d/pf.conf: extended with bulk-unban, command `actionflush` in order to flush all bans at once.
* Introduced new parameters for logging within fail2ban-server (gh-1980).
Usage `logtarget = target[facility=..., datetime=on|off, format="..."]`:
- `facility` - specify syslog facility (default `daemon`, see https://docs.python.org/2/library/logging.handlers.html#sysloghandler
for the list of facilities);
- `datetime` - add date-time to the message (default on, ignored if `format` specified);
- `format` - specify own format how it will be logged, for example for short-log into STDOUT:
`fail2ban-server -f --logtarget 'stdout[format="%(relativeCreated)5d | %(message)s"]' start`;
* Automatically recover or recreate corrupt persistent database (e. g. if failed to open with
'database disk image is malformed'). Fail2ban will create a backup, try to repair the database,
if repair fails - recreate new database (gh-1465, gh-2004).
ver. 0.10.1 (2017/10/12) - succeeded-before-friday-the-13th
-----------
### Fixes
* fix Gentoo init script's shebang to use openrc-run instead of runscript (gh-1891)
* jail "pass2allow-ftp" supply blocktype and returntype parameters to the action (gh-1884)
* avoid using "ANSI_X3.4-1968" as preferred encoding (if missing environment variables
'LANGUAGE', 'LC_ALL', 'LC_CTYPE', and 'LANG', see gh-1587).
* action.d/pf.conf: several fixes for pf-action like anchoring, etc. (see gh-1866, gh-1867);
* fixed ignoreself issue "Retrieving own IPs of localhost failed: inet_pton() argument 2 must be string, not int" (see gh-1865);
* fixed tags `<fq-hostname>` and `<sh-hostname>`, could be used without ticket (a. g. in `actionstart` etc., gh-1859).
* setup.py: fixed several setup facilities (gh-1874):
- don't check return code by dry-run: returns 256 on some python/setuptool versions;
- `files/fail2ban.service` renamed as template to `files/fail2ban.service.in`;
- setup process generates `build/fail2ban.service` from `files/fail2ban.service.in` using distribution related bin-path;
- bug-fixing by running setup with option `--dry-run`;
### New Features
* introduced new command-line options `--dp`, `--dump-pretty` to dump the configuration using more
human readable representation (opposite to `-d`);
### Enhancements
* nftables actions are IPv6-capable now (gh-1893)
* filter.d/dovecot.conf: introduced mode `aggressive` for cases like "disconnected before auth was ready" (gh-1880)
ver. 0.10.0 (2017/08/09) - long-awaited 0.10th version
-----------
TODO: implementing of options resp. other tasks from PR #1346
documentation should be extended (new options, etc)
### Fixes
* `filter.d/apache-auth.conf`:
- better failure recognition using short form of regex (url/referer are foreign inputs, see gh-1645)
* `filter.d/apache-common.conf` (`filter.d/apache-*.conf`):
- support of apache log-format if logging into syslog/systemd (gh-1695), using parameter `logging`,
parameter usage for jail:
filter = apache-auth[logging=syslog]
parameter usage for `apache-common.local`:
logging = syslog
* `filter.d/pam-generic.conf`:
- [grave] injection on user name to host fixed
* `filter.d/sshd.conf`:
- rewritten using `prefregex` and used MLFID-related multi-line parsing
(by using tag `<F-MLFID>` instead of buffering with `maxlines`);
- optional parameter `mode` rewritten: normal (default), ddos, extra or aggressive (combines all),
see sshd for regex details)
* `filter.d/sendmail-reject.conf`:
- rewritten using `prefregex` and used MLFID-related multi-line parsing;
- optional parameter `mode` introduced: normal (default), extra or aggressive
* `filter.d/haproxy-http-auth`: do not mistake client port for part of an IPv6 address (gh-1745)
* `filter.d/postfix.conf`:
- updated to latest postfix formats
- joined several postfix filter together (normalized and optimized version, gh-1825)
- introduced new parameter `mode` (see gh-1825): more (default, combines normal and rbl), auth, normal,
rbl, ddos, extra or aggressive (combines all)
- postfix postscreen (resp. other RBL's compatibility fix, gh-1764, gh-1825)
* `filter.d/postfix-rbl.conf`: removed (replaced with `postfix[mode=rbl]`)
* `filter.d/postfix-sasl.conf`: removed (replaced with `postfix[mode=auth]`)
* `filter.d/roundcube-auth.conf`:
- fixed regex when `X-Real-IP` or/and `X-Forwarded-For` are present after host (gh-1303);
- fixed regex when logging authentication errors to journal instead to a local file (gh-1159);
- additionally fixed more complex injections on username (e. g. using dot after fake host).
* `filter.d/ejabberd-auth.conf`: fixed failregex - accept new log-format (gh-993)
* `action.d/complain.conf`
- fixed using new tag `<ip-rev>` (sh/dash compliant now)
* `action.d/sendmail-geoip-lines.conf`
- fixed using new tag `<ip-host>` (without external command execution)
* fail2ban-regex: fixed matched output by multi-line (buffered) parsing
* fail2ban-regex: support for multi-line debuggex URL implemented (gh-422)
* fixed ipv6-action errors on systems not supporting ipv6 and vice versa (gh-1741)
* fixed directory-based log-rotate for pyinotify-backend (gh-1778)
### New Features
* New Actions:
* New Filters:
### Enhancements
* Introduced new filter option `prefregex` for pre-filtering using single regular expression (gh-1698);
* Many times faster and fewer CPU-hungry because of parsing with `maxlines=1`, so without
line buffering (scrolling of the buffer-window).
Combination of tags `<F-MLFID>` and `<F-NOFAIL>` can be used now to process multi-line logs
using single-line expressions:
- tag `<F-MLFID>`: used to identify resp. store failure info for groups of log-lines with the same
identifier (e. g. combined failure-info for the same conn-id by `<F-MLFID>(?:conn-id)</F-MLFID>`,
see sshd.conf for example);
- tag `<F-MLFFORGET>`: can be used as mark to forget current multi-line MLFID (e. g. by connection
closed, reset or disconnect etc);
- tag `<F-NOFAIL>`: used as mark for no-failure (helper to accumulate common failure-info,
e. g. from lines that contain IP-address);
Opposite to obsolete multi-line parsing (using buffering with `maxlines`) it is more precise and
can recognize multiple failure attempts within the same connection (MLFID).
* Several filters optimized with pre-filtering using new option `prefregex`, and multiline filter
using `<F-MLFID>` + `<F-NOFAIL>` combination;
* Exposes filter group captures in actions (non-recursive interpolation of tags `<F-...>`,
see gh-1698, gh-1110)
* Some filters extended with user name (can be used in gh-1243 to distinguish IP and user,
resp. to remove after success login the user-related failures only);
* Safer, more stable and faster replaceTag interpolation (switched from cycle over all tags
to re.sub with callable)
* substituteRecursiveTags optimization + moved in helpers facilities (because currently used
commonly in server and in client)
* New tags (usable in actions):
- `<fid>` - failure identifier (if raw resp. failures without IP address)
- `<ip-rev>` - PTR reversed representation of IP address
- `<ip-host>` - host name of the IP address
- `<bancount>` - ban count of this offender if known as bad (started by 1 for unknown)
- `<bantime>` - current ban-time of the ticket (prolongation can be retarded up to 10 sec.)
- `<F-...>` - interpolates to the corresponding filter group capture `...`
- `<fq-hostname>` - fully-qualified name of host (the same as `$(hostname -f)`)
- `<sh-hostname>` - short hostname (the same as `$(uname -n)`)
* Introduced new action command `actionprolong` to prolong ban-time (e. g. set new timeout if expected);
Several actions (like ipset, etc.) rewritten using net logic with `actionprolong`.
Note: because ban-time is dynamic, it was removed from jail.conf as timeout argument (check jail.local).
* Allow to use filter options by `fail2ban-regex`, example:
fail2ban-regex text.log "sshd[mode=aggressive]"
* Samples test case factory extended with filter options - dict in JSON to control
filter options (e. g. mode, etc.):
# filterOptions: {"mode": "aggressive"}
* Introduced new jail option "ignoreself", specifies whether the local resp. own IP addresses
should be ignored (default is true). Fail2ban will not ban a host which matches such addresses.
Option "ignoreip" affects additionally to "ignoreself" and don't need to include the DNS
resp. IPs of the host self.
* Regex will be compiled as MULTILINE only if needed (buffering with `maxlines` > 1), that enables:
- to improve performance by the single line parsing (see gh-1733);
- make regex more precise (because distinguish between anchors `^`/`$` for the begin/end of string
and the new-line character '\n', e. g. if coming from filters (like systemd journal) that allow
the parsing of log-entries contain new-line chars (as single entry);
- if multiline regex however expected (by single-line parsing without buffering) - prefix `(?m)`
could be used in regex to enable it;
* Implemented execution of `actionstart` on demand (conditional), if action depends on `family` (gh-1742):
- new action parameter `actionstart_on_demand` (bool) can be set to prevent/allow starting action
on demand (default retrieved automatically, if some conditional parameter `param?family=...`
presents in action properties), see `action.d/pf.conf` for example;
- additionally `actionstop` will be executed only for families previously executing `actionstart`
(starting on demand only)
* Introduced new command `actionflush`: executed in order to flush all bans at once
e. g. by unban all, reload with removing action, stop, shutdown the system (gh-1743),
the actions having `actionflush` do not execute `actionunban` for each single ticket
* Add new command `actionflush` default for several iptables/iptables-ipset actions (and common include);
* Add new jail option `logtimezone` to force the timezone on log lines that don't have an explicit one (gh-1773)
* Implemented zone abbreviations (like CET, CEST, etc.) and abbr+-offset functionality (accept zones
like 'CET+0100'), for the list of abbreviations see strptime.TZ_STR;
* Introduced new option `--timezone` (resp. `--TZ`) for `fail2ban-regex`.
* Tokens `%z` and `%Z` are changed (more precise now);
* Introduced new tokens `%Exz` and `%ExZ` that fully support zone abbreviations and/or offset-based
zones (implemented as enhancement using custom `datepattern`, because may be too dangerous for default
patterns and tokens like `%z`);
Note: the extended tokens supported zone abbreviations, but it can parse 1 or 3-5 char(s) in lowercase.
Don't use them in default date-patterns (if not anchored, few precise resp. optional).
Because python currently does not support mixing of case-sensitive with case-insensitive matching,
the TZ (in uppercase) cannot be combined with `%a`/`%b` etc (that are currently case-insensitive),
to avoid invalid date-time recognition in strings like '11-Aug-2013 03:36:11.372 error ...' with
wrong TZ "error".
Hence `%z` currently match literal Z|UTC|GMT only (and offset-based), and `%Exz` - all zone
abbreviations.
* `filter.d/courier-auth.conf`: support failed logins with method only
* Config reader's: introduced new syntax `%(section/option)s`, in opposite to extended interpolation of
python 3 `${section:option}` work with all supported python version in fail2ban and this syntax is
like our another features like `%(known/option)s`, etc. (gh-1750)
* Variable `default_backend` switched to `%(default/backend)s`, so totally backwards compatible now,
but now the setting of parameter `backend` in default section of `jail.local` can overwrite default
backend also (see gh-1750). In the future versions parameter `default_backend` can be removed (incompatibility,
possibly some distributions affected).
ver. 0.10.0-alpha-1 (2016/07/14) - ipv6-support-etc
-----------
### Fixes
* [Grave] memory leak's fixed (gh-1277, gh-1234)
* [Grave] Misleading date patterns defined more precisely (using extended syntax
`%Ex[mdHMS]` for exact two-digit match or e. g. `%ExY` as more precise year
pattern, within same century of last year and the next 3 years)
* [Grave] extends date detector template with distance (position of match in
log-line), to prevent grave collision using (re)ordered template list (e.g.
find-spot of wrong date-match inside foreign input, misleading date patterns
by ambiguous formats, etc.)
* Distance collision check always prefers template with shortest distance
(left for right) if date pattern is not anchored
* Tricky bug fix: last position of log file will be never retrieved (gh-795),
because of CASCADE all log entries will be deleted from logs table together with jail,
if used "INSERT OR REPLACE" statement
* Asyncserver (asyncore) code fixed and test cases repaired (again gh-161)
* testSocket: sporadical bug repaired - wait for server thread starts a socket (listener)
* testExecuteTimeoutWithNastyChildren: sporadical bug repaired - wait for pid file inside bash,
kill tree in any case (gh-1155)
* purge database will be executed now (within observer).
* restoring currently banned ip after service restart fixed
(now < timeofban + bantime), ignore old log failures (already banned)
* Fixed high-load of pyinotify-backend,
see https://github.com/fail2ban/fail2ban/issues/885#issuecomment-248964591
* Database: stability fix - repack cursor iterator as long as locked
* File filter backends: stability fix for sporadically errors - always close file
handle, otherwise may be locked (prevent log-rotate, etc.)
* Pyinotify-backend: stability fix for sporadically errors in multi-threaded
environment (without lock)
* Fixed sporadically error in testCymruInfoNxdomain, because of unsorted values
* Misleading errors logged from ignorecommand in success case on retcode 1 (gh-1194)
* fail2ban.service - systemd service updated (gh-1618):
- starting service in normal mode (without forking)
- does not restart if service exited normally (exit-code 0, e.g. stopped via fail2ban-client)
- does not restart if service can not start (exit-code 255, e.g. wrong configuration, etc.)
- service can be additionally started/stopped with commands (fail2ban-client, fail2ban-server)
- automatically creates `/var/run/fail2ban` directory before start fail2ban
(systems with virtual resp. memory-based FS for `/var/run`), see gh-1531
- if fail2ban running as systemd-service, for logging to the systemd-journal,
the `logtarget` could be set to STDOUT
- value `logtarget` for system targets allowed also in lowercase (stdout, stderr, syslog, etc.)
* Fixed UTC/GMT named time zone, using `%Z` and `%z` patterns
(special case with 0 zone offset, see gh-1575)
* `filter.d/freeswitch.conf`
- Optional prefixes (server, daemon, dual time) if systemd daemon logs used (gh-1548)
- User part rewritten to accept IPv6 resp. domain after "@" (gh-1548)
### New Features
* IPv6 support:
- IP addresses are now handled as objects rather than strings capable for
handling both address types IPv4 and IPv6
- iptables related actions have been amended to support IPv6 specific actions
additionally
- hostsdeny and route actions have been tested to be aware of v4 and v6 already
- pf action for *BSD systems has been improved and supports now also v4 and v6
- name resolution is now working for either address type
- new conditional section functionality used in config resp. includes:
- [Init?family=inet4] - IPv4 qualified hosts only
- [Init?family=inet6] - IPv6 qualified hosts only
* Increment ban time (+ observer) functionality introduced.
Thanks Serg G. Brester (sebres)
* Database functionality extended with bad ips.
* New reload functionality (now totally without restart, unbanning/rebanning, etc.),
see gh-1557
* Several commands extended and new commands introduced:
- `restart [--unban] [--if-exists] <JAIL>` - restarts the jail \<JAIL\>
(alias for `reload --restart ... <JAIL>`)
- `reload [--restart] [--unban] [--all]` - reloads the configuration without restarting
of the server, the option `--restart` activates completely restarting of affected jails,
thereby can unban IP addresses (if option `--unban` specified)
- `reload [--restart] [--unban] [--if-exists] <JAIL>` - reloads the jail \<JAIL\>,
or restarts it (if option `--restart` specified), at the same time unbans all IP addresses
banned in this jail, if option `--unban` specified
- `unban --all` - unbans all IP addresses (in all jails and database)
- `unban <IP> ... <IP>` - unbans \<IP\> (in all jails and database) (see gh-1388)
- introduced new option `-t` or `--test` to test configuration resp. start server only
if configuration is clean (fails by wrong configured jails if option `-t` specified)
* New command action parameter `actionrepair` - command executed in order to restore
sane environment in error case of `actioncheck`.
* Reporting via abuseipdb.com:
- Bans can now be reported to abuseipdb
- Catagories must be set in the config
- Relevant log lines included in report
### Enhancements
* Huge increasing of fail2ban performance and especially test-cases performance (see gh-1109)
* Datedetector: in-place reordering using hits and last used time:
matchTime, template list etc. rewritten because of performance degradation
* Prevent out of memory situation if many IP's makes extremely many failures (maxEntries)
* Introduced string to seconds (str2seconds) for configuration entries with time,
use `1h` instead of `3600`, `1d` instead of `86400`, etc
* seekToTime - prevent completely read of big files first time (after start of service),
initial seek to start time using half-interval search algorithm (see issue gh-795)
* Ticket and some other modules prepared to easy merge with newest version of 'ban-time-incr'
* Cache dnsToIp, ipToName to prevent long wait during retrieving of ip/name,
especially for wrong dns or lazy dns-system
* FailManager memory-optimization: increases performance,
prevents memory leakage, because don't copy failures list on some operations
* fail2ban-testcases - new options introduced:
- `-f`, `--fast` to decrease wait intervals, avoid passive waiting, and skip
few very slow test cases (implied memory database, see `-m` and no gamin tests `-g`)
- `-g`, `--no-gamin` to prevent running of tests that require the gamin (slow)
- `-m`, `--memory-db` - run database tests using memory instead of file
- `-i`, `--ignore` - negate [regexps] filter to ignore tests matched specified regexps
* Background servicing: prevents memory leak on some platforms/python versions, using forced GC
in periodic intervals (latency and threshold)
* executeCmd partially moved from action to new module utils
* Several functionality of class `DNSUtils` moved to new class `IPAddr`,
both classes moved to new module `ipdns`
* Pseudo-conditional section introduced, for conditional substitution resp.
evaluation of parameters for different family qualified hosts,
syntax `[Section?family=inet6]` (currently use for IPv6-support only).
* All the backends were rewritten to get reload-possibility, performance increased,
so fewer greedy regarding cpu- resp. system-load now
* Numeric log-level allowed now in server (resp. fail2ban.conf);
* Implemented better error handling in some multi-threaded routines; shutdown of jails
rewritten (faster and safer, does not breaks shutdown process if some error occurred)
* Possibility for overwriting some configuration options (read with config-readers)
with command line option, e. g.:
```bash
## start server with DEBUG log-level (ignore level read from fail2ban.conf):
fail2ban-client --loglevel DEBUG start
## or
fail2ban-server -c /cfg/path --loglevel DEBUG start
## keep server log-level by reload (without restart it)
fail2ban-client --loglevel DEBUG reload
## switch log-level back to INFO:
fail2ban-client set loglevel INFO
```
* Optimized BanManager: increase performance, fewer system load, try to prevent
memory leakage:
- better ban/unban handling within actions (e.g. used dict instead of list)
- don't copy bans resp. its list on some operations;
- added new unbantime handling to relieve unBanList (prevent permanent
searching for tickets to unban)
- prefer failure-ID as identifier of the ticket to its IP (most of the time
the same, but it can be something else e.g. user name in some complex jails,
as introduced in 0.10)
* Regexp enhancements:
- build replacement of `<HOST>` substitution corresponding parameter
`usedns` - dns-part will be added only if `usedns` is not `no`,
also using fail2ban-regex
- new replacement for `<ADDR>` in opposition to `<HOST>`, for separate
usage of 2 address groups only (regardless of `usedns`), `ip4` and `ip6`
together, without host (dns)
* Misconfigured jails don't prevent fail2ban from starting, server starts
nevertheless, as long as one jail was successful configured (gh-1619)
Message about wrong jail configuration logged in client log (stdout, systemd
journal etc.) and in server log with error level
* More precise date template handling (WARNING: theoretically possible incompatibilities):
- datedetector rewritten more strict as earlier;
- default templates can be specified exacter using prefix/suffix syntax (via `datepattern`);
- more as one date pattern can be specified using option `datepattern` now
(new-line separated);
- some default options like `datepattern` can be specified directly in
section `[Definition]`, that avoids contrary usage of unnecessarily `[Init]`
section, because of performance (each extra section costs time);
- option `datepattern` can be specified in jail also (e. g. jails without filters
or custom log-format, new-line separated for multiple patterns);
- if first unnamed group specified in pattern, only this will be cut out from
search log-line (e. g.: `^date:[({DATE})]` will cut out only datetime match
pattern, and leaves `date:[] ...` for searching in filter);
- faster match and fewer searching of appropriate templates
(DateDetector.matchTime calls rarer DateTemplate.matchDate now);
- several standard filters extended with exact prefixed or anchored date templates;
* Added possibility to recognize restored state of the tickets (see gh-1669).
New option `norestored` introduced, to ignore restored tickets (after restart).
To avoid execution of ban/unban for the restored tickets, `norestored = true`
could be added in definition section of action.
For conditional usage in the shell-based actions an interpolation `<restored>`
could be used also. E. g. it is enough to add following script-piece at begin
of `actionban` (or `actionunban`) to prevent execution:
`if [ '<restored>' = '1' ]; then exit 0; fi;`
Several actions extended now using `norestored` option:
- complain.conf
- dshield.conf
- mail-buffered.conf
- mail-whois-lines.conf
- mail-whois.conf
- mail.conf
- sendmail-buffered.conf
- sendmail-geoip-lines.conf
- sendmail-whois-ipjailmatches.conf
- sendmail-whois-ipmatches.conf
- sendmail-whois-lines.conf
- sendmail-whois-matches.conf
- sendmail-whois.conf
- sendmail.conf
- smtp.py
- xarf-login-attack.conf
* fail2ban-testcases:
- `assertLogged` extended with parameter wait (to wait up to specified timeout,
before we throw assert exception) + test cases rewritten using that
- added `assertDictEqual` for compatibility to early python versions (< 2.7);
- new `with_foreground_server_thread` decorator to test several client/server commands
ver. 0.9.8 (2016/XX/XXX) - wanna-be-released
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
### Fixes
* Fix for systemd-backend: fail2ban hits the ulimit (out of file descriptors), see gh-991.
Partially back-ported from v.0.10.
* action.d/bsd-ipfw.conf
- Make the rule number, the action starts looking for a free slot to insert
the new rule, configurable (gh-1689)
- Replace not posix-compliant grep option: fgrep with `-q` option can cause
141 exit code in some cases (gh-1389)
* filter.d/apache-overflows.conf:
- Fixes resources greedy expression (see gh-1790);
- Rewritten without end-anchor ($), because of potential vulnerability on very long URLs.
* filter.d/apache-badbots.conf - extended to recognize Jorgee Vulnerability Scanner (gh-1882)
* filter.d/asterisk.conf
- fixed failregex AMI Asterisk authentification failed (see gh-1302)
- removed invalid (vulnerable) regex blocking IPs using forign data (from header "from")
thus not the IP-address that really originates the request (see gh-1927)
- fixed failregex for the SQL-injection attempts with single-quotes in connect-string (see gh-2011)
* filter.d/dovecot.conf:
- fixed failregex, see gh-1879 (partially cherry-picked from gh-1880)
- extended to match pam_authenticate failures with "Permission denied" (gh-1897)
* filter.d/exim.conf
- fixed failregex for case of flood attempts with `D=0s` (gh-1887)
- fixed failregex of "AUTH command used when not advertised" to better handle the foreign
input SMTP command (lower/mixed case auth command, prevent injection) (gh-1979)
* filter.d/postfix-*.conf - added optional port regex (gh-1902)
* filter.d/sendmail-auth.conf - extended daemon for Fedora 24/RHEL - the daemon name is "sendmail" (gh-1632)
* filter.d/nginx-http-auth.conf - match usernames with spaces (gh-2015)
### New Features
### Enhancements
* action.d/cloudflare.conf - Cloudflare API v4 implementation (gh-1651)
* action.d/firewallcmd-ipset.conf - new parameter `actiontype`, provides `allports` capability (gh-1167)
* filter.d/kerio.conf - filter extended with new rules (see gh-1455)
* filter.d/phpmyadmin-syslog.conf - new filter for phpMyAdmin using syslog for auth logging
* filter.d/zoneminder.conf - new filter for ZoneMinder (gh-1376)
ver. 0.9.7 (2017/05/11) - awaiting-victory
-----------
### Fixes
* Fixed a systemd-journal handling in fail2ban-regex (gh-1657)
* filter.d/sshd.conf
- Fixed non-anchored part of failregex (misleading match of colon inside
IPv6 address instead of `: ` in the reason-part by missing space, gh-1658)
(0.10th resp. IPv6 relevant only, amend for gh-1479)
* config/pathes-freebsd.conf
- Fixed filenames for apache and nginx log files (gh-1667)
* filter.d/exim.conf
- optional part `(...)` after host-name before `[IP]` (gh-1751)
- new reason "Unrouteable address" for "rejected RCPT" regex (gh-1762)
- match of complex time like `D=2m42s` in regex "no MAIL in SMTP connection" (gh-1766)
* filter.d/sshd.conf
- new aggressive rules (gh-864):
- Connection reset by peer (multi-line rule during authorization process)
- No supported authentication methods available
- single line and multi-line expression optimized, added optional prefixes
and suffix (logged from several ssh versions), according to gh-1206;
- fixed expression received disconnect auth fail (optional space after port
part, gh-1652)
and suffix (logged from several ssh versions), according to gh-1206;
* filter.d/suhosin.conf
- greedy catch-all before `<HOST>` fixed (potential vulnerability)
* filter.d/cyrus-imap.conf
- accept entries without login-info resp. hostname before IP address (gh-1707)
* Filter tests extended with check of all config-regexp, that contains greedy catch-all
before `<HOST>`, that is hard-anchored at end or precise sub expression after `<HOST>`
### New Features
* New Actions:
- action.d/netscaler: Block IPs on a Citrix Netscaler ADC (gh-1663)
* New Filters:
- filter.d/domino-smtp: IBM Domino SMTP task (gh-1603)
### Enhancements
* Introduced new log-level `MSG` (as INFO-2, equivalent to 18)
ver. 0.9.6 (2016/12/10) - stretch-is-coming
-----------
### Fixes
* Misleading add resp. enable of (already available) jail in database, that
induced a subsequent error: last position of log file will be never retrieved (gh-795)
* Fixed a distribution related bug within testReadStockJailConfForceEnabled
(e.g. test-cases faults on Fedora, see gh-1353)
* Fixed pythonic filters and test scripts (running via wrong python version,
uses "fail2ban-python" now);
* Fixed test case "testSetupInstallRoot" for not default python version (also
using direct call, out of virtualenv);
* Fixed ambiguous wrong recognized date pattern resp. its optional parts (see gh-1512);
* FIPS compliant, use sha1 instead of md5 if it not allowed (see gh-1540)
* Monit config: scripting is not supported in path (gh-1556)
* `filter.d/apache-modsecurity.conf`
- Fixed for newer version (one space, gh-1626), optimized: non-greedy catch-all
replaced for safer match, unneeded catch-all anchoring removed, non-capturing
* `filter.d/asterisk.conf`
- Fixed to match different asterisk log prefix (source file: method:)
* `filter.d/dovecot.conf`
- Fixed failregex ignores failures through some not relevant info (gh-1623)
* `filter.d/ignorecommands/apache-fakegooglebot`
- Fixed error within apache-fakegooglebot, that will be called
with wrong python version (gh-1506)
* `filter.d/assp.conf`
- Extended failregex and test cases to handle ASSP V1 and V2 (gh-1494)
* `filter.d/postfix-sasl.conf`
- Allow for having no trailing space after 'failed:' (gh-1497)
* `filter.d/vsftpd.conf`
- Optional reason part in message after FAIL LOGIN (gh-1543)
* `filter.d/sendmail-reject.conf`
- removed mandatory double space (if dns-host available, gh-1579)
* filter.d/sshd.conf
- recognized "Failed publickey for" (gh-1477);
- optimized failregex to match all of "Failed any-method for ... from <HOST>" (gh-1479)
- eliminated possible complex injections (on user-name resp. auth-info, see gh-1479)
- optional port part after host (see gh-1533, gh-1581)
### New Features
* New Actions:
- `action.d/npf.conf` for NPF, the latest packet filter for NetBSD
* New Filters:
- `filter.d/mongodb-auth.conf` for MongoDB (document-oriented NoSQL database engine)
(gh-1586, gh-1606 and gh-1607)
### Enhancements
* DateTemplate regexp extended with the word-end boundary, additionally to
word-start boundary
* Introduces new command "fail2ban-python", as automatically created symlink to
python executable, where fail2ban currently installed (resp. its modules are located):
- allows to use the same version, fail2ban currently running, e.g. in
external scripts just via replace python with fail2ban-python:
```diff
-#!/usr/bin/env python
+#!/usr/bin/env fail2ban-python
```
- always the same pickle protocol
- the same (and also guaranteed available) fail2ban modules
- simplified stand-alone install, resp. stand-alone installation possibility
via setup (like gh-1487) is getting closer
* Several test cases rewritten using new methods assertIn, assertNotIn
* New forward compatibility method assertRaisesRegexp (normally python >= 2.7).
Methods assertIn, assertNotIn, assertRaisesRegexp, assertLogged, assertNotLogged
are test covered now
* Jail configuration extended with new syntax to pass options to the backend (see gh-1408),
examples:
- `backend = systemd[journalpath=/run/log/journal/machine-1]`
- `backend = systemd[journalfiles="/run/log/journal/machine-1/system.journal, /run/log/journal/machine-1/user.journal"]`
- `backend = systemd[journalflags=2]`
ver. 0.9.5 (2016/07/15) - old-not-obsolete
-----------
### Fixes
* `filter.d/monit.conf`
- Extended failregex with new monit "access denied" version (gh-1355)
- failregex of previous monit version merged as single expression
* `filter.d/postfix.conf`, `filter.d/postfix-sasl.conf`
- Extended failregex daemon part, matching also `postfix/smtps/smtpd`
now (gh-1391)
* Fixed a grave bug within tags substitutions because of incorrect
detection of recursion in case of multiple inline substitutions
of the same tag (affected actions: `bsd-ipfw`, etc). Now tracks
the actual list of the already substituted tags (per tag instead
of single list)
* `filter.d/common.conf`
- Unexpected extra regex-space in generic `__prefix_line` (gh-1405)
- All optional spaces normalized in `common.conf`, test covered now
- Generic `__prefix_line` extended with optional brackets for the
date ambit (gh-1421), added new parameter `__date_ambit`
* `gentoo-initd` fixed `--pidfile` bug: `--pidfile` is option of
`start-stop-daemon`, not argument of fail2ban (see gh-1434)
* `filter.d/asterisk.conf`
- Fixed security log support for PJSIP and Asterisk 13+ (gh-1456)
- Improved log support for PJSIP and Asterisk 13+ with different
callID (gh-1458)
### New Features
* New Actions:
- `action.d/firewallcmd-rich-rules` and `action.d/firewallcmd-rich-logging`
(gh-1367)
* New filters:
- slapd - ban hosts, that were failed to connect with invalid
credentials: error code 49 (gh-1478)
### Enhancements
* Extreme speedup of all sqlite database operations (gh-1436),
by using of following sqlite options: