Log inUsernamePassword
Log me on automatically each visit    
Register
Register
Log in to check your private messages
Log in to check your private messages
SMS Forum Index » SMS User Support

Post new topic   Reply to topic
system time and imap Goto page Previous  1, 2
View previous topic :: View next topic  
Author Message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Wed Mar 02, 2011 10:34 pm    Post subject: Reply with quote

OK! I also add lines for pam and ldap in your dovecot-attacks.conf

Code:
failregex = \(auth failed.*rip=<HOST>
            \[<HOST>\].*authentication failed
            warning: <HOST>.*verification failed
            lost connection after.*\[<HOST>\]
            auth\(default\): pam\(.*,<HOST>\): pam_authenticate\(\) failed:
            auth\(default\): Info: ldap\(.*,<HOST>\): unknown user
            auth\(default\): Info: ldap\(.*,<HOST>\): Password mismatch


Thanks for the suggestions

gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 166
Location: Romania

PostPosted: Wed Mar 02, 2011 10:51 pm    Post subject: Reply with quote

Ok then.
I'll post every new rule.

By the the way, do you think that shorter rules can increase performance?
Back to top
View user's profile Send private message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Thu Mar 03, 2011 8:11 am    Post subject: Reply with quote

Either creating a new rule or searching more queries on one rule I believe it's the same.
I haven't test findtime though since searching more queries might increase
find time.
But in some cases needs to divide them for instance

\[<HOST>\].*authentication failed

is found at /var/log/maillog but

\(auth failed.*rip=<HOST>
auth\(default\): Info: ldap\(.*,<HOST>\): unknown user
auth\(default\): Info: ldap\(.*,<HOST>\): Password mismatch

is found at /var/log/dovecot-info.log

Another way is to set
logpath =
so dovecot logs to /var/log/maillog
although some info is logged in /var/log/maillog either way.

gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 166
Location: Romania

PostPosted: Thu Mar 03, 2011 8:46 am    Post subject: Reply with quote

Yes, you are right about logging everything in maiilog. This is my option and I log also clamd/freshclam and mailscanner logs in maillog. This way is easyer for me to follow a message having all it's route in one file.
Of course considerations can be made here about the resulting maillog size, especially when there are a big number of clients involved(or rised verbosity).
Also is better for fail2ban having less log files to watch over.

As far as I know, each line in <failregex> is tested against the logfile and whenever a match is met, it triggers the action. In my question about performance I meant if you think that lets say:
Code:

ldap\(.*,<HOST>\): unknown user


(shorter rule)is easier to be matched than
Code:

auth\(default\): Info: ldap\(.*,<HOST>\): unknown user

or even a rule like this
Code:

(?: pop3-login|imap-login): .*(?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
Back to top
View user's profile Send private message
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 166
Location: Romania

PostPosted: Thu Mar 03, 2011 9:31 am    Post subject: Reply with quote

I found a very interesting long thread here.
The essence of this is below:

Code:

failregex = : warning: [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed
            (?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P<host>\S*),.*


Beware at linefeed, that there are only 2 lines of code.
Back to top
View user's profile Send private message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Thu Mar 03, 2011 9:37 am    Post subject: Reply with quote

MailScanner logs by default in /var/log/maillog and can cause quite a problem if something wrong happens and could take over you space quite fast.

Well as for shorter rule, theoretically, the more accurate the rule is, the faster will respond.
But in action, shorting the rule it's better as long as it's still accurate and output the same results.

So
Code:
ldap\(.*,<HOST>\): unknown user

real    0m1.337s
user    0m1.331s
sys     0m0.006s

It's faster than
Code:
auth\(default\): Info: ldap\(.*,<HOST>\): unknown user

real    0m1.344s
user    0m1.336s
sys     0m0.009s

And this isn't accurate enough
Code:
ldap\(.*,<HOST>\).*

real    0m1.729s
user    0m1.607s
sys     0m0.027s


gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 166
Location: Romania

PostPosted: Thu Mar 03, 2011 9:41 am    Post subject: Reply with quote

Thanks very much. It confirms my assumptions. How could you measure the responding time. THIS I find very interesting..... Smile
Back to top
View user's profile Send private message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Thu Mar 03, 2011 9:53 am    Post subject: Reply with quote

I measure with "time"
Code:
time fail2ban-regex [log] [filter]

And for sure the less queries a filter has, it's faster, and the 2 line rule above seems great.

gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Thu Mar 03, 2011 7:28 pm    Post subject: Reply with quote

Well after doing some test I found that, splitting logs is faster, because when you are searching /var/log/maillog takes twice the time since it's almost double in size too, comparing to dovecot-info.log Smile

So in conclusions we need shorter rules with same accuracy, and having to look at different logs it's not a bad thing after all, we just have to load more filters at start.

gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Thu Mar 03, 2011 10:45 pm    Post subject: Reply with quote

After testing and testing I ended up with the following rules.
dovecot-imap.conf looking at /var/log/dovecot-info.log
Code:
failregex = (?: pop3-login|imap-login):.* (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
            (?: pop3-login|imap-login):.* (?:Internal login failure \(auth failed).*rip=(?P<host>\S*),.*
            auth\(default\):.* (?:ldap|pam|shadow|sql)\(.*,<HOST>\): (?:unknown user|Password mismatch)


dovecot-maillog.conf looking at /var/log/maillog
Code:
failregex = [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed
            <HOST>.* authentication failed
            <HOST>.* verification failed

Any suggestions are welcome.

gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 166
Location: Romania

PostPosted: Fri Mar 04, 2011 10:26 am    Post subject: Reply with quote

There are indeed some comments I belive there have to be made.
First of all, the purpose of those rules are to prevent two type of events: brute-force attacks and some kind of ddos attacks(even rejected by rbl checks or sqlgrey). To achieve this, it is enough to have a recurring(let me call it) <marker> and a related <HOST> to be banned a proper amount of time. And of course we want this to be done with minimum effort from "The Machine". Considering all above, I really appreciate the actual rules, very smart to match nearly an entire log line, but I belive they are too accurate for our needs.
So, I belive that:

Code:

(?: pop3-login|imap-login):.* (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
            (?: pop3-login|imap-login):.* (?:Internal login failure \(auth failed).*rip=(?P<host>\S*),.*


can be "summarized" in:

Code:

(?:Authentication failure|\(auth failed).*rip=<HOST>


the rule:
Code:

auth\(default\):.* (?:ldap|pam|shadow|sql)\(.*,<HOST>\): (?:unknown user|Password mismatch)

can be shortened with
Code:

<HOST>\): (?:unknown user|Password mismatch)


this one:
Code:

[-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed


is shorter(faster) like this:
Code:

\[<HOST>\].*authentication failed


I didn't found yet a log line to match this:
Code:

<HOST>.* authentication failed


I mean somenting like 123.456.789.000.*authentication failed(no brackets of any kind), but if it is, it can be easily included in the above.

I don't reccomend anymore:
Code:

<HOST>.* verification failed


because legitimate HOST's can fail postfix(fqdn) verification but can still be legitimate to send emails.

I further support this:
Code:

lost connection after.*\[<HOST>\]


especially against ddos attacks, even than most of them are rejected by rbl/dnsbl checks(for who use them). It's better to ban him 3 hours than to have him knoking on my door every couple of minutes.

My hope in all above is that I followed the slackware rules of simplicity so please, anyone finding I'm mistaking, correct me. I probably post corrections myself in the near future, but you now how such things goes....

Cheers!
Back to top
View user's profile Send private message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Fri Mar 04, 2011 6:20 pm    Post subject: Reply with quote

Shorting too match can cause high loads, for instance
Code:
auth\(default\):.* (?:ldap|pam|shadow|sql)\(.*,<HOST>\): (?:unknown user|Password mismatch)

real    0m1.378s
user    0m1.370s
sys     0m0.008s


Code:
<HOST>\): (?:unknown user|Password mismatch)

real    0m1.938s
user    0m1.924s
sys     0m0.013s


In other situations shorting can output false alarms for instance
Code:
(?:Authentication failure|\(auth failed).*rip=<HOST>

take almost half the time compare to
Code:
(?: pop3-login|imap-login):.* (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
            (?: pop3-login|imap-login):.* (?:Internal login failure \(auth failed).*rip=(?P<host>\S*),.*

but you will ban every user that he's inactive
Code:
Disconnected: Inactivity (auth failed


As I said, after testing and testing I ended up with those rules, which certainly aren't perfect.
I appreciate your effort and contribution and I'm always opened in suggestions.

gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 166
Location: Romania

PostPosted: Fri Mar 04, 2011 6:53 pm    Post subject: Reply with quote

I totally agree with you and it will be allways a pleasure to post here.
Back to top
View user's profile Send private message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1757
Location: Greece

PostPosted: Fri Mar 04, 2011 8:23 pm    Post subject: Reply with quote

A little update, this long rule covers the loss of time (takes half the time) and it's accurate enough
Code:
(?: pop3-login|imap-login):.* (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed|Internal login failure \(auth failed).*rip=(?P<host>\S*),.*


gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    SMS Forum Index » SMS User Support All times are GMT + 2 Hours
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum

SMS - Superb! Mini Server Project © 2016
Powered by phpBB © 2001, 2002 phpBB Group
iCGstation v1.0 Template By Ray © 2003, 2004 iOptional