Log inUsernamePassword
Log me on automatically each visit    
Register
Register
Log in to check your private messages
Log in to check your private messages
SMS Forum Index » Suggestions

Post new topic   Reply to topic
Sqlgrey twice random failure
View previous topic :: View next topic  
Author Message
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 162
Location: Romania

PostPosted: Mon Nov 08, 2010 8:22 pm    Post subject: Sqlgrey twice random failure Reply with quote

Hi all,

I run several mailservers, any of them with active sqlgrey service working fine. One of them only(just one), alive since October 2009 had today it's second sqlgrey service failure(first one was in June or July). That means that email messages stopped comming and the maillog showing:
Quote:
Nov 8 10:54:01 nstest postfix/smtpd[24618]: connect from unknown[77.81.253.107]
Nov 8 10:54:22 nstest postfix/smtpd[24618]: warning: connect to 127.0.0.1:2501: Connection refused
Nov 8 10:54:22 nstest postfix/smtpd[24618]: warning: problem talking to server 127.0.0.1:2501: Connection refused
Nov 8 10:54:23 nstest postfix/smtpd[24618]: warning: connect to 127.0.0.1:2501: Connection refused
Nov 8 10:54:23 nstest postfix/smtpd[24618]: warning: problem talking to server 127.0.0.1:2501: Connection refused
Nov 8 10:54:23 nstest postfix/smtpd[24618]: NOQUEUE: reject: RCPT from unknown[77.81.253.107]: 451 4.3.5 Server configuration problem; from=<info@mail.com> to=<info@mysite.com> proto=ES
MTP helo=<77.81.253.107>


Also in messages log, I found:
Quote:
messages: kernel: sqlgrey[2370]: segfault at 4 ip 77b72b94 sp 7fbc6a80 error 4 in libmysqlclient.so.15.0.0[77b36000+115000]


Googling for this errors I found something about a possible mysql issue, but nothing to clarify the problem.
In both cases, after several minutes of "mail silence" I did a
Code:
/etc/rc.d/rc.sqlgrey restart

and everything became to normal.

But because this is not a normal behaviour, I thought a Fail2Ban jail in order to prevent future failures:

In jail.conf:
Code:
[sqlgrey-restart]

enabled  = true
filter   = sqlgrey-restart
action   = sqlgrey-restart
           sendmail-sqlgrey[name=SqlGrey, dest=root@localhost]
logpath  = /var/log/maillog
maxretry = 1
findtime = 3    #I'm not very sure if needed
bantime = 1    #I'm not very sure if needed
ignoreip =


sqlgrey-restart.conf filter:
Code:
[Definition]
# to test set up use this
# /usr/bin/fail2ban-regex /var/log/maillog /etc/fail2ban/filter.d/sqlgrey-restart.conf

failregex =  <HOST>:2501: Connection refused

ignoreregex =


sqlgrey-restart.conf action:
Code:

[Definition]
actionstart =
actionstop =
actioncheck =
actionban = /etc/rc.d/rc.sqlgrey restart
actionunban =


As you can see I also adapted an allready defined action on order to send a mail whenever the restart occures.

I felt I have to share this since I was the one proposing sqlgrey package.

Cheers to "watch"-eners!
[/code]
Back to top
View user's profile Send private message
baboo
Senior Member


Joined: 04 Sep 2007
Posts: 676

PostPosted: Mon Nov 08, 2010 9:57 pm    Post subject: Reply with quote

thanks for sharing. I had this same problem awhile back in 1.5.2. Gerasimos had me do an upgrade to mysql. I'm not at my computer but if you search the forums you will find what he suggested.

and we don't hold it against you. Smile
Back to top
View user's profile Send private message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1753
Location: Greece

PostPosted: Tue Nov 09, 2010 5:39 am    Post subject: Reply with quote

Which versions of SMS you failure machines are running?
I'll add your suggestion, since we need to be safe about mail server.

Thanks
gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 162
Location: Romania

PostPosted: Tue Nov 09, 2010 7:33 am    Post subject: Reply with quote

Hello,

The machine I'm talking about runs SMS 1.4.5 with mysql 5.0.84. I use to maintain a copy with /SMS-current/ files and when I have to install a new machine, I use it.

Cheers.
Back to top
View user's profile Send private message
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 162
Location: Romania

PostPosted: Tue Nov 09, 2010 7:46 am    Post subject: Reply with quote

@ baboo,

I found your post here, but it doesen't show the way you managed to solve the problem.

Cheers!
Back to top
View user's profile Send private message
baboo
Senior Member


Joined: 04 Sep 2007
Posts: 676

PostPosted: Tue Nov 09, 2010 7:50 am    Post subject: Reply with quote

bummer! I don't remember how it was fixed. I usually write things down, so perhaps its in my notes. I will look and if I have the fix will post what I did.

regards
Back to top
View user's profile Send private message
gerasimos_h
Site Admin


Joined: 09 Aug 2007
Posts: 1753
Location: Greece

PostPosted: Tue Nov 09, 2010 8:26 am    Post subject: Reply with quote

Have you try doing a "mysql_upgrade" also check for error logs in mysql for innoDB and similar errors.
There was some issues with mysql tables in previous versions.
I'll do some testing with your fail2ban rule to add it in fail2ban package.

@baboo
probably it was on your private messages.

gerasimos_h

_________________
Superb! Mini Server Project Manager
http://sms.it-ccs.com
Back to top
View user's profile Send private message Visit poster's website
keopp
Senior Member


Joined: 08 Nov 2008
Posts: 162
Location: Romania

PostPosted: Tue Nov 09, 2010 9:55 am    Post subject: Reply with quote

... hmmm... "never change a running system" says a wellknown law Smile. I don't care for 2 fails per year if the system is running well. I am a perfectionist, but this system performs good enough for now.

In mysql error log, the last 2 records are:
Quote:
100825 16:36:32 mysqld started
100825 16:36:34 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295
100825 16:36:34 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295
100825 16:36:36 InnoDB: Started; log sequence number 0 44044
100825 16:36:36 [Note] Recovering after a crash using mysql-bin
100825 16:36:36 [Note] Starting crash recovery...
100825 16:36:36 [Note] Crash recovery finished.
100825 16:36:37 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.0.84-log' socket: '/var/run/mysql/mysql.sock' port: 0 Source distribution
101011 14:37:11 mysqld started
101011 14:37:12 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295
101011 14:37:12 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295
101011 14:37:14 InnoDB: Started; log sequence number 0 44044
101011 14:37:14 [Note] Recovering after a crash using mysql-bin
101011 14:37:14 [Note] Starting crash recovery...
101011 14:37:14 [Note] Crash recovery finished.
101011 14:37:15 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.0.84-log' socket: '/var/run/mysql/mysql.sock' port: 0 Source distribution
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    SMS Forum Index » Suggestions All times are GMT + 2 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum

SMS - Superb! Mini Server Project © 2016
Powered by phpBB © 2001, 2002 phpBB Group
iCGstation v1.0 Template By Ray © 2003, 2004 iOptional