duplicate emails

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
VRSpock
New user
New user
Posts: 10
Joined: 2009-07-15 05:14
Location: Smithville TN
Contact:

duplicate emails

Post by VRSpock » 2009-07-26 03:56

I've been using te hMailServer for a few weeks now and I must say that I am quite impressed by what it offers for a free mail server. Everything is working great, but I have one minor issue.

With some emails that I get, the mail server gets hit multiple times for the same message. It seems to be just with mailing lists that I am subscribed to and has happened with various mailing lists.

For example I must have received about 20 to 30 copies of one particular email. At first I thought that maybe I was just subscribe to the lists multiple times, so I began unsubscribing from those lists, but it is happening to too many different lists to be that.

I also temporarily disabled all scripts and rules to see if there was something strange going on with those, but that had no effect. Looking in the logs, the emails in question appear to have been delivered without a problem, so I'm scratching my head on this one.

There was a similar issue with the previous mail server we had at one point and it turned out to be that the mail server was not giving the correct SMTP response code to the sending mail server, making the sending server believe that the mail delivery did not complete, prompting for a redelivery attempt. I'm starting to wonder if that is the issue here as I'm not having the problem with all mailing lists, only certain ones. and even then, it is not consistent.

Any ideas?
Fred Dickey (aka VRSpock)
Kustom Printing Products, Inc. - http://www.kustomproducts.net

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: duplicate emails

Post by ^DooM^ » 2009-07-26 10:43

Without seeing some SMTP logs it is hard to guess what is going on. Do you still have a few of the duplicate emails?
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

VRSpock
New user
New user
Posts: 10
Joined: 2009-07-15 05:14
Location: Smithville TN
Contact:

Re: duplicate emails

Post by VRSpock » 2009-07-27 00:01

Below is a couple of log entries from yesterday for one email that I am still receiving duplicates of today. One is at 12:06AM and the other from 12:46AM for the same email message where the sending server keeps re-sending as if it thinks the message was never delivered successfully.

I just noticed that the time it took between the initial connection and the actual acknowledgement of receipt is pretty long. Both times I received the message successfully. Could it be that the sending server is timing out before the successful receipt notice is sent back? If so, would there not be an entry in the log about the connection closing prematurely? Other messages in the log have about a 9 second or so session time.

"SMTPD" 3228 9207 "2009-07-25 00:06:26.390" "207.97.240.173" "SENT: 220 Welcome to hope-fm mail server"
"SMTPD" 3228 9207 "2009-07-25 00:06:26.437" "207.97.240.173" "RECEIVED: EHLO mcsv9.net"
"SMTPD" 3228 9207 "2009-07-25 00:06:26.437" "207.97.240.173" "SENT: 250-mail.thinkhope.com[nl]250-SIZE 20480000[nl]250 AUTH LOGIN"
"SMTPD" 3228 9207 "2009-07-25 00:06:26.484" "207.97.240.173" "RECEIVED: MAIL FROM:<bounce-122945-169061-fdickey=thinkhope.com@mcsv9.net>"
"SMTPD" 3228 9207 "2009-07-25 00:06:45.264" "207.97.240.173" "SENT: 250 OK"
"SMTPD" 3228 9207 "2009-07-25 00:06:45.311" "207.97.240.173" "RECEIVED: RCPT TO:<fdickey@thinkhope.com>"
"SMTPD" 3228 9207 "2009-07-25 00:06:45.327" "207.97.240.173" "SENT: 250 OK"
"SMTPD" 3228 9207 "2009-07-25 00:06:45.374" "207.97.240.173" "RECEIVED: DATA"
"SMTPD" 3228 9207 "2009-07-25 00:06:45.374" "207.97.240.173" "SENT: 354 OK, send."
"APPLICATION" 3256 "2009-07-25 00:18:35.632" "SMTPDeliverer - Message 1676: Delivering message from bounce-122945-169061-fdickey=thinkhope.com@mcsv9.net to fdickey@thinkhope.com. File: C:\Program Files\hMailServer\Data\{CCC9A634-E1A0-4A63-9AF5-304AB5056BFD}.eml"
"SMTPD" 3200 9207 "2009-07-25 00:18:35.632" "207.97.240.173" "SENT: 250 Queued (710.250 seconds)"

"SMTPD" 3228 9236 "2009-07-25 00:46:45.969" "207.97.240.173" "SENT: 220 Welcome to hope-fm mail server"
"SMTPD" 3228 9236 "2009-07-25 00:46:46.016" "207.97.240.173" "RECEIVED: EHLO mcsv9.net"
"SMTPD" 3228 9236 "2009-07-25 00:46:46.031" "207.97.240.173" "SENT: 250-mail.thinkhope.com[nl]250-SIZE 20480000[nl]250 AUTH LOGIN"
"SMTPD" 3228 9236 "2009-07-25 00:46:46.078" "207.97.240.173" "RECEIVED: MAIL FROM:<bounce-122945-169061-fdickey=thinkhope.com@mcsv9.net>"
"SMTPD" 3228 9236 "2009-07-25 00:47:03.406" "207.97.240.173" "SENT: 250 OK"
"SMTPD" 3228 9236 "2009-07-25 00:47:03.453" "207.97.240.173" "RECEIVED: RCPT TO:<fdickey@thinkhope.com>"
"SMTPD" 3228 9236 "2009-07-25 00:47:03.484" "207.97.240.173" "SENT: 250 OK"
"SMTPD" 3228 9236 "2009-07-25 00:47:03.531" "207.97.240.173" "RECEIVED: DATA"
"SMTPD" 3228 9236 "2009-07-25 00:47:03.531" "207.97.240.173" "SENT: 354 OK, send."
"APPLICATION" 3256 "2009-07-25 00:58:48.789" "SMTPDeliverer - Message 1684: Delivering message from bounce-122945-169061-fdickey=thinkhope.com@mcsv9.net to fdickey@thinkhope.com. File: C:\Program Files\hMailServer\Data\{33170AA2-5445-41E2-8E2A-CCA350D2A5B6}.eml"
"SMTPD" 3200 9236 "2009-07-25 00:58:48.805" "207.97.240.173" "SENT: 250 Queued (705.266 seconds)"
Fred Dickey (aka VRSpock)
Kustom Printing Products, Inc. - http://www.kustomproducts.net

VRSpock
New user
New user
Posts: 10
Joined: 2009-07-15 05:14
Location: Smithville TN
Contact:

Re: duplicate emails

Post by VRSpock » 2009-07-27 00:09

One thing I did forget to do when installing...my resident anti-virus on the server did not have the hmailserver folder set to exclude. I just enabled that to see if the resident antivirus may be the culprit in this.
Fred Dickey (aka VRSpock)
Kustom Printing Products, Inc. - http://www.kustomproducts.net

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: duplicate emails

Post by ^DooM^ » 2009-07-27 00:34

I definitely think this is your AV or Firewall or Defender or whatever interfering with the SMTP commands. I would disable AV completely for a while and see if that delay stops (Remember just because you close an AV app it doesn't mean it's service is not still running so check the services too :) )
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

VRSpock
New user
New user
Posts: 10
Joined: 2009-07-15 05:14
Location: Smithville TN
Contact:

Re: duplicate emails

Post by VRSpock » 2009-07-27 01:21

Aha! Windows Defender was also running on the server. I was still getting duplicates after disabling Symantec, but hadn't thought about Defender.

It was in the middle of receiving one of the duplicate messages (was watching from the real-time log screen) and as soon as I killed Defender it received the message and sent the completed delivery notification. I'll keep monitoring and see if that resolves the issue.
Fred Dickey (aka VRSpock)
Kustom Printing Products, Inc. - http://www.kustomproducts.net

VRSpock
New user
New user
Posts: 10
Joined: 2009-07-15 05:14
Location: Smithville TN
Contact:

Re: duplicate emails

Post by VRSpock » 2009-07-27 05:18

Ok...after disabling the antivirus and window defender services, I was still experiencing the duplicates. So, I went back to going through all of the hMailServer settings and scratching my head over this some more.

Finally, I decided to go back through and systematically disable all of the DNSbl and URLbl lists (I had added several DNSbls and one URLbl from our previous mail server/antispyware setup) just to double check and make sure the culprit of this issue was not one of them.

The first of the mailing lists that have been resending to me multiple times hit the server shortly thereafter and appears to have completed with the following lines of log that was missing from these duplicates log entries before:

SMTPD" 1960 46 "2009-07-26 21:59:40.915" "72.15.222.66" "RECEIVED: QUIT"
"SMTPD" 1960 46 "2009-07-26 21:59:40.915" "72.15.222.66" "SENT: 221 goodbye"

So apparently, one of my black lists is taking forever to reply back and causing everything to timeout. I suspect that it is the URLbl one since the most common denominator of these duplicates is that they all contain a slew of URLs in them common to news letters.
Fred Dickey (aka VRSpock)
Kustom Printing Products, Inc. - http://www.kustomproducts.net

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: duplicate emails

Post by ^DooM^ » 2009-07-27 10:32

Ok did disabling the RBL's stop the duplicates?

Either way this seems like the sending mailservers problem rather than a hMail issue wouldn't you agree?
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

VRSpock
New user
New user
Posts: 10
Joined: 2009-07-15 05:14
Location: Smithville TN
Contact:

Re: duplicate emails

Post by VRSpock » 2009-07-28 03:27

ok...it did for one of the mailing lists....but I'm also on the webpronews mailing list and got a bunch of duplicates from them today.

I'm going to white list the webpronews mailing list and see if that changes anything.

I'm not sure if it is an issue on their end or not. It appears that way, as this is not happening to every single email that comes through and is only occuring to a very small percentage of the emails I receive. However, the long session time between the beginning of the sending of data and when the mail transaction completes is an anomoly that I'm going to continue to investigate. I'll post again if I've determined that anything in my settings, server config, etc is playing a role in this.

On a positive (and off topic) note, I've discovered myroundcube and am now even more grateful for migrating to hMailServer.
Fred Dickey (aka VRSpock)
Kustom Printing Products, Inc. - http://www.kustomproducts.net

Post Reply