Grey Listing Appears Bypassed Coming From MX Backup Server

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
User avatar
jim.bus
Senior user
Senior user
Posts: 390
Joined: 2011-05-28 11:49
Location: US

Grey Listing Appears Bypassed Coming From MX Backup Server

Post by jim.bus » 2020-02-28 23:11

It seems email servers are able to bypass, at least, my Grey Listing function.

I have a Primary HMailServer on Computer A in Network A.

I have an MX Backup hMailServer on Computer B in Network B.

Email Servers send to my Primary hMailServer on Computer A in Network A. The email is Grey Listed and asked to try again later.
Email Servers then immediately try my MX hMailServer on Computer B in Network B which then accepts the email and relays it back to Primary hMailServer on Computer A in Network A.
My Primary hMailServer on Computer A in Network A accepts and delivers the Email to the Email Account on Computer A in Network A even though the sending email server has not sent the message after the Grey listing time has expired.

My MX Backup hMailServer IP Address is listed in the Incoming Relays of my Primary hMailServer on Computer A in Network A.
My MX Backup hMailServer IP Address is Whitelisted on the Whitelist tab of Greylisting on Computer A in Network A.
My MX Backup hMailServer IP Address is listed as an Incoming Relay on the MX Backup hMailServer on Computer B in Network B.
My Primary hMailServer on Computer A in Network A does not have Bypass Greylisting on SPF Pass selected.
My MX Backup hMailServer on computer B in Network B does not have Greylisting Enabled.
The IP Address of the originating Sending Email Server is the same on both my Primary hMailServer and on my MX Backup hMailServer.

Incoming Relays on my Primary hMailServe on Computer a in Network A is supposed to indicate to hMailServer that the MX Backup hMailServer IP Address on Computer B in Network B is not the originating Email Server of the message. I would expect my Primary hMailServer to look at the IP Address of the originating Email Server. I would then expect my Primary hMailServer on Computer A in Network A to issue 'Please Try Again Later' response since the Grey Listing time limit had not yet expired. Do I have something configured incorrectly?

palinka
Senior user
Senior user
Posts: 1968
Joined: 2017-09-12 17:57

Re: Grey Listing Appears Bypassed Coming From MX Backup Server

Post by palinka » 2020-02-28 23:16

jim.bus wrote:
2020-02-28 23:11
Do I have something configured incorrectly?
Yes. You have greylisting enabled. Greylisting is dead. Disable it for a worry free lifestyle.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8647
Joined: 2011-09-08 17:48

Re: Grey Listing Appears Bypassed Coming From MX Backup Server

Post by jimimaseye » 2020-02-28 23:32

Hopefully something a bit more helpful.....

The issue here is that the sending server immediately tries again (most servers well wait several minutes if configured correctly) and that it chooses to go for your backup mx record - again something that usual configurations of servers wouldn't do when being greylisted. This is something you can't control. HOWEVER try setting your highest and lowest priority Mx records to point to your primary server (A) and then have a middle one pointing to your backup (B). This way those cheeky servers (and some spam servers) that automatically try the lowest priority will find the same protected server (A) whilst normal servers will defaults to the backup (B) correctly when primary is none contactable. (This is also a good anti-spam measure too).

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jim.bus
Senior user
Senior user
Posts: 390
Joined: 2011-05-28 11:49
Location: US

Re: Grey Listing Appears Bypassed Coming From MX Backup Server

Post by jim.bus » 2020-02-29 00:27

jimimaseye wrote:
2020-02-28 23:32
Hopefully something a bit more helpful.....

The issue here is that the sending server immediately tries again (most servers well wait several minutes if configured correctly) and that it chooses to go for your backup mx record - again something that usual configurations of servers wouldn't do when being greylisted. This is something you can't control. HOWEVER try setting your highest and lowest priority Mx records to point to your primary server (A) and then have a middle one pointing to your backup (B). This way those cheeky servers (and some spam servers) that automatically try the lowest priority will find the same protected server (A) whilst normal servers will defaults to the backup (B) correctly when primary is none contactable. (This is also a good anti-spam measure too).

[Entered by mobile. Excuse my spelling.]
Thanks for your suggestion on the additional MX Record. I did understand the Sending Email Server was behaving in an unexpected way. My question though was also to question why my configuration did not account for the Sending Email Server using my MX Backup Server instead of the Primary for Greylisted emails. The configuration should have accounted for this situation because on a second attempt to send the Greylisted email the Primary Email Server could have been down and then the same situation would have happened. By adding the IP Address of MX Backup hMailSerrver on Network B to the Incoming Relays this was to tell the Primary Email Server (hMailServer) that the MX Backup hMailServer was not the originating Sending Email Server and then to look for the originating Email Server's IP Address to be able to check for Greylisting. It apparently did not do this. I was concerned that, although it appeared to be correctly configured according to the Help Documentation, it was possible that since the Primary MX Backup hMailServer was Whitelisted on the Whitelist Tab of Greylisting that possibly this caused the Primary hMailServer to not look for the Originating Sending Email Server.

I still believe, though, your suggestion to add the additional MX Record could prove to be useful as well.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8647
Joined: 2011-09-08 17:48

Re: Grey Listing Appears Bypassed Coming From MX Backup Server

Post by jimimaseye » 2020-02-29 00:44

Incoming relay will only apply to anti-spam measures which involve data that is recorded in the message data such as checking helo (with addresses stored in the received headers on the event the connecting address in a relay); the greylist function, however, relies on actual connecting ip address and will not refer to previous received headers in the message data. As the route via your backup mx means a different connecting address (which in itself is whitelisted) then the message passes through.

Quote documentation:
Please note that incoming relays does not affect grey listing. Grey listing always takes place before the Received headers have been transmitted to hMailServer. 
There you go.

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jim.bus
Senior user
Senior user
Posts: 390
Joined: 2011-05-28 11:49
Location: US

Re: Grey Listing Appears Bypassed Coming From MX Backup Server

Post by jim.bus » 2020-02-29 02:03

jimimaseye wrote:
2020-02-29 00:44
Incoming relay will only apply to anti-spam measures which involve data that is recorded in the message data such as checking helo (with addresses stored in the received headers on the event the connecting address in a relay); the greylist function, however, relies on actual connecting ip address and will not refer to previous received headers in the message data. As the route via your backup mx means a different connecting address (which in itself is whitelisted) then the message passes through.

Quote documentation:
Please note that incoming relays does not affect grey listing. Grey listing always takes place before the Received headers have been transmitted to hMailServer. 
There you go.

[Entered by mobile. Excuse my spelling.]
Thanks. I remember reading that Quote about Incoming Relays now a long time ago, but I never really made that association as to why they didn't affect Grey Listing. Now I understand. As I indicated, I thought the Whitelisting of the MX Backup was the contributing to the issue and again I now know why.

User avatar
SorenR
Senior user
Senior user
Posts: 3624
Joined: 2006-08-21 15:38
Location: Denmark

Re: Grey Listing Appears Bypassed Coming From MX Backup Server

Post by SorenR » 2020-02-29 16:24

Take it from someone who have used a BackupMX for 10+ years... I recently switched ISP so now I don't have one anymore.

BackupMX and Greylisting DO NOT MIX!

For as long as I can remember my BackupMX'es (3 IP's in a round-robin setup) has been on the Whitelist list under Greylisting.

As for the other SPAM fighting options, your BackupMX MUST be listed as incoming Relay for them to function reliably. Furthermore I did configure IP Ranges for all my BackupMX'es with a higher priority than AutoBAN, only allow SMTP (Antispam, Antivirus) and only allow deliveries from External to local e-mail addresses (I can log on remotely with SMTP or WebMail so I don't need to use outside server to send mail from own domain to own domain)

Allowing mail via BackupMX removes the possibility to BAN sending servers, so back in the day I had to come up with a fix for that.

Code: Select all

Private Const ADMIN    = "Administrator"
Private Const PASSWORD = "#SECRETPASSWORD"         ' <--- CHANGE THIS!
Private Const BACKUPMX = "backup-mx.post.tele.dk"  ' <--- CHANGE THIS! = HELO/EHLO value from BackupMX

Function Lookup(strRegEx, strMatch) : Lookup = False
    If strRegEx = "" Then Exit Function
    With CreateObject("VBScript.RegExp")
        .Pattern = strRegEx
        .Global = False
        .MultiLine = True
        .IgnoreCase = True
        If .Test(strMatch) Then Lookup = True
    End With
End Function

Function oLookup(strRegEx, strMatch, bGlobal)
    If strRegEx = "" Then strRegEx = StrReverse(strMatch)
    With CreateObject("VBScript.RegExp")
        .Pattern = strRegEx
        .Global = bGlobal
        .MultiLine = True
        .IgnoreCase = True
        Set oLookup = .Execute(strMatch)
    End With
End Function

Function ip2num(strIP)
    Dim a, i, N : N = 0
    a = Split(strIP, ".")
    For i = 0 To UBound(a)
        N = N + CLng(a(i)) * (256 ^ (3 - i))
    Next
    ip2num = N
End Function

Function isBanned(oMessage) : isBanned = False
    Dim a, strIP, strLowerIP, strUpperIP
    Dim oApp : Set oApp = CreateObject("hMailServer.Application")
    Call oApp.Authenticate(ADMIN, PASSWORD)
    strIP = ip2num(oMessage.HeaderValue("X-Envelope-IPAddress"))
    For a = 0 To oApp.Settings.SecurityRanges.Count-1
        If (oApp.Settings.SecurityRanges.Item(a).Priority = 20) Then
            strLowerIP = ip2num(oApp.Settings.SecurityRanges.Item(a).LowerIP)
            strUpperIP = ip2num(oApp.Settings.SecurityRanges.Item(a).UpperIP)
            If (strUpperIP >= strIP) And (strIP >= strLowerIP) Then
                isBanned = True
                Set oApp = Nothing
                Exit Function
            End If
        End If
    Next
    Set oApp = Nothing
End Function

Sub XEnvelope(oMessage)
    Dim i, strTo, strOriginalTo
    For i = 0 To oMessage.Recipients.Count-1
        If (i = 0) Then
            strTo = oMessage.Recipients(i).Address
            strOriginalTo = oMessage.Recipients(i).OriginalAddress
        Else
            strTo = strTo & ", " & oMessage.Recipients(i).Address
            strOriginalTo = strOriginalTo & ", " & oMessage.Recipients(i).OriginalAddress
        End If
    Next
    oMessage.HeaderValue("X-Envelope-To") = strTo
    oMessage.HeaderValue("X-Envelope-OriginalTo") = strOriginalTo
    oMessage.HeaderValue("X-Envelope-From") = oMessage.FromAddress
    oMessage.Save
End Sub

Sub getXServer(oClient, oMessage)
    Dim i, a, strIP, strRegEx, oMatch, oMatches
    If Lookup("from " & BACKUPMX, oMessage.HeaderValue("Received")) Then
        For i = 0 To oMessage.Headers.Count-1
            If (oMessage.Headers(i).Name = "Received") Then
                If Lookup("by " & BACKUPMX & " with", oMessage.Headers(i).Value) Then
                    a = Split( oMessage.Headers(i).Value, " " )
                    oMessage.HeaderValue("X-Envelope-HELO") = Trim(a(1))
                    strRegEx = "(?:\[)((?:[0-9]{1,3}\.){3}[0-9]{1,3})(?:\])"
                    Set oMatches = oLookup(strRegEx, oMessage.Headers(i).Value, False)
                    For Each oMatch In oMatches
                        If oMatch.SubMatches.Count > 0 Then
                            oMessage.HeaderValue("X-Envelope-IPAddress") = oMatch.SubMatches(0)
                        Else
                            oMessage.HeaderValue("X-Envelope-IPAddress") = ""
                        End If
                    Next
                    oMessage.Save
                    Exit For
                End If
            End If
        Next
    Else
        oMessage.HeaderValue("X-Envelope-HELO") = Trim(oClient.HELO)
        oMessage.HeaderValue("X-Envelope-IPAddress") = Trim(oClient.IPAddress)
        oMessage.Save
    End If
    Set oMatch = Nothing
    Set oMatches = Nothing
End Sub

Sub OnAcceptMessage(oClient, oMessage)
    Dim Client_IP, Client_HELO
    '
    '   Add X-Envelope... headers
    '
    Call XEnvelope(oMessage)
    '
    '   Add more X-Envelope... headers
    '
    Call getXServer(oClient, oMessage)
    Client_IP = oMessage.HeaderValue("X-Envelope-IPAddress")
    Client_HELO = oMessage.HeaderValue("X-Envelope-HELO")
    '
    '   Check for banned sender via Backup-MX ?
    '
    If (oClient.IPAddress <> oMessage.HeaderValue("X-Envelope-IPAddress")) Then
        If isBanned(oMessage) Then
            Result.Value = 2
            Result.Message = "5.3.0 [Origin Banned] The SMTP service (" & Client_HELO & ") originating on IP address (" & Client_IP & ") is not welcome here."
            Exit Sub
        End If
    End If
    '
    '
    '
End Sub
SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

User avatar
SorenR
Senior user
Senior user
Posts: 3624
Joined: 2006-08-21 15:38
Location: Denmark

Re: Grey Listing Appears Bypassed Coming From MX Backup Server

Post by SorenR » 2020-02-29 16:54

Perhaps some explanation to part of the code... How to find the original sender.

We assume BackupMX identify as "backup-mx.post.tele.dk"
The headers we are focusing on are...

Received: from backup-mx.post.tele.dk (backup-mx2.post.tele.dk [80.160.77.115]) by MX.MYDOMAIN.TLD ; Sat, 15 Jun 2019 22:17:57 +0200
Received: from sonic321-37.consmr.mail.gq1.yahoo.com (sonic321-37.consmr.mail.gq1.yahoo.com [98.137.66.100]) by backup-mx.post.tele.dk with ESMTP id 55ED474C007 for <soren@MYDOMAIN.TLD>; Sat, 15 Jun 2019 22:17:56 +0200 (CEST)

Code: Select all

First part of "Sub getXServer()"...

    '   IF the header "Received:" says "from backup-mx.post.tele.dk ....." then we are on the right track ;-)
    If Lookup("from " & BACKUPMX, oMessage.HeaderValue("Received")) Then

        '   Go through ALL headers from the start - There are multiple "Received:" headers and we can only select the first by name.
        For i = 0 To oMessage.Headers.Count-1

            '   Only process "Received:" headers
            If (oMessage.Headers(i).Name = "Received") Then
 
                '   Find the Received: header that says "by backup-mx.post.tele.dk with ....." This is the header in which to find the sender.
                If Lookup("by " & BACKUPMX & " with", oMessage.Headers(i).Value) Then

                    '   Split the text
                    a = Split( oMessage.Headers(i).Value, " " )
 
                    '   Save the sender HELO/EHLO string"
                    oMessage.HeaderValue("X-Envelope-HELO") = Trim(a(1))
 
                    '   Extract the IP address using RegEx
                    strRegEx = "(?:\[)((?:[0-9]{1,3}\.){3}[0-9]{1,3})(?:\])"
                    Set oMatches = oLookup(strRegEx, oMessage.Headers(i).Value, False)
                    For Each oMatch In oMatches
                        If oMatch.SubMatches.Count > 0 Then
                            oMessage.HeaderValue("X-Envelope-IPAddress") = oMatch.SubMatches(0)
                        Else
                            oMessage.HeaderValue("X-Envelope-IPAddress") = ""
                        End If
                    Next

                    '   Save the new headers for later use (IMPORTANT!)
                    oMessage.Save
                    Exit For
                End If
            End If
        Next
SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

Post Reply