DKIM not working correctly

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

DKIM not working correctly

Post by RvdH » 2020-02-27 10:29

Now we are on the topic of checking if hmailserver SPF check mechanism works or not, it came to my attention that email received from any outlook, hotmail, office365, exchange online account(s) (basically everything received from any obe.outbound.protection.outlook.com servers) seem to fail hmailserver internal DKIM check

Code: Select all

"DEBUG"	4300	"2020-02-27 01:15:54.366"	"DKIM: Error when retrieving public key. No key for signature."
"DEBUG"	4300	"2020-02-27 01:15:54.366"	"DKIM: Retrieval of public key failed."
I noticed published DNS record to 'selector1._domainkey.outlook.com' is a CNAME record to 'selector1._domainkey.outbound.protection.outlook.com' which holds the DKIM record (2048 bit):

Code: Select all

v=DKIM1;k=rsa;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvWyktrIL8DO/+UGvMbv7cPd/Xogpbs7pgVw8y9ldO6AAMmg8+ijENl/c7Fb1MfKM7uG3LMwAr0dVVKyM+mbkoX2k5L7lsROQr0Z9gGSpu7xrnZOa58+/pIhd2Xk/DFPpa5+TKbWodbsSZPRN8z0RY5x59jdzSclXlEyN9mEZdmOiKTsOP6A7vQxfSya9jg5
N81dfNNvP7HnWejMMsKyIMrXptxOhIBuEYH67JDe98QgX14oHvGM2Uz53if/SW8MF09rYh9sp4ZsaWLIg6T343JzlbtrsGRGCDJ9JPpxRWZimtz+Up/BlKzT6sCCrBihb/Bi3pZiEBB4Ui/vruL5RCQIDAQAB;n=2048,1452627113,1468351913
Can anyone confirm? Maybe it doesn't resolve this CNAME record? Or is it key length that is the issue here?

FYI, Spamassassin reports DKIM as being valid:

Code: Select all

DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-02-27 17:48

Oops i forgot to mention, this seems only a issue on 5.7 (64-bit
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

palinka
Senior user
Senior user
Posts: 1968
Joined: 2017-09-12 17:57

Re: DKIM not working correctly

Post by palinka » 2020-02-27 18:43


User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-02-28 10:31

I doubt it, same client mailing to a instance of hmailserver 5.6.8-x passes hmailservers internal DKIM verification without issues, others like gmail, yahoo validate do as well
The issue seems to be a combo of, hmailserver 5.7.x + outlook DKIM... most likely due to it being a CNAME i think

...bit surprised that nobody confirmed it though

I'ts no biggie, i usually don't use any of hmailservers spam checking mechanisms, i just stumbled on it while doing the SPF debug check
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
mattg
Moderator
Moderator
Posts: 20897
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: DKIM not working correctly

Post by mattg » 2020-02-28 11:38

I get

"DEBUG" 13196 "2020-02-28 17:12:27.608" "DKIM: Message passed validation."
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-02-28 12:02

Weird, i've pinpointed the location that returns this error, it located here and @ line 580 in DKIM.cpp

If i have read it correctly, at line 566 TXT records are checked, and obviously it find the specific record otherwise that error would have been displayed

Code: Select all

      if (results.size() == 0)
      {
         /*
            3.  If the query for the public key fails because the corresponding
            key record does not exist, the verifier MUST immediately return
            PERMFAIL (no key for signature).
         */

         LOG_DEBUG("DKIM: Error when retrieving public key. No key for signature.");
         return PermFail;
      }
Last edited by RvdH on 2020-02-28 12:08, edited 1 time in total.
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 3623
Joined: 2006-08-21 15:38
Location: Denmark

Re: DKIM not working correctly

Post by SorenR » 2020-02-28 12:06

RvdH wrote:
2020-02-28 12:02
Weird, i've pinpointed the location that returns this error, it located here and @ line 580 in DKIM.cpp

If i have read it correctly, at line 566 TXT records are checked, and obviously it find those otherwise that error would have been displayed

Code: Select all

      if (results.size() == 0)
      {
         /*
            3.  If the query for the public key fails because the corresponding
            key record does not exist, the verifier MUST immediately return
            PERMFAIL (no key for signature).
         */

         LOG_DEBUG("DKIM: Error when retrieving public key. No key for signature.");
         return PermFail;
      }
I believe the "LOG_DEBUG()" statement is when running the debug build. I see this statement in other places in the code but never in my logs.
SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-02-28 12:11

Nah, LOG_DEBUG() entries are displayed when debug logging is enabled, verified
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-02-28 12:40

Something fishy is going on...looks related to ipv6 (which i don't have?)

Code: Select all

C:\Users\Ruud>nslookup -type=TXT outlook.com
Server:  mijnmodem.kpn.home
Address:  fe80::36da:b7ff:fe8d:ab5e

*** mijnmodem.kpn.home can't find outlook.com: Query refused

Code: Select all

C:\Users\Ruud>nslookup -type=TXT outlook.com 8.8.8.8
Server:  dns.google
Address:  8.8.8.8

Non-authoritative answer:
outlook.com     text =

        "v=spf1 include:spf-a.outlook.com include:spf-b.outlook.com ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all"
outlook.com     text =

        "google-site-verification=0iLWhIMhXEkeWwWfFU4ursTn-_OvoOjaA0Lr7Pg1sEM"
outlook.com     text =

        "google-site-verification=DC2uC-T8kD33lINhNzfo0bNBrw-vrCXs5BPF5BXY56g"
Once i disable IPv6 in my network connection it works as expected

Code: Select all

C:\Users\Ruud>nslookup -type=TXT outlook.com
Server:  dns.google
Address:  8.8.8.8

Non-authoritative answer:
outlook.com     text =

        "v=spf1 include:spf-a.outlook.com include:spf-b.outlook.com ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all"
outlook.com     text =

        "google-site-verification=0iLWhIMhXEkeWwWfFU4ursTn-_OvoOjaA0Lr7Pg1sEM"
outlook.com     text =

        "google-site-verification=DC2uC-T8kD33lINhNzfo0bNBrw-vrCXs5BPF5BXY56g"
Think i have to disable ipv6 in my modem somewhere....i'll keep you posted
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-02-28 13:09

Well disabling IPv6 in my modem fixed the nslookup issue, but the DKIM issue persist

Off topic, anyone else has problems accessing forum from time to time?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 8644
Joined: 2011-09-08 17:48

Re: DKIM not working correctly

Post by jimimaseye » 2020-02-28 15:30

RvdH wrote:
2020-02-28 13:09
Off topic, anyone else has problems accessing forum from time to time?
It was down earlier (mysql socket error).

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 20897
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: DKIM not working correctly

Post by mattg » 2020-02-29 02:03

Just checked my today's log for the error that you show, and I did get one from a mailinglist (not a hosted office365 domain) and one from seek (the employment website), but my Office365 hosted account passed

I get a different result doing this

nslookup -type=TXT outlook.com 8.8.8.8

than doing this

nslookup -type=TXT outlook.com 10.10.10.100


Where my Bind9 DNS server sits at 10.10.10.100, however, the machine hMailserver is on is set to use 8.8.8.8 as priority DNS, with 10.10.10.100 as secondary DNS
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-03-04 13:01

mattg wrote:
2020-02-29 02:03
Just checked my today's log for the error that you show, and I did get one from a mailinglist (not a hosted office365 domain) and one from seek (the employment website), but my Office365 hosted account passed

I get a different result doing this

nslookup -type=TXT outlook.com 8.8.8.8

than doing this

nslookup -type=TXT outlook.com 10.10.10.100


Where my Bind9 DNS server sits at 10.10.10.100, however, the machine hMailserver is on is set to use 8.8.8.8 as priority DNS, with 10.10.10.100 as secondary DNS
A totally different result or just a different order of the same results? TXT Records have no priority
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
mattg
Moderator
Moderator
Posts: 20897
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: DKIM not working correctly

Post by mattg » 2020-03-05 02:09

Just the name server lists by the looks, and in a different order
I have a feeling that when I originally posted this, that I got a different answer, but I can't be sure

and Outlook has a soft fail on their spf record...

Code: Select all

C:\>nslookup -type=TXT outlook.com 8.8.8.8
Server:  dns.google
Address:  8.8.8.8

Non-authoritative answer:
outlook.com     text =

        "google-site-verification=0iLWhIMhXEkeWwWfFU4ursTn-_OvoOjaA0Lr7Pg1sEM"
outlook.com     text =

        "google-site-verification=DC2uC-T8kD33lINhNzfo0bNBrw-vrCXs5BPF5BXY56g"
outlook.com     text =

        "v=spf1 include:spf-a.outlook.com include:spf-b.outlook.com ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all"

C:\>nslookup -type=TXT outlook.com 10.10.10.100
Server:  UnKnown
Address:  10.10.10.100

Non-authoritative answer:
outlook.com     text =

        "google-site-verification=DC2uC-T8kD33lINhNzfo0bNBrw-vrCXs5BPF5BXY56g"
outlook.com     text =

        "google-site-verification=0iLWhIMhXEkeWwWfFU4ursTn-_OvoOjaA0Lr7Pg1sEM"
outlook.com     text =

        "v=spf1 include:spf-a.outlook.com include:spf-b.outlook.com ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all"

outlook.com     nameserver = ns2.msft.net
outlook.com     nameserver = ns4.msft.net
outlook.com     nameserver = nse21.o365filtering.com
outlook.com     nameserver = nse24.o365filtering.com
outlook.com     nameserver = ns3.msft.net
outlook.com     nameserver = nse12.o365filtering.com
outlook.com     nameserver = nse13.o365filtering.com
outlook.com     nameserver = ns1.msft.net
ns2.msft.net    internet address = 208.84.2.53
ns4.msft.net    internet address = 208.76.45.53
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-03-06 16:24

I got another one with same DKIM error message (mail from my bank), like the outlook one this is also a CNAME record... i really think something fishy is happening on 5.7...at least in combination with Windows Server 2016, i know some changes are made to 5.7.x DNS lookups within hmailserver

Code: Select all

2019.fjts._domainkey.ing.nl      3600	IN	CNAME	2019.fjts._domainkey.ing.net
2019.fjts._domainkey.ing.net     3600	IN	TXT	"v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCzHC5CrxbsedW5hpSzm8FkgSFfTHlX/MsCt9cuVaoaE/xp7CcaYzlKwsTu0ms3aEnzZJsWWWWNIiLMKWhelpenm5ZgGrA9GRzCLCbIho8H/M/jWoaC8OyzGKSDxNc30n3/SSJsSW+rMWViQ9+8Mg7uNS3BLEcAJj9fMyVZBAyP+wIDAQAB"
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 8644
Joined: 2011-09-08 17:48

Re: DKIM not working correctly

Post by jimimaseye » 2020-03-06 22:03

Somewhere in github is a report from me that lookups of mx servers that were in a cname were failing in 5.7 (even though cnsmes for mx's are against rfc rules) . I believe Martin 'fixed' this. Perhaps his fix is relayed to the problem you are now seeing with cname records in dkim.

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8644
Joined: 2011-09-08 17:48

Re: DKIM not working correctly

Post by jimimaseye » 2020-03-06 22:09

5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-03-07 11:54

Bingo!

And i already might have a fix,
https://github.com/hmailserver/hmailser ... r.cpp#L118

Code: Select all

   bool 
   DNSResolver::GetTXTRecords(const String &sDomain, std::vector<String> &foundResult)
   {
      DNSResolverWinApi resolver;

      std::vector<DNSRecord> foundRecords;

      bool result = resolver.Query(sDomain, DNS_TYPE_TEXT, foundRecords);

      if (foundRecords.size() == 0)
      {
         // The queries for TXT didn't return any records. Attempt to look up via CNAME
         std::vector<DNSRecord> foundCNames;
         bool cnameQueryResult = resolver.Query(sDomain, DNS_TYPE_CNAME, foundCNames);

         // A CNAME should only point at a single host name.
         if (cnameQueryResult && foundCNames.size() == 1)
         {
            auto cnameHostName = foundCNames[0].GetValue();
            return GetTXTRecords(cnameHostName, foundResult);
         }
      }

      foundResult = GetDnsRecordsValues_(foundRecords);

      return result;
   }
the added part is

Code: Select all

      if (foundRecords.size() == 0)
      {
         // The queries for TXT didn't return any records. Attempt to look up via CNAME
         std::vector<DNSRecord> foundCNames;
         bool cnameQueryResult = resolver.Query(sDomain, DNS_TYPE_CNAME, foundCNames);

         // A CNAME should only point at a single host name.
         if (cnameQueryResult && foundCNames.size() == 1)
         {
            auto cnameHostName = foundCNames[0].GetValue();
            return GetTXTRecords(cnameHostName, foundResult);
         }
      }
Same outlook address that failed earlier now reports:
"DEBUG" 5016 "2020-03-07 10:46:15.923" "DKIM: Message passed validation."
"DEBUG" 5016 "2020-03-07 10:46:15.923" "Spam test: SpamTestDKIM, Score: 0"
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 8644
Joined: 2011-09-08 17:48

Re: DKIM not working correctly

Post by jimimaseye » 2020-03-07 12:36

You're welcome. :mrgreen:
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 1085
Joined: 2008-06-27 14:42
Location: Netherlands

Re: DKIM not working correctly

Post by RvdH » 2020-03-07 19:12

This change is now included in latest artifact for 5.7.x
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

Post Reply