Transfer email from one hmailserver to another

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
panagouras
New user
New user
Posts: 6
Joined: 2020-05-18 15:54

Transfer email from one hmailserver to another

Post by panagouras » 2020-05-18 16:13

Hi All,
I have been using hmailserver for a while now and made many tests with this great Mail Server. I wanted to try the following and wanted you input before I start.

I want to create a Cloud server running hmail that has MX records pointing. Then create a 2nd hmail server in a remote place with limited bandwidth connectivity. Both servers need to have the same domain e.g. mailtests.com. incoming and outgoing emails should always go through Cloud hmailserver,

I wanted to transfer emails between these servers using eml files, in order to be able to, compress, transfer and import the .eml files to each server.
Initially I thought of using Data Directory Sync, but instead of scan a specific folder and add the eml messages it will go through the whole database which is not something that could run every 5 minutes!

I know I have to proceed with scripting but I am not sure what the best way is.
1. If I somehow export the eml file from the remoteServer and copy it to Cloudhmailserver how can I tell the hmailserver to pickup the file and generate an email from the .eml file? What function should I use?
2. Is it possible to disable to delete the message and only run the script? (already tried to create a rule but when I choose delete message it also sends it via SMTP).
3. What other option is there to import an .eml from a specific folder to a specific account using script?

thank you for reading, any suggestions are welcome. :D :D
Panagouras

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Transfer email from one hmailserver to another

Post by palinka » 2020-05-18 16:49

What's the purpose of the private server? Just a realtime backup?

panagouras
New user
New user
Posts: 6
Joined: 2020-05-18 15:54

Re: Transfer email from one hmailserver to another

Post by panagouras » 2020-05-18 18:37

No, I want to make it a production private Mail server, all emails should go through the CloudServerHmailserver

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Transfer email from one hmailserver to another

Post by palinka » 2020-05-18 19:50

No one can serve two masters

And no two masters can serve email.

You need to pick one or the other for your domain.

If you just want to mirror the two servers, you can set them both up identically and use imapcopy or something like that to copy messages between them.

User avatar
jimimaseye
Moderator
Moderator
Posts: 10060
Joined: 2011-09-08 17:48

Re: Transfer email from one hmailserver to another

Post by jimimaseye » 2020-05-18 20:04

The local/private mail server could POP3 (External Download) the Inbound emails that have been received by the Cloud server. This will only server to be a mirror of inbound mails (ie, the INBOX) at time of receipt. Any emails then moved/deleted or sent through either servers will not be reflected to the other.

If the private server has, say, private users (eg, geoff_home@mailtests.com) whilst the cloud server hosts business accounts (eg, geoff_work@mailtests.com) then this is perfectly doable using routes. All emails will come to the cloud server and will be routed through to the private server accordingly. And when sending, all emails that are sent via the private server will be routed to the cloud server for final delivery.

Once again, though, I reiterate that the SENT folders will reside on the individual servers and will not be mirrored.

As Palinka says, you can only have one primary server behind the MX records.
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

panagouras
New user
New user
Posts: 6
Joined: 2020-05-18 15:54

Re: Transfer email from one hmailserver to another

Post by panagouras » 2020-05-19 09:17

Thank you for your replies.

@jimimaseye My initial thought was POP3 Accounts and connectors as you describe but as I mentioned the Bandwidth is very limited and there might be many disconnections since we are using SAT for the Private Server. I need to be able to make the transfer between them using a Protocol that can resume the transfer from where it left. That is why I was thinking of .eml files and then using scp and rsync to transfer them between the servers.

@palinka I understand that if I was using SMTP protocol this could only work with routes and emails that reside only in one of the two servers. I could easily also keep a backup in a secondary mailbox in the cloud server using rules before being delivered to the Private site which would solve my problem.

My main goal is to manage and sync those two servers using other ways from POP3 and SMTP that do not compress and do not resume transfer from where it left.

thank you for your time. ;)

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Transfer email from one hmailserver to another

Post by palinka » 2020-05-19 11:59

Which server would you point your mail client to for sending and receiving mail?

panagouras
New user
New user
Posts: 6
Joined: 2020-05-18 15:54

Re: Transfer email from one hmailserver to another

Post by panagouras » 2020-05-19 12:26

Clients will only be inside the Private site! and they will connect using IMAP from the local network.
I was thinking of only keep a backup of the mails with rules in a backup mailbox for each user in the cloud site.

BR,
Panagouras

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Transfer email from one hmailserver to another

Post by palinka » 2020-05-19 14:42

Is this for a ship at sea? I think it's better that the main server is connected to the stable land based internet, and clients at sea simply use that server. Forget about the private server altogether. Is there a reason it shouldn't be done this way? Because its the most stable way to do it.

Get ye crew to use a mail client that doesn't automatically download images and attachments. :wink:

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Transfer email from one hmailserver to another

Post by SorenR » 2020-05-19 18:52

palinka wrote:
2020-05-19 14:42
Is this for a ship at sea? I think it's better that the main server is connected to the stable land based internet, and clients at sea simply use that server. Forget about the private server altogether. Is there a reason it shouldn't be done this way? Because its the most stable way to do it.

Get ye crew to use a mail client that doesn't automatically download images and attachments. :wink:
IMAP4 upload the email 2 times when sending - first to send to other host and second to put in sent mail folder.

Why not have a landbased server and a local server with POP3 fetch and SMTP relay? It is very likely the SMTP relay can be scripted to store a copy of outgoing mail in the proper account folder.
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

panagouras
New user
New user
Posts: 6
Joined: 2020-05-18 15:54

Re: Transfer email from one hmailserver to another

Post by panagouras » 2020-05-21 08:45

SorenR wrote:
2020-05-19 18:52
palinka wrote:
2020-05-19 14:42
Is this for a ship at sea? I think it's better that the main server is connected to the stable land based internet, and clients at sea simply use that server. Forget about the private server altogether. Is there a reason it shouldn't be done this way? Because its the most stable way to do it.

Get ye crew to use a mail client that doesn't automatically download images and attachments. :wink:
IMAP4 upload the email 2 times when sending - first to send to other host and second to put in sent mail folder.

Why not have a landbased server and a local server with POP3 fetch and SMTP relay? It is very likely the SMTP relay can be scripted to store a copy of outgoing mail in the proper account folder.
Good Morning!
@palinka Yes this is for a ship!
@SorenR Having mailboxes at main server and then using POP3 connectors to retrieve the emails was my initial thought. BUT, (correct me if I am wrong) POP3 doesn't compress and doesn;t continue form where it stoped. Meaning in case there are two emails 4mb each in the mailbox, and it keeps having disconnections it will never retrieve those emails and it will keep trying / downloading half messages.

That is the reason that I wanted to
Incoming mails: take the .eml from the main site zip it,transfer it with scp or rsync (supports resume) to the other side and then extract and import .eml file using scripts to the hmail server.
Outgoing mails: find the .eml from the SQL database locate in DATA folder -> zip -> transfer with scp or rsync -> import in some way directly to outbox folder of hmail server????

regarding the first step I am really close and have found various ways to import the .eml to hserver.
The big problem is when the ship wants to send an email! how can i import the .eml file to hmail and then hmail sent the email?

thanks guys for helping
Panagouras

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Transfer email from one hmailserver to another

Post by SorenR » 2020-05-21 13:54

panagouras wrote:
2020-05-21 08:45
@SorenR Having mailboxes at main server and then using POP3 connectors to retrieve the emails was my initial thought. BUT, (correct me if I am wrong) POP3 doesn't compress and doesn;t continue form where it stoped. Meaning in case there are two emails 4mb each in the mailbox, and it keeps having disconnections it will never retrieve those emails and it will keep trying / downloading half messages.

That is the reason that I wanted to
Incoming mails: take the .eml from the main site zip it,transfer it with scp or rsync (supports resume) to the other side and then extract and import .eml file using scripts to the hmail server.
Outgoing mails: find the .eml from the SQL database locate in DATA folder -> zip -> transfer with scp or rsync -> import in some way directly to outbox folder of hmail server????

regarding the first step I am really close and have found various ways to import the .eml to hserver.
The big problem is when the ship wants to send an email! how can i import the .eml file to hmail and then hmail sent the email?

thanks guys for helping
Panagouras
Hmm... On the sending side you can actually script hMailServer to drop the message (oMessage.Filename) into a directory and run 7Zip on the file.

On the receiving side picking up the compressed file and unpacking it before feeding it to hMailServer is possible. I have previously "cloned" emails so not entirely impossible. :mrgreen:

I have to get my thinking hat...
inventor.jpg
inventor.jpg (10.06 KiB) Viewed 5482 times
When modyfying this code please remember that "oMessage.Filename" should be the the filename of the file in the pick-up directory.
The magic occurs here:

Code: Select all

        strFilename = .Filename
        With CreateObject("Scripting.FileSystemObject")
            .CopyFile oMessage.Filename, strFilename, True
        End With
        .RefreshContent
oMessage.Filename (the current message) is copied into file "strFilename" which is the filename of the new message we created.
.RefreshContent simply refreshes the content of the new message and voila - it's now 100% identical to the original message.

You need to extract the intended recipient (from the To: header) and add that to the message to make hMailServer deliver the message.
I was thinking that perhaps adding a few headers to the message BEFORE dropping it into the "delivery folder" could help delivering it at the other end since any mails sent as BCC may impose challenges otherwise later.

"strHeader" is a temporary header I use to verify that messagecloning only occur once per message 8)

Code: Select all

Function CloneMail(oMessage, strRecipient, strAddress, strHeader)
    '
    '   CloneMail(oMessage, "Wile E. Coyote", "wile.e.coyote@acme.inc", "X-hMailServer-Cloned")
    '
    Dim strFilename
    If oMessage.HeaderValue(strHeader) = "YES" Then Exit Function
    With CreateObject("hMailServer.Message")
        strFilename = .Filename
        With CreateObject("Scripting.FileSystemObject")
            .CopyFile oMessage.Filename, strFilename, True
        End With
        .RefreshContent
        .AddRecipient strRecipient, strAddress
        .HeaderValue(strHeader) = "YES"
        .Save
    End With
End Function
As always - no guarantees what so ever.
mad-scientist-lab-conducting-experiment-600w-464558135.jpg
mad-scientist-lab-conducting-experiment-600w-464558135.jpg (17.1 KiB) Viewed 5482 times
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

palinka
Senior user
Senior user
Posts: 4461
Joined: 2017-09-12 17:57

Re: Transfer email from one hmailserver to another

Post by palinka » 2020-05-21 14:02

panagouras wrote:
2020-05-21 08:45
The big problem is when the ship wants to send an email! how can i import the .eml file to hmail and then hmail sent the email?
I still think it's more efficient to have only land based server. If the concern is poor connection due to limited bandwidth satellite:

* Land based server never has that problem
* Sea based server always has that problem
* Servers give up after fixed number of tries
* Clients never give up - they'll keep trying until they get satisfaction
* Clients can drop connection and retry at will, or manually if the user is impatient
* Distributing bandwidth among the clients makes more sense, and there's no jumping through hoops to solve a problem that's already solved
* All messages are safely stored on land at "reliable" server

To me it makes sense. But sometimes I think different, so there could be reasons against I just overlook. :D But from a reliability / anti-fragile perspective, land based server only makes the most sense to me.

User avatar
SorenR
Senior user
Senior user
Posts: 6315
Joined: 2006-08-21 15:38
Location: Denmark

Re: Transfer email from one hmailserver to another

Post by SorenR » 2020-05-21 14:07

I'd like to give it a go this weekend on building a solution, I just don't have the time today - and I also need to service my test (180 day trial) 2016 Server in order to have a two-server setup.
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

panagouras
New user
New user
Posts: 6
Joined: 2020-05-18 15:54

Re: Transfer email from one hmailserver to another

Post by panagouras » 2020-05-22 16:03

Hi All,

I suppose the first code should run every time a message arrives! So, it would create a .eml file.
What the 2nd code exactly does? Is it reading the .eml file and creates an Object (message) inside hmail server? Will it send by reading the Headers?
SorenR Thank you for your efforts

@palinka By keeping a 2nd hmail server instance on the remote side, you make it bullet proof to broken pst files, malware and user faults on client computers. Also, the main ingredient that makes the difference is the compression that POP3 and SMTP do not offer.!!!

guys thank you again for your help!
Panagouras

Post Reply