TotalChoiceMailProblem

I finally solved a mail problem that has been bugging me for several years, so it seemed a good idea to write it up.

I have never been able to send mail from my server (fugazi.hollenback.net) to any domain hosted by totalchoicehosting.com. This has always been a problem because my good friend Scotto hosts a number of domains there, including his own (empireiondustry.com). To send mail to Scotto, I am forced to send to a junk hotmail account he checks very infrequently.

The failure is that mail sent to any of these domains is rejected with a 451 message:

<mike_at_empireindustry.com>: host empireindustry.com[216.180.241.234] said: 451
     Temporary local problem - please try later (in reply to RCPT TO command)

Which results in my mail sitting in my queue on fugazi for several days as the mail server tries every few hours to redeliver the message and fails. Finally, four days later, the mail bounces.

What the heck was going on? I thought my system was pretty well configured. I send mail to a wide range of other sites and never get bounces due to misconfiguration. I tried tweaking various settings in both my mail reader (mutt) and my mail transfer agent (qmail). No success.

Finally, a response from Total Choice Hosting to Scotto provided the clue that led to an answer. Here's what they said:

There is a problem on the remote server or in the email setup for the
account sending to you. You can see that there is no MX set for the domain
he is using at
http://www.dnsreport.com/tools/mail.ch?domain=phil_at_mail.hollenback.net.

My guess is that he has phil_at_mail.hollenback.net set as the From and or
Reply To address in the program he is using to send the email. It should
be phil_at_hollenback.net (without the mail. in front.) The server is
rejecting the message becuase the sender cannot be verified.

This analysis was not exactly correct - the problem has nothing to do with the From or Reply-To address I was setting in my mail program. Here's the full header from the test mail that failed:

From phil_at_mail.hollenback.net
Content-Description: Undelivered Message
Received: from integer.pobox.com (localhost [127.0.0.1])
        by integer.pobox.com (Postfix) with ESMTP id 533E69DA82
        for <mike_at_empireindustry.com>; Wed, 21 Jul 2004 09:22:54 -0400 (EDT)
Delivered-To: mikescotto_at_pobox.com
Received: from colander (localhost [127.0.0.1])
        by integer.pobox.com (Postfix) with ESMTP id 26D539DA58
        for <mikescotto_at_pobox.com>; Wed, 21 Jul 2004 09:22:52 -0400 (EDT)
Received: from fugazi.hollenback.net (dsl081-053-022.sfo1.dsl.speakeasy.net [64.81.53.22])
        by integer.pobox.com (Postfix) with SMTP
        for <mikescotto_at_pobox.com>; Wed, 21 Jul 2004 09:22:32 -0400 (EDT)
Received: (qmail 7992 invoked by uid 1000); 21 Jul 2004 13:22:30 -0000
Date: Wed, 21 Jul 2004 06:22:29 -0700
From: "Philip J. Hollenback" <philiph_at_pobox.com>
To: mikescotto_at_pobox.com
Subject: test message
Message-ID: <20040721132229.GA5026@hollenback.net>
Reply-To: philiph_at_pobox.com
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.4i

You can see from that header that my From: and Reply-To: don't mention phil_at_mail.hollenback.net at all. The only place that address shows up is in the envelope-from. That is the very first line of the header - the From line (not From:). This line is generated by the mail transfer agent (qmail) that my mail delivery agent (mutt) hands the message off to when I send it. Qmail is the program which actually sends the mail off my machine.

I dug around in my qmail config and found that I had set my defaultdomain to mail.hollenback.net (this is controlled by the file /var/qmail/control/defaultdomain). Thus, qmail was setting my envelope from to phil_at_mail.hollenback.net. Note also that the envelope-from is stripped out of the bounce message error report - you only see it on original mails.

This is a perfectly vaild thing to do. The specs say that the envelope-from address on an email must resolve, and mail.hollenback.net is a valid hostname. However, Total Choice Hosting decided to go a step further and try and check that the domain in the envelope-from of an incoming mail had a valid MX record in DNS. The MX record specifies what server actually receives mail for a given domain. Thus Total Choice Hosting was doing an MX lookup on mail.hollenback.net, which failed (because the MX record actually belongs to hollenback.net), so they rejected my mail.

Once I changed my defaulthost in qmail to hollenback.net, everything started working fine. I'm really not sure if what Total Choice Hosting is doing is correct or not. I assume they are checking the envelope-from to weed out spam, and that also catches servers (such as mine) that have slightly unusual (but perfectly valid) mail server configurations.

I do have to chastise them for the way their mail server is set up. Instead of sending that 451 message with a non-descriptive error message, their mail server should reply "refusing mail because your envelope-from domain doesn't have an MX record" or something similar. Also, it probably should be a 5xx (permanent) error, which would cause my mail server to give up delivery immediately instead of a 4xx (temporary) error, which causes my mail server to keep retrying the delivery.

I sent my analysis to Total Choice Hosting, but never heard back. Well, I fixed the problem on my end, anyway.