Forum

Bounced and Other Undelivered Emails

Bounced and Other Undelivered emails

Without a doubt the most (un)popular subject on the Forum since I became a member has been that of undelivered emails. Fortunately it seems to be less of a global issue these days but still individually frequent enough to raise blood pressures from Penzance to Penrith. I know from my role as an administrator that tackling the problem is rather like a game of ‘whack-a-mole’. No sooner do you deal with one issue than another appears.

There are a number of (slightly repetitive) Beacon web pages giving information on the handling of email (click on title to open link)
Email Delivery explained
Email Delivery reports
Undeliverable emails

The most frustrating cases are those where Beacon says the email has been delivered but the recipient insists they have never had sight, sound nor smell of the message. The most likely cause of this is that the email has ended up in a spam or similar folder on the recipient’s email server*. Trying to direct a member with only basic skills to locate and examine this folder and then to ‘whitelist’ emails from Beacon can require the patience of Job (or more accurately ‘the persistence of Job’).

Alternatively it may be that the email address was valid but simply wrong, for example john.smith123@gmail.com instead of john.smith123@ymail.com . It is very easy for this sort of error to creep in between a member’s application form and the Beacon entry. In such cases the emails will simply disappear into the ether unless the hapless real owner of the address replies to say that he really doesn’t want these emails addressed to the other John Smith.

If the email address in Beacon does not exist then the email will be ‘bounced’ and reported back to Beacon as undeliverable. The Beacon mail agent then notifies the sender and the system administrator and also ‘blacklists’ the faulty email address.

A similar fate awaits emails that are detected as spam by the recipient’s email provider*. This is the source of most flurries of email delivery failures. Each email provider – gmail, btinternet, talktalk, yahoo etc – attempts to improve the service to its customers by filtering out spam. The algorithms each uses to do this are sophisticated** and continually changing in the war against the spammers. Such algorithms may include email content, reference to lists of known spam sources, email despatch patterns and other factors; the algorithms are tuned to maximise detection of real spam and minimise spurious spam detection. A revision to the algorithm may cause a given email provider to reject a batch of emails - spuriously but still causing our email agent to blacklist those addresses. Sending the email outside Beacon will often reach the recipient in these cases since it will not have all the characteristics that caused the original email to be identified as spam.

No further attempts are made to deliver emails to a blacklisted email address and no more warning emails are sent to the sender/administrator but they are reported as ‘dropped’ in the email delivery report. Once the sender or the administrator has confirmed that the email address is correct and that the recipient is willing to receive emails from the U3A then the administrator can remove the address from the blacklist. Note that this is only possible using the ‘admin’ login.

Beacon’s emails are despatched by Beacon’s email agent (currently SendGrid). The ‘reputation’ of the IP addresses from which the emails are sent are one of the factors considered by the spam filtering algorithms. The email agent protects its reputation by blacklisting addresses which report its emails as spam to prevent further potential spam emails being sent. The email agent also uses its own algorithms to detect spam in the mails it is despatching and if it even suspects a client of being used by a spammer it will cut off service to that client. This is what led to the suspension of email services via Beacon on two separate occasions in October. Be careful, be very careful, when including any links in your Beacon emails!

Finally, blocked emails reported by Beacon are those cases where the email agent has been unable to deliver the emails for reasons that may well be temporary, eg the recipient’s email box is full, or the email may be too long for that recipient or the email provider is off-line. These addresses are not blacklisted so the next Beacon email may well go through. To confuse the issue emails are often termed blocked when the sending address has been blacklisted by an organisation such as SpamHaus which maintains lists of spam sources which are used by spam filters to detect spam.

No wonder the subject leads to frustration and appeals for help from other Forum readers! However approximately 99.7% of emails sent through Beacon are successfully delivered, a good result; the remainder are a mixture of bounces, blocked by ISP, spam reports and invalid addresses. The software team are developing additional safeguards within Beacon to scan emails for potentially malicious links and content that may have been inserted in error (such as a misspelled link) before handing off to the email agent. Could this be the end of undelivered emails or is it just the start of a new round of ‘Whack-a-mole’?

Alan Swindale

U3A Support Forum Moderator.

 

* I have used the term ‘email provider’ to mean the domain hosting the email address and ‘email server’ to mean the software presenting emails to the recipient; the email server may be running at the email provider or on the recipient’s own machine.

 

** Examples of such spam filtering algorithms include “Deep Learning, Naïve Bayes, Support Vector Machines, Neural Networks, K-Nearest Neighbour, Rough sets, and Random Forests” !
(Machine Learning for email spam filtering 2019)

*** You may have noticed that at no point have I blamed an error in Beacon. Beacon is of course PERFECT!