The Outbound Index is about providing a service useful to businesses in eliminating the problems caused by spam. It is not a search for a theoretical utopian solution to email spam - that is the purvey of fine discussion groups such as ASRG, IETF standards groups and others - who are doing a much needed service which will benefit the global community in the future.

I come from the "practical applications of technology to a business problem" perspective. My primary goals are:

 - Make sure all revenue generating emails arrive to the business and get seen

 - Employees waste very little time each day on emails such as spam, viruses, scams - energy or actions which may harm or at least do nothing positive for the business

The Outbound Index was conceived and designed to function on a global scale. It has been in use for several years in the service of business customers of a small hosting company (a few thousand users.) Beyond this fairly local use, developer/users from Malta, France and Romania have successfully queried the Outbound Index millions of times.

We have long term experience using the Outbound Index with real business customers - as the only decision factor in tagging "suspicioius" mail. Customers tell us they "love it" - with the user interface of MailCruncher.com. So from our viewpoint: Does it solve the business problem? YES.

Can the Outbound Index, and should it, be expanded to help more ISPs, more businesses and more end user recipients? That is the topic this article explores.

( Screenshot showing results of queries to the Outbound Index )

As you consider whether or not the Outbound Index is valid or has value, I'm imagining the thought process or objective reasoning, falling mainly into three categories:

1. Central authorities are a bad thing - they represent a point of failure, a place where ethical corruption potential means trust cannot be placed in them.

I would agree in the sense that one should stay very well informed about who owns and who directs any central authority one trusts. I hope you know who is behind the root DNS servers you depend on, and every DNS server you depend on, because they control very close to everything. You should also check up on who controls the internet backbone data flows as well as who can turn your electricity on and off.

Look over those behind the Outbound Index carefully. There are major technology corporations who we would not accept as partners or investors. We see that this role requires a precise vision of service to recipients, not to service to senders or highest bidders.

Senders could self-declare what servers they authorize, and some do by using SPF et al. Business recipients still need additional verification beyond that, for two reasons. One, businesses will lose mail they want - if they reject mail without an authorized server match. (See article explaining Non-Malicious Forging.) Two, spammers are now the most likely to authorize their own servers - if you accept all mail from self-authorized servers, you'll accept a lot of spam and be right back where you started with lost employee productivity.

2. Scaleability. I've heard it said that it is impossible to list all email servers or to catalog all the relevant data.

Our practical experience has been that cataloging the data necessary to make the system work "pretty good" (as in Pretty Good Privacy or other modest, realistic and pretty damn good products and services) is very possible. We've done it and are doing it, and going further or all the way wouldn't be that big of a deal. Even at the present stage - it accomplishes the goals relevant and useful to businesses.

3. Anything you can do, I can do better - myself, alone, on my own email server, for free - without a central authority.

CACHING
First consider caching. If the Outbound Index has handled a million unique IP+domain requests today and your own server has handled 1,000 - would the Outbound Index cache be more likely to have the next answer you need in its cache, or would your own server be more likely to have the answer you need cached?

The Outbound Index was conceived from its inception as a community-beneficial cache. We are only dealing with data related to email servers and domains used in email - a fraction of the larger Internet world of IPs and domains. One of our top technical goals is to have a super fast cached answer to your question about IPs or domains used in email.

RESOURCE CONSUMPTION
Next, let's look at resource consumption. For an inbound email server to do a large number of  DNS lookups for each piece of mail, WHOIS lookup and parsing, possibly grey-listing, local lookups of  RHS white or black rules, SPF checking,  Habeus, Bonded Sender, IADB, etc is far more resource intensive than doing one Server Index Query protocol query to the Outbound Index.

(The SIQ protocol by the way  is unencumbered and in the public domain: clients already exist for it for Sendmail, Exchange, postfix policy daemon, plus command line clients for both Linux and Windows. Founders and contributors to the Outbound Index project created the SIQ protocol - it has an RFC at the IETF and we placed it in the public domain immediamente, just as we placed VARA and .mxout in the public domain.)

UPDATING & STABILITY

If you are going to duplicate, by yourself, on your own inbound server, the useful stuff that the Outbound Index does - I'm thinking you are a single workaholic, get little sleep and have no responsibilities other than this one task of tuning your inbound mail server. You're obsessed with anti-spam and avidly read mountains of material daily and spend feverish nights dreaming up new ways to stop spammers. You're a highly competent bit level programmer who knows all the protocols inside and out and can actualize any kind of anti-spam test you or the SpamTools gang can dream up. Bravo - let me know what mailing lists you are on so that I can learn from your trials and triumphs.

Most email admins don't fit the above description. They don't have time to be an expert in this niche area. They can't write a milter or an SMTP interfaced C or python program every day. Smaller operations install Spam Assassin and periodically update it or mess with it a bit, or find another such solution or combination of solutions. If the solution requires manual attention, updating or upgrading, it may or may not get done. It may or may not be optimally configured. After all, the people creating these solutions are themselves just learning what might be optimal before the scene shifts again.

On the downside, stability of email service is at risk every time the inbound email server is modified - thus with every required update, upgrade or "new kind of test." In an ideal world, I'd like to leave my email server untouched and let it run reliably forever just as it does today. Customers get really pissy when something goes wrong with email - more so than even their website or their workstation.

The Outbound Index was conceived as a resource-offloading, offsite, outsourced powerhouse of databases about email servers. Domains and IPs used in email. Equivalent for inbound servers - to what a render farm is for animated film makers.

ECONOMY OF ACTION

If you are operating an inbound server alone, you may not care about the 100,000 other operators like you, or the economy of action provided by adding a new kind of anti-spam test in ONE location (the Outbound Index) - automatically benefiting 100,000 guys who didn't need to risk their own server stability by installing an update or spend the time to do so. Not to mention if they each had to dream it up, test it over a wide variety of scenarios, write it and interface it to their various systems, and maintain the additional servers required to keep the data behind it up to date.

Some tests done or data the Outbound Index uses - just wouldn't be practical for you as a single server operator to carry out. But when we are serving a wide spectrum of inbound servers, it's very practical.

SUBJECTIVE vs OBJECTIVE "FACTS"

This all ties together that we conceived the Outbound Index as a community-beneficial repository of facts about email servers. What do I mean by facts?

 - NO "samples of spam" - we actually never get any headers or bodies of mail and have no clue of the content

 - NO subjective judgements or judgements applied to servers with the same statistical profile differently

Facts are things like "date the domain of the envelope-from was registered" or "churn rate of the name server" or "correlated identity data" or "number of gosh darn it had another affiliate fool us" public statements.

Leave a comment here or at the bottom of any http://wiki.OutboundIndex.net/ page and give me your feedback on the question this article title poses.