Design and implementation for sending E-mails

This has been implemented (and is being improved atm)

References

Concepts, ideas, requirements, contracts

Contracts & requirements

  • The implementation must persist messages that are not yet sent. For example on a store on a file system. Preferably in the Maildir format
  • The implementation must asynchronously handle sending messages and must trigger the "msg_sent" event to all observers when a message got sent
  • The implementation must atomically move the sent messages from an outbox to the sent folder
  • The implementation must allow a central sent folder (a sent folder shared by all transport accounts)
  • The implementation may allow a central outbox folder (an outbox folder shared by all transport accounts)
  • If possible, a classification between the available transport accounts must happen automatically when a new message is to be send. Or it must be possible to implement such a classifier

Tests

  • Sending a message to an account that is correctly configured (the service delivers it to an IMAP store): is the message received?
  • Sending a message to a NOP transport account
    • Are the messages being moved to the sent folder?
  • Sending a message to a NOP transport account that thinks it's offline
    • Are the messages being queued in the outbox?
  • Interrupting a queue (killing the application)
    • Once online, will the remainder of the queue be send?
    • Are messages lost (checking for race situations too)
  • Taking the connectivity down
    • Are the messages being queued in the outbox?
    • Once online, will the remainder of the queue be send?

Proposed UML Class diagram

I am not a bot

TODO for this diagram

  • Remove the get_draftbox method from TnySendQueue: that folder is the worry of the application developer. It has few things to do with the framework itself.

Extra & to know about this diagram

Non technical

  • Please use ArgoUML and the original .zargo file when modifying the UML class diagram (see below)
  • This UML class diagram is not the final design. While coding things might change, of course. Use it as a way of communicating idea and concepts. (if you dislike UML, make sure you do an extremely good job of explaining your ideas in extremely high detail)
  • ArgoUML doesn't make it possible to have aggregation between interfaces (nor does it make it possible to have properties on interfaces). Hence why TnyFolder and TnyMsg are marked as abstract classes

The idea

The developer will only use the TnySendQueue interface/API. The TnyTransportAccount is not usually what the developer is interested in (it becomes an implementation detail for most developers who'll use tinymail for building an E-mail client software). Both the classifier and the actual queue types implement this interface. A classifier is a queue just like any other queue with the specific fact about it, that it contains multiple queues which it processes simultaneously. It does this by aggregating one or more normal queue instances. For the outside world, a classifier is just-another-TnySendQueue.

Observers will subscribe to the queue (the top classifier or send-queue instance) to know when a message got sent. The queue will know when a message is sent because the blocking transport account has finished sending it. It's the queue that makes the account async (not the account itself). Classifiers that uses other queues will proxy the events over to notify its own observers.

Being a classifier or not is actually just an implementation detail. Using a bunch of TnySendQueues or rather one TnyTransportAccount is an implementation detail of the !TnySendQueue's implementation. TnySendQueues can therefore be nested (and we will call TnySendQueue implementations that nest other queues, classifiers).

The final queue type (one that is responsible of sending stuff) will usually, but not must, aggregate a transport account instance. The transport account instance knows how to send one message at the time. That is its only functionality (to send one message, and wait for that message to be sent). Implementations of a transport account can use SMTP, Sendmail or maybe a human slave, a pigeon, a penguin, a bear, a Jabber server and the XMPP standard or a phone that converts it to an SMS?

Example
  • The TnyAvahiClassifier knows about which network is being used for connectivity
    • In case it's a trusted or known network, it has another TnySendQueue instance of type TnyFromHeaderClassifier that
    • In case it's a non-trusted or alien network, it has another TnySendQueue instance of type TnyFromHeaderClassifier that

Configuration of this example:

UML of this example:

I am not a bot

Worse example (it's not a typical one)
  • The TnyAvahiClassifier knows about which network is being used for connectivity
    • In case it's a GPRS network, it has another TnySendQueue instance of type TnyFromHeaderClassifier that
    • In case it's a SMS capable phone network, it has another TnySendQueue instance of type !TnySMSSendQueue that will send the E-mail using "an E-mail over SMS proxy service"
    • In case it's a OLPC mesh network, it has another TnySendQueue instance of type TnyMeshNetworkSendQueue that will send the E-mail using a mesh network
    • In case it's a trusted or known network, it has another TnySendQueue instance of type TnyFromHeaderClassifier that
    • In case it's a non-trusted or alien network, it has another TnySendQueue instance of type TnyFromHeaderClassifier that

Configuration of this example:

Step by step development

This idea allows to step by step develop the components. Once a simple TnySendQueue is in place, people can already do what most E-mail clients support: sending E-mail using one specific account without any network detection or classification.

The flow

  • The start with a TnyMsg instance which is a composition of a TnyHeader and a bunch of TnyMimePart instances that are usually created using a TnyStream like a memory stream or a file stream
  • We deliver this message to a TnySendQueue implementation's instance like on the diagram the TnyAvahiSendClassifier
  • A classifier, decorating one or more TnySendQueue instances, picks one of the queues and adds the message to it. Implementing TnySendQueue can the classifier again be used inside a parent classifier this way
  • The TnySendQueue instance usually has a thread (or another way to make itself asynchronous) and usually a TnyTransportAccount which it will use as delegate for getting the message sent. It can also contain yet another TnySendQueue (in which case we typically call the send-queue a classifier)
  • Once sent, the message is moved from the queue's outbox to the queue's sentbox. This moving works recursively (so a classifier, once it got notified that the other queue is done with it, moves from its own outbox to its own sentbox, the child queue also does this for his outbox and sentbox).

Extra & to know general

  • Ask yourself what would happen if the power goes down. Will the user lose the message without them being sent? If that is the case, then that's a major problem. It's better to send it first, and then remove. Things like that. Or flag as "tried to send", "send" and then "remove". In this example, upon restart, a scan for "tried to send" messages would happen so that the ui developer could hook-in and let the user answer a question about it (try to send it again?).
  • Feel free to add your to-knows here

Current version (keep this up to date)

* sending_mails_tinymail_03.zargo

Attachments