When new messages arrive

Note: This has been implemented using RFC 2177 (IMAP IDLE) on IMAP servers. Support for poking IMAP servers that don't have IMAP IDLE support is planned.

About

Whenever new messages arrive, tinymail should add them to the model that is being used by the folder-summary view.

Morale

Because a list that implements both a TnyObservable and the TnyList is inconvenient (each type that wants to register itself, must also implement the observable interface), the idea is to provide a decorator that implements being observable and decorates a list so that it can add and remove the changed items from the lists (the models) that need updates.

Status

The API for this idea has already been committed (the interfaces, the observer paradigm, etc). An implementation that uses IMAP IDLE has been committed to libtinymail-camel and camel-lite in the repository. This implementation is being tested.

Other implementations that are likely going to hit the repository very soon are SyncML and one that allows to 'poke' the IMAP server to tell me about changes. Most likely using UIDNEXT and comparing that uid with the last retrieved message summary's uid.

Contract and specifications

• It must be possible to invoke from the client when the updating should happen
• If not "Push E-mail", the updating must not happen automatically (the client developer must provoke the updating at the time he thinks it's interesting to perform it)
  • A background worker process (in a singleton) that makes it happen automatically is optional (would-be nice)
• Updating full accounts is not desirable because this implies that each folder must be instantiated (which consumes a lot memory)
• Therefore it's desirable to have the possibility to specify which folders must be checked for updates
• When updates arrive, it must be possible to let the client developer implement the action to undertake (it must be 'typed' as an interface)
  • Not always is this action to add and remove headers from a TnyList, most often the action is going to be more simple and less intrusive
  • An example such action can be updating an integer called "unread messages" and/or "read messages", while not caring about the headers that where added themselves
• A default inheritable implementation must add added headers to a TnyList (prepend) and remove removed headers from it

Poking

Using the STATUS IMAP command you can get the next uid (UIDNEXT) and the amount of messages (MESSAGES) in a folder. If of that folder the last UID in the summary isn't that UIDNEXT value, the poke command can then decide to retrieve the latest information and invoke a notification of changes to the observers with a TnyFolderChange instance that is filled in.

Push E-Mail using IMAP IDLE (RFC 2177)

THIS IETF RFC describes a "Push E-Mail" technique for IMAP (IDLE). It basically comes down to that a very low traffic connection is kept open with the client, and that the server throws (or pushes) events to the client (in stead of passively waiting for the client to ask for events).

New message got added

C: IDLE
S: + idling
time passed by ....
S: * 501 EXISTS  
S: * 1 RECENT 
C: DONE
C: A02 UID FETCH 501 (UID FLAGS ..)
S: * 501 FETCH ... 
S: A02 OK FETCH completed 
client updates its summary ...
C: IDLE
S: + idling
time passed by ....

Old message got removed

C: IDLE
S: + idling
time passed by ....
S: * 25 EXPUNGE  
client updates its summary ...
C: DONE

At the event that the server pushes a RECENT, EXPUNGE and/or EXISTS during the idle-tuime. A TnyFolderChange instance will be created and used to notify the observers with it.

Information to collect upon such a change

Use the _tny_camel_header_set_as_memory API of libtinymail-camel to create a new TnyHeader instance using a CamelMessageInfo instance.

On removals

• The UID unique id of the added message
• All the others are optional (and the same as on addition)

On additions

• The UID unique id of the added message
• The Cc header of the added message
• The From header of the added message
• The To header of the added message
• The Subject header of the added message
• The Received header of the added message
• The Date (date sent) header of the added message
• The Message Size in octets of the added message
• The Flags of the added message (Seen, Deleted, etc)

Class diagram of the current idea

About the diagram

• The software does not allow certain features in interfaces, therefore have all interfaces been marked as abstract classes (italic name)
• You can find the original .zargo file (which is produced by the free software Java package ArgoUML) below
• The latest version is "when-new-messages-arrive.3.zargo"
• The "IMAP Service" interface is RFC 3501 or 2060 with optionally IMAP IDLE (RFC 2177)
• Or it's the ad-hoc folder interface of CamelPOP3Folder, or NNTP, or ...

Example in libtinymail-camel's TnyCamelFolder

Note that this is internal implementation code. This is in no-way how you would use the API of tinymail (this is sample code that implements the API internally)

When adding a message happened

static void
on_header_got_added (TnyCamelFolder *self, const gchar *uid)
{
	TnyFolderChange *change = tny_folder_change_new (TNY_FOLDER (self));
	TnyHeader *hdr_addded = tny_camel_header_new ();
	CamelMessageInfo *added;
	struct _camel_header_raw *headers = NULL;

	/* Code to get the message-info for uid */

	camel_header_raw_clear(&headers);
	camel_header_raw_append(&headers, "From", "me@me.com", -1);
	camel_header_raw_append(&headers, "Cc", "me@me.com", -1);
	camel_header_raw_append(&headers, "To", "me@me.com", -1);
	camel_header_raw_append(&headers, "Subject", "Testing something", -1);
	camel_header_raw_append(&headers, "Date", "Fri, 19 Jan 2007 20:31:12 +0100", -1);
	camel_header_raw_append(&headers, "Received", "Fri, 19 Jan 2007 20:31:12 +0100", -1);

	/* For removals, just fill in the uid (and use NULL for headers) */
	added = camel_folder_summary_info_new_from_header_with_uid (NULL, headers, uid);
	((CamelMessageInfoBase *) added)->size = 1024;
	camel_message_info_set_flags (added, 1, ~0);

	/* Prepare the TnyHeader instance (it becomes the owner too) */
	_tny_camel_header_set_as_memory (TNY_CAMEL_HEADER (hdr_addded), added);

	/* Add it to the TnyFolderChange */
	tny_folder_change_add_added_header (change, hdr_addded);

	/* And notify everybody about this great event! */
	notify_observers_about (TNY_FOLDER (self), change);

	g_object_unref (G_OBJECT (hdr_addded));
	g_object_unref (G_OBJECT (change));
}

When removing a message happened

static void
on_header_got_removed (TnyCamelFolder *self, const gchar *uid)
{
	TnyFolderChange *change = tny_folder_change_new (TNY_FOLDER (self));
	TnyHeader *hdr_removed = tny_camel_header_new ();
	CamelMessageInfo *removed;

	/* For removals, just fill in the uid (and use NULL for headers) */
	removed = camel_folder_summary_info_new_from_header_with_uid (NULL, NULL, uid);

	/* Prepare the TnyHeader instance (it becomes the owner too) */
	_tny_camel_header_set_as_memory (TNY_CAMEL_HEADER (hdr_addded), removed);

	/* Add it to the TnyFolderChange */
	tny_folder_change_add_added_header (change, hdr_removed);

	/* And notify everybody about this great event! */
	notify_observers_about (TNY_FOLDER (self), change);

	g_object_unref (G_OBJECT (hdr_removed));
	g_object_unref (G_OBJECT (change));
}