Note: This idea is outdated.

Query folders

Abstract

The idea here is to loop all folders (that match a folder-query of TnyFolderStoreIface?) and then loop all its headers. After looping all its headers, destroying the folder (killing the needed memory for the folder, which is a lot).

While reading the headers, store (in an mmap()able file) the headers that matched the query. And then mmap that file and build a TnyQueryFolderIface? with TnyHeaderIface? instances in it.

This bypasses Camel, so the uid of each header and the Folder-id of the message behind the header must be stored per header.

What is the key idea here

The fact that you can destroy the folder instances. They consume most of the memory. In Evolution, the VFolder feature is implemented by keeping all folder instances alive. This is why Evolution consumes a lot of memory. You can't do this with tinymail, which has a focus on low memory consumption.

The folder instances of Camel contain the data itself to the summary information. Like the from, to, subject, dates and content-info (like whether or not there are attachments and the mime-part types) of your messages. So basically, these instances contain "the" information you are querying.

In stead of keeping "that" information in memory (for ALL messages, which is what Evolution does) and then simply pointing to that information from the VFolder, this idea would "copy" the information "that matches" (and only THAT information) to a file on the filesystem and then use that file as if it represents a folder (a TnyFolderIface? implementation will do that trick). The implementation that implements this idea would do that using mmap(). Just like the mmap() patches make Camel already do it for all its folders (being the person who implemented that, I hereby say that it's definitely not impossible).

Why not just store full headers in memory

The focus of tinymail is to work on devices with few memory resources. There's typically much more flash disk space than ram. Therefore writing it to a file on the file system and then mmap()ing that file.

What if there's no flash disk space

Would be strange because then also tinymail wouldn't work. But a second implemenation of TnyQueryFolderIface? could create TnyHeaderIface? instances that are stored in memory in stead of read from an mmap().

Changes to the information

The query folder would not show that until it's rebuilt. This is indeed a problem of the idea. But on a mobile device, closing a query folder probably means that the user will not need the query folder anymore).

But but??

Look. The folder type of tinymail is an interface. The header type of tinymail is an interface. How I implement these types doesn't matter, as long as they implement the interface. They don't have to use Camel for doing what they do. So I can create my own "virtual folder" type that uses "its own mmap".

I will have to fix a few hacks in libtinymail-camel that (at this moment) makes mixing different type-implementations impossible. A virtual folder feature is worth the work for fixing this.