haiku/src/servers/mail/HISTORY

Haiku Mail Daemon Replacement v3.0.3

VERSION HISTORY

November 5 2005 - v3.0.3
- Fixed up source code to compile under the newest Haiku build system.
- AGMSBayesianSpamServer renamed to "spamdbm", runtime interface between the filter and server repaired, spam MIME types and icons also revived.
- BeMail supported MIME type added for "application/x-vnd.Be.URL.mailto" as requested by tqh on BeBits.  So now opening a mailto link in NetPositive only opens one BeMail window rather than two.

January 30 2005 - v3.0.2
- Added SMTP SSL support
- Fixed a nasty bug with Message-ID generation that was caused by incorrect headers when building on BONE

December 6 2004 - v3.0.0
- Fixed all known bugs
- Removed timeout crash in IMAP
- Added POP3-SSL support
- Fixed code so IMAP could compile if OpenSSL not used.
- Integrated into Haiku source tree
- Changed installation system
- Massive API update

March 9 2004 - v2.3.0
- Fix date parsing to handle time zone offset numbers inside round brackets.
- Handle spam messages with a malformed MIME Content-Type header that specifies multipart, but no boundary marker string.  The messages show up as empty.  They also sneakily have a second Content-Type that says the message is text.  So, only use the multipart entry if there is no other, and only if the boundary string exists too.
- Makefiles changed to do a parallel make on multiprocessor systems.
- BeMail no longer breaks URLs in outgoing mails, at least it tries to do that. It's kind of a hack like the whole word-wrap algorithm, so it shouldn't do much more harm :)
- E-mail addresses are now properly recognized without the "mailto:" prefix.
- Doesn't keep as many file handles open as before (was using 4 per window, with the BeOS limit of 100 files open, that was a maximum of 25 BeMail windows).
- Included new icons by Stefano
- Fixed a horrible bug that would cause deletion of entire mailboxes on newly set-up IMAP accounts.
- IMAP-SSL support

December 29 2003 - v2.2.6
- Don't use "!" in file names of e-mail messages, the bash shell doesn't like it.
- Got the e-mail MAIL:priority attribute working.  It stores it as a text string, perferably the standard 1 to 5 (high to low) levels.  The words High, Normal, Low, Urgent and Non-Urgent may also show up in the attribute depending on which mailer sent the message.
- Added MS Exchange support to IMAP, fixed tons of other IMAP bugs (not that there were any in the first place.... :P)
- Added a status log notification method

December 4 2003 - v2.2.5
- Queries changed to be multivolume, so you can have mail directories (inbox, outbox, draft) on volumes other than /boot.
- Added new error-handling, so there aren't alert boxes. Whooo!
- Fixed all outstanding bugs in IMAP.
- More memory leaks fixed.
- General happiness.

October 12 2003 - v2.2.4
- Fixed IMAP's mailbox look-up routines so that they use LSUB instead of LIST. This fixes a few bugs with certain mail servers, and allows better interoperability with other mail clients on the same account.
- Added the X-Mailer header to satisfy Feature Request 788700.
- Made it so that the message count uses the correct singular/plural form, given the number of messages, in the daemon status window.
- CHECK_MEM build simply does not work with BONE, changed to get it to partially work.
- Fixed a socket handle leak in POP3.
- Fixed the magic non-dissapearing mail bug (when you use the delete-mail option in match header it would leave a temporary file in the inbox).
- Fixed a bug that caused thousands of error messages.
- Fixed a bug that caused RemoteStorageProtocols not to sync on startup.
- Fixed a bug that caused folder names to be appended to paths set by the Match Header filter.
- Some memory leaks fixed.

September 20 2003 - v2.2.3
- Fixed crash in POP3 protocol when the mail server was not reachable (was trying to send a QUIT command over a dead connection).
- Allowed users to drag people files (and other sorts of files) onto the Mail Status window and have them open in the user's preferred mail reader, which, in the case of people files, creates a new message addressed to the person in question.
- More IMAP bug fixes.
- Some memory leak bugs fixed.
- Added a command line script which will go through all your People files and make sure that the e-mail addresses contain valid characters.  Useful for bulk e-mailing (the SMTP server will reject the whole batch if one is bad), so look for ValidatePeopleEmails.sh in the Mass Mailing folder.
- Temporary file for incoming mail created in /tmp or now in a local directory, so that the in-box can be on a different drive than /boot.  Well, maybe not, all the query stuff still looks on /boot only.

August 20 2003 - v2.2.2
- Automatic index creation changed, use an int for the MAIL:flags, not a string!
- Use a query to find mail to send rather than scanning the Out directory (a certain person keeps thousands of his old files there and was getting annoyed at the time delay for checking mail).
- Fixed some bugs in IMAP, including one that would block ChainRunner's message queue forever.
- Fixed some random bugs in IMAP and made it so MDR doesn't redownload your whole mailbox if it crashes! Now *there* is a feature.
- Fixed window position save in BeMail - last closed window position is used for the window the next time you start BeMail.  Got broken when the settings file format got changed.
- Fixed race condition where RemoteStorageProtocol was locking the node, so init was failing in the daemon.  So no longer need the long delay before sending mail kludge.

August 3 2003 - v2.2.1
- Changed networking operations to use sockets directly rather than the buggy BeOS network kit.

July 30 2003 - v2.2.0
- Starting a new version for Nathan's second overhaul of the architecture: adding a fancy RemoteStorageProtocol for better IMAP (and others in the future) local vs remote mail folder synchronization.
- Fixed recently added bug in AGMSBayesianSpamServer which wouldn't let you double classify a message, even if the IgnorePreviousClassification option was on.
- Added hack to wait for spell checking dictionaries to load when you start up BeMail with the command line options to create a new message.  Avoids the warning about not finding the dictionaries.

July 15 2003 - v2.1.0rc3
- Added Mark Heeren's alternate new mail deskbar icons to the mail daemon resource.  They're a blue and red envelope, rather than the empty and full mailbox icon.  Use QuickRes to copy them over the existing icons in /boot/beos/system/servers/mail_daemon if you want to use them.
- Fixed AGMSBayesianSpamServer race bug with the classification choices dialog box coming up mangled due to default sizes of controls being zero.  Happens when the RefsReceived message gets processed before ReadyToRun has finished setting things up.
- Make Drafts query non-temporary, so it doesn't disappear after a few days.  Also split the Today's Mail query into Received Today and Sent Today queries.
- Turn on auto-training by default in the spam filter, at "BiPolar"'s suggestion.  Also modified the other default options - spam in subject off (status now shown in window title), leave server running (slow startup with hundreds of thousands of words), cutoffs made more strict (more uncertain messages, fewer errors).
- Added a couple of Japanese translations.
- IMAP bugs fixed, now handles network disconnection better.
- Also stops when it encounters an error while reading message contents from the server, rather than putting up error alerts for each and every remaining message.
- Changed BeMail to use a flattened BMessage for its settings, rather than the unreliable binary dump.
- Added an option to start with spell checking on.
- Added people's names to the auto-complete word list.  So if you want to send e-mail to Dane, type a double quote then the first few letters of his name; typing "Da… will expand to "Dane Scott" <dane@somewhere.com>.  The usual pure e-mail address expansion is unchanged, typing da… without the quote mark will expand to only dane@somewhere.com.  We don't have it include the name since the auto-complete is simple text substitution - would need a fancy post processing stage to figure out the right addresses and full names.
- Found some buffer overwrite bugs in SMTP which prevented some authentication methods from working.
- Moved the Save button furthur from the Send button; "Skiver" didn't like accidentally hitting Send.

July 6 2003 - v2.1.0rc2
- Fixed some IMAP bugs.  Not all IMAP servers are the same, or even standard.
- Wipe out the /boot/beos/etc/word_index/ directory when installing, since the corrupt spell checking indices crash BeMail, and they get automatically regenerated if they aren't there.
- Installer now creates the symbolic link /boot/develop/lib/(x86|ppc)/libmail.so which the older installers accidentally deleted.
- Added Robert Paciorek's central beep modification.  Use this notification method if you have several e-mail accounts and just want one beep when there's new mail in any of them (rather than separate beeps for each account with new mail).  Didn't add his RS232 CTS/RTS signal line toggling code that turns on a light plugged into the serial port - not that many users would use it and it would need preferences changes to choose a serial port, etc.  Contact him (http://www.bebits.com/devprofile/2771) if you want the modifications.
- Changed POP protocol to remove the period escape code at the start of a line, rather than having the editor hide it.  This avoids slightly incorrect message size counts (two periods should count as one), and lots of other little related off by one problems and partial or missing message problems.
- Fixed multiple send bug.  If you create several new messages per second for the same outbound account, it fired off several sender threads, and ended up sending each message several times.  The threads fought over access to their configuration files (and other unrelated outbound configuration files for some reason), which often resulted in wiped out account settings and other weirdness.  Now uses only one sender thread per account.
- Installer completely regenerates the MIME types for the applications and the data types text/x-email, text/x-partial-email so that the correct icons (particularly the partial e-mail one) show up.  So if you had custom attributes registered for e-mail files, you'll have to re-enter them.  On a related note, the Daemon now takes a command line argument of "-M" to forcibly delete the old MIME types and rebuild them, plus the existing -E argument that exits immediately if you don't have the auto-run option turned on (so it can be simply started from the boot script).
- Added a command to AGMSBayesianSpamServer to declassify mail, and the related GUI changes.  So now if you decide a message shouldn't be in the database, you can change it back to Uncertain status, which will also remove its word counts from the database.  Incidentally, I had to make a new dialogue box to choose spam/genuine/uncertain after exceeding the maximum number of buttons (3) for a BAlert.  Also changed the logo to remove the reference to SPAM, since Hormel doesn't like having their meat product associated with junk e-mail.
- Added the spam button to BeMail.  Click it to train the current message as spam and then delete the message.  Right click it to bring up a menu with other training options and long descriptions of what they do (you need to train it on genuine messages too!).  Also added spam menu items to the Message menu and a spam status display in the window title bar.  The button, menu items, ALT-K hotkey, window title modifications all only show up if you have a spam filter configured in any one of your e-mail accounts.
- For partial e-mail downloads, the spam filter will only auto-train on the partial download, it will ignore the complete message (leaving it classified as before).  That's because it doesn't know how to untrain the partial message (doesn't have the old partial message size at that point).  Of course, if the spam system isn't in headers only mode, it will read the whole message anyway.

June 26 2003 - v2.1.0rc1
- Added US-ASCII encoding and decoding, which also uses 7bit rather than quoted printable.
- Remove leading and trailing spaces from the thread string (subject minus the Re: etc bits), since some other mail software likes adding a trailing space, which messes up the thread sorting.
- Replace NUL bytes after conversion to UTF-8 with the substitute character.  This is because the BeOS ISO-2022-JP to UTF-8 conversion sometimes gets it wrong and inserts a NUL.
- Added an optional alert to warn you if you try to send a message which contains characters unencodable in the currently selected character set.
- More intelligently picks 7bit encoding if there are no 8 bit characters (for all character sets).  Also uses 8bit rather than quoted printable for Latin-1, ISO-2022-JP, EUC-KR.  Base64 used for SHIFT-JIS and EUC-JP.  Quoted printable for all others that have 8 bit data.
- Fixed a crash bug with malformed MIME empty text attachments (had a negative contents length).
- IMAP completely rewritten from scratch. Everything that was bad about the old IMAP is gone. Everything that was good about it is gone too. New goodnesses and badnesses are here (although mostly goodnesses – I hope)
- POP3 massively upgraded for slightly greater speed (gets sizes of all messages in one bulk command, receive buffer size bumped to 10K from 1K), much less CPU usage on slow systems.
- Partial Message Downloading (double click on the message file to finish downloading).
- Reworked the protocol framework. And the add-on framework. And everything else. Except BeMail. AGMS did things to that... I'm scared of it.  AGMS is too.
- Miscellaneous really awesome things.
- Spam checker now looks at file names of attachments to see if they are text files, sometimes the MIME type is specified incorrectly by those naughty spammers.
- Added instructions on how to do customized bulk e-mailing.

April 13 2003 - v2.0.1b2
- Make the encoding menu slide around when resizing the window.  Can only do it for reading a mail message, not when composing, since pop-up menus don't resize (a BeOS bug).
- Changed encoding to work on a per-word basis to reduce the chance of making words longer than the 75 character RFC2047 limit.  Will even split up a Japanese subject line with no spaces into encoded "words" which should get reassembled without spaces.
- Changed header encoding to base64 rather than quoted-printable for Asian languages, for cell phone system compatability.
- Word wrap long header lines so that if you send a message to hundreds of people, the To: header won't go over the 1000 byte line limit.
- Also recognize Content-Disposition: filename="logo.jpg" as specifying the name, though they really should use "name" rather than "filename".
- Removed query "leash" limit of 1024 max e-mail addresses in the typing auto-completion list.  Note that as usual the addresses are found via query from files only on the boot volume with the META:email attribute (and META:email[2-5] too), and groups with the META:group attribute.
- Removed 128 item leash on the pop-up email address choice menu, also avoid deadlock when choosing hundreds of names (which used to exceed the message queue size) by batching them up into one operation.
- Add All People menu item, now only people not in any group are listed at the bottom of the pop-up address menu.
- Fixed a PPC (PowerPC CPU version of BeOS) bug in AGMSBayesianSpamServer (atoll wasn't working so word database wasn't read), and changed sound effect format so they work for PPC.
- Also, besides double quotes, don't encode <>,@() in the headers so that the mail system can see them.
- AGMSBayesianSpamServer now shuts down immediately and returns true if it is asked to quit by the registrar.  Previously if you left it running, the system shutdown would be cancelled at AGMSBayesianSpamServer.
- Added momoziro's Undo/Redo stack to BeMail.  Thanks momoziro!

February 20 2003 - v2.0.1b1
- Yet more Japanese translations, courtesy of Jun Suzki this time.
- Added an international install script feature.
- Fixed crash bug when using Open With…BeMail on people files (was a buffer size bug).
- Add BeOS USer Guide Replacement for the E-mail preferences aaplication.
- Do a "sync" to flush data to disk if any messages received, so that a crash soon after receiving e-mail won't lose data.
- Sort the pop-up lists of e-mail addresses by last name (formerly e-mail address).  Also list people in the groups they belong to and in the main list too.  Note it uses an n-squared stupid sort algorithm, so it will be slow if you have thousands of People files.
- Added an Encoding menu to chose the character set when sending mail, in case you want to temporarily override the default set in preferences.
- Also have the encoding pop-up menu let you change the character set to use for reading a message, though by default it will automatically determine it.
- Fixed up excessive <<>> around e-mail addresses in list when typing a group name.

February 6 2003 - v2.0.0b9
- Use quoted printable for the headers to avoid base64 extra CRLF insertion which makes a mess if the header (like Subject) is too long to fit on a line.
- Added 7bit and 8bit encoding to the library, and made BeMail use 7bit for ISO-2022-JP (base64 for other Japanese character sets).
- If a header field (CC, Subject, etc) is made empty in BeMail, also remove that header.  Was causing a reply (with an automatically generated CC) to get sent to extra people even though the CC seemed to have been erased.  Library code changed to accept NULL and empty value strings as meaning remove the header.
- Made install script into a one use script, so you can't accidentally run it twice (it won't work since it moved a lot of the installation files to their final resting places).
- Reduced the size of the spam database from around 1500 messages to just 10 spam and 10 genuine examples (after complaints it was too big).  Accuracy will be reduced, but then you can quickly make up for it by training it when it gets it wrong (and even faster with auto-train).
- As requested by "dolgogi" in the BeBits.com talkback, I've added the ksc5601, ks_c_5601-1987 character sets as an alias for euc-kr in incoming mail (they seem to be subsets of euc-kr).  Outgoing is always EUC-KR.  Also, the output for EUC-KR is sent as 8bit rather than base64 or quoted printable.  Sorry, unable to support iso-2022-kr.
- Fixed bug in date parsing (affecting the MAIL:when attribute) where it didn't understand the new numeric +0000 style of time zone.
- Changed file names to be a bit more readable, hopefully avoiding characters which cause problems.
- Avoid having a single period on a line; fix SMTP protocol to insert an extra period on all lines starting with a period.  POP protocol should do the opposite, but for now the extra period is hidden by the display code!
- Fixed bug with missing last letter in Japanese headers by forcing it to switch to the Roman character set (could also switch to ASCII, but they are almost the same) at the end of the header.
- Compensate for Yahoo! address inside the name goofyness, find "Joe" for the name when given: "Joe <joe@yahoo.com>" <joe@yahoo.com>
- Added Koki's first batch of Japanese translations for text within MDR.
- Fixed up word wrapping code to fall back to the user's preferred font if the system fixed width font didn't have as many of the characters used in the message.  Doesn't depend on character set choice now.
- Added some more queries to the pop-up daemon menu and changed it to largely use a zip file so it will be easier to add new ones: Today's mail, mail from someone, mail with a subject containing something.

January 22 2003 - v2.0.0b8
- Date format changed slightly to make it compliant to RFC-822 (some other mail apps had problems understanding it).
- decode_base64() should now work with single character line breaks as well.
- Changed default spam cutoff ratio to 0.95 to correspond to the default Chi-Squared scoring method.
- Added Uncertain spam classification as suggested by BiPolar.  This also means adding a Genuine cutoff number, a new sound effect, change the prefs window, etc.
- Fixed up BeMail to use the user's character set choice (was ignoring it).
- Added a few more character set encodings, including UTF-8 for BeOS native messages.
- Made the character set choice affect the encoding of the Subject, To, CC and other header lines.  So now your subject can contain international characters.
- Made it internationalize (using utf-8, no choice) the file attachment headers so file names with weird characters will now work.
- Don't convert words to lower case for spam checking, the case can be an important clue!
- Updated sample spam database: now has capitalized words and includes fresh spam.  Old uncapitalized databases won't work as well.  Either replace your existing one with the sample one, or be prepared for worse results until more training corrects it.
- Headers in ISO-2022-JP should now appear correctly.  Fixed up header encoding so that it also turns on the encoding markers for escapes and control characters, which you see in the 7 bit encodings like ISO-2022-JP (previously it only did them when it found 8 bit characters).
- Missing Japanese text at the end, or garbage characters at the end bug fixed.  Increased the conversion buffer size calculation so that the ballooning of text due to the excessive number of escape characters in ISO-2022-JP doesn't run it past the end of the buffer (both body text and headers were affected).

December 13 2002 - v2.0.0b7
- Fixed vman's bug with TABs after the colon in the headers making it ignore the header.
- Fixed the Forwarding with Attachments bug so it now works with X-BFile type attachments.
- Added a couple of forwarding menu items, so you get the complete set on the main menu and the right-click on the Forward button menu.
- Added full set of reply options to the right-click-on-the-reply-button menu.
- Removed old word wrap menu code from BeMail.
- Added a preferences option to attach files in BeMail without the attributes, so that users of other operating systems don't get confused by the extra attachment.
- Fixed some bugs with reading (garbage appended bug) and forwarding text attachments (didn't copy text so it sometimes crashed).
- Include preamble text when making a reply from selected text in a message.
- Added ChiSquared (Greek X is called chi, capital is Χ, lower case is χ) statistical error scoring method to AGMSBayesianSpamServer; you lose the shades of gray and get better spam/genuine/don't-know results instead.
- Reclassifying a message as spam or genuine will now also update the spam ratio attribute (it runs an EvaluateFile on it).
- Subject prefix of [Spam 99.9%] removed from the subject attribute when you evaluate a message (the number would become incorrect, and we can't tell if it is spam since the cut-off value is in the filter client, not the server).
- Implemented self-training mode in the spam filter.
- Load the database when the spam server window is displayed.
- Hide the spam server window so it doesn't flash briefly onto the screen when the program starts up.

December 1 2002 - v2.0.0b6
- Fixed Reply-All bug where it would get stuck in an infinite loop when it tried to remove the sender's address from the list of addresses.
- Forgot to mention spam database was updated in Beta 5 to reflect MIME decoding improvements (base64 text now decoded) and to add new spam examples.
- Fixed Reply-All CC bug where some CCed addresses weren't included.
- Added reply preamble options for full name and date, so you can have something like "Joe Who wrote on December 23 2001 18:33:" inserted in your replies.
- Fixed Draft loading so you get the From, To, CC etc instead of blanks.
- Changed MAIL:draft attribute type to be Int32 to match the index type, also create the index as INT32 (in daemon and BeMail) hopefully stopping the invisible drafts problem.  You might have to do a "cd /boot ; rmindex MAIL:draft" command to remove a bad old TEXT index from before (use "cd /boot ; lsindex -l" to see what you have).

November 29 2002 - v2.0.0b5
- Changed readfoldedline and MIME parsing to both handle lines ending in either CRLF (the standard for Internet mail) or LF (sometimes used for local file storage).  Also handles EOF and errors better.
- Fixed a bug where an empty message sub-part would cause a crash due to BString UnlockBuffer(0) not working for empty strings.  This generally made it crash when it tried to open a malformed message.
- Check for IO errors while writing generic headers and copying existing MIME components.
- New "Reply With" option for the Match Header filter.
- New "Set As Read" option for the Match Header filter.
- Old style name extraction fixed so that StripGook "joe@foo.com (Joe)" will return "Joe" rather than "Jo".
- MessageIO fixed so that it doesn't reread the whole message when doing a SEEK_END, doubles performance when checking for spam!
- Added a close window box in the title bar of the E-mail preferences, acts like the Cancel button.
- Changed AGMSBayesianSpamFilter so that it checks the server's tokenize mode, and if it is JustHeaders then it will only download the headers for spam testing, rather than the entire message.
- Updated AGMSBayesianSpam documentation with a trick for spam checking without downloading the whole message, and explained the word display.

November 20 2002 - v2.0.0b4
 - Zillions of bug fixes. Too much to mention, really
 - Lots of new features (we will let you have fun discovering them)
 - Profuse apologies for waiting so long to release this
 - Bayesian Spam Filter included

February 12 2002 - v2.0.0b1
  - Serious speed and stability improvements
  - Friendly addon names
  - Finalized, beautified, and FBC-protected API
  - Crash on shutdown bug fixed
  - Colored quotes
  - Attachments work now :P
  - Lots of other things are much better too, but I don't remember them now

November 21 2001 - v2.0.0a2
  - Attachments that used not to be handled are handled now.
  - HTML emails are displayed as the HTML source.
  - Implements all stub functions programs like Postmaster expects (most functions do nothing).
  - Mail handling is much faster
  - Various API improvements
  - Blinking lights support (Notifier)
  - Really improved Deskbar menu! Uses NavMenu. Check it out!
  - Right-clicking the deskbar replicant while pressing the SHIFT key gives more options
  - Central Notification (uses only one Alert for all accounts).
  - Changed links for bug reporting and others
  - Button to enable configuration of the Deskbar Replicant menu
  - Do not crash when opening pre-MDR2 emails
  - Reply to Sender
  - Reply to all
  - Resend/Forward works
  - Better Notifier Filter
  - Better RuleFilter (Descriptive filter names)
  - Better Folder Filter (No more email filename colisions)
  - SMTP Auth now works for real

November 06 2001 - v2.0.0 a1
  - Complete rewrite from the ground up
  - Filters
  - BeMail
  - IMAP
  - SMTP Auth
  - POP3 Auth for SMTP
  - Sort emails by thread in Tracker
  - New Mail Kit
  - All sorts of other cool stuff

July 05 2001 - v1.0.0 b5
  - Outgoing (SMTP) mail servers are now configured per account.
  - Leave messages on server now works as expected.
  - You can select your default acount from the Deskbar menu (fixed).
  - Add-on loading problem under BONE seems to be fixed now.
  - Various other bug-fixes and performance improvements.

June 12 2001 - v1.0.0 b4
  - E-mail preferences app completelly rewritten.  Now it is cleaner,
    more intuitive, and font sensitive!  Please send us suggestions for
    improving it.
  - Added buttons to create/erase accounts.
  - The Mail Status window now remembers its size and position.
  - Option to never show the Mail Status window.
  - More bug fixes. More error checking is now being done.
  - First version of the filter API.
  - Some brand new bugs for you to play with. :)
  - The old mail_daemon icon is back! Thanks to Syn.Terra for it.

June 08 2001 - v1.0.0 b3
  - Installation script now creates an index for the MAIL:account
    attribute.
  - mail_daemon detects if the MAIL:account exists and creates it
    if it does not.
  - Various bug fixes. *ALL* Deskbar crashes should be gone.
  - Improved Deskbar Icon menu.
  - Does not check for email immediately after starting.

June 06 2001 - v1.0.0 b2
  - Fixed lots of bugs!
  - The status window can now be made persistent.
  - E-mail preferences app works better (for some definition
    of better).

June 05 2001 - v1.0.0 b1
  - First public release.