Re: [kde] Once again kmail's folder creation fails



gene heskett posted on Fri, 19 Aug 2011 04:53:39 -0400 as excerpted:

On Friday, August 19, 2011 04:00:01 AM Duncan did opine:

You mention the kde version (good), but do note that kmail is part of
kdepim, which unfortunately makes things rather more complicated than
that.

Kmail help says 1.13.7

So the still not akonadified kmail1. The below kdepim info confirms it
as well.

The problem is that kdepim was stuck at the 4.4.x level for some time
as they worked out problems in the new "akonadified" kmail2. There was
a kdepim 4.6.0 and 4.6.1 release, but they came out much later than the
rest of kde 4.6 and apparently were even then for early-adopters only.

This is a pclos install, about 12 hours overdue to update, so one could
call it up to date. According to the /var/cache/apt/archives contents,

kdepim is 4.4.11-1. Dated April 23 2011

kdepimlibs4-4.6.5 dated July 1 2011

OK, kdepimlibs, despite the name, is not part of kdepim tarball, but
rather, part of the kdelibs tarball, so it's going to be the same as
mainline kde.

But kdepim 4.4.11 tells the story. You're still on the 4.4 series there,
so the not yet akonadified kmail1, confirming the above kmail version
info.

Meanwhile, kdepim got back in sync with the rest of kde for 4.7.0, so
with the 4.7 series, you can again refer simply to the kde version when
talking about kmail (except that some distros /may/ decide to stick
with the older kdepim 4.4 thru 4.7, too, but that'd be distro-specific
if they did).

pclos has been very aggressive, passing your updates on to the users
quickly, often the next day, after a new release.

Well, not /too/ aggressive, it seems. 4.7.0 has been out for a couple
weeks, now. The date on my archived binary package (created with the
gentoo build-from-source as I have FEATURES=buildpkg set) is July 28.

And IMO they're wise to be playing it a bit cautious with the early 4.7
series, particularly since 4.7.0 is the first to reintegrate kdepim into
mainstream kde releases again, and the 4.6 kdepims were rather rough,
kmail-wise, as I said.

Provided kde doesn't backslip with the supposed-to-be-bugfix-only
releases as they did with the 4.6 series on 4.6.1 and 4.6.2, I'd guess
you'll get 4.7.2 or 4.7.3 (they might skip 4.7.1 too, if they're
cautious) perhaps a bit after release, and further 4.7 series releases
will be pretty close to upstream kde timing.

Here's the warning. I suspect you, as me, aren't going to be
particularly happy with the new akonadified kmail. However, if you
have a large existing local mail store (not IMAP, local mbox or
maildir) as I did, migrating to claws-mail won't be particularly easy.
OTOH, unless they make migration from kmail-1 easier, that migration
won't be particularly easy either.

Oh oh. My email corpus is about 10Gb, reaching back 9+ years for 3 of
the lists I am on.

OK, so your archive is way bigger (12X or so, since mine is "only" 800-ish
MB), tho my archive goes back further if we include the import from MSOE,
or about the same if not.

That's not unexpected tho, I guess, since I'm not a particularly heavy
mail user, and all my list traffic is handled by pan (news client), using
gmane.org's list2news. I have about 800 meg in my text-group pan
instance as well, having loaded a couple of the gmane list-groups as far
back as gmane carried them (2002), tho not all of them, and I still have
my ISP's old groups archived as well, altho I don't have an active server
carrying them, ATM.

But still, that'd be only about a sixth the size of your archive.

Back on topic, if my problems were mostly with the older posts imported
from MSOE, you may still avoid them. Another of my problems that you may
or may not avoid, was that I was using a non-default kmail maildata-dir
location. If you're using the default, the auto-upgrader will likely
spot it just fine and manage it better than the manual import process I
used when the auto-upgrade apparently didn't detect the datadir, as I
found out later that that the import-converter is using older and
possibly stale code.

It's also sure that the upgrade process is continuing to mature. Kevin
Krammer, one of the devs who worked on it, took my feedback and that of
someone else (also running breaking-edge gentoo) on the process with the
kdepim 4.6.0 version, and I'm sure the upgrader is the better for it,
now. I believe they've gotten feedback from a number of others who ran
that version too. 4.7 was the first they could have really integrated
any of that feedback, and by 4.7.2 or so, when I expect it'll hit the
main binary distros, refinements on that should have been possible as
well.

But either way, assuming you are still using the older kdepim 4.4
series kmail1, expect that you may have some issues when you upgrade to
kde 4.7 and thus the akonadified kmail2. If you're thinking about
switching to something else, doing it now, before that upgrade, will
save you the hassle of both that upgrade, deciding you don't like it,
and then switching to something else, and either way, upgrading kmail
or switching, you can expect some difficulties in the process. How bad
those are depends on a lot of factors, including whether you want to be
sure and take all your existing mail with you, whether you're on IMAP
or POP3+local-folders, how many mail addresses you have, whether and to
what extent you take advantage of kmail's filtering system, etc.

I use the hell out of kmails filtering system, but because kmail is
single threaded, all mail suckage is handled by
fetchmail/procmail/spamassassin, and what passes that gauntlet, with
mailfilter in front of fetchmail, is dropped into the
/var/spool/mail/gene mailfile, which kmail can then process without the
lengthy composer stalls caused by kmail pulling the mail from the ISP's
servers. That has been my solution to the composer stalls, and works
exceedingly well since I figured out a way to notify kmail when new mail
is available using inotifywait, so kmail is now synchonized to the mail
appearing in that file. It is now pulled, filtered and sorted a few
milliseconds after procmail deposits it in the above mailfile.
Effectively solving the single, most maddening issue with kmail,
something I have railed about, at length, probably enough that I have
been in Ingo K's killfile for a couple years. His priorities apparently
didn't match mine. Now that I have solved the problem my way, its a
shrug. ;-)

OK, I had (obviously) forgotten the details of the earlier threads. Now
I'm remembering... it was /you/ that was working on that inotifywait
thing for kmail. =:^)

Given that level of hacking, and the backups you mention below, you
should handle it OK. But as I said, definitely expect to spend some time
with it, working out any weird issues with the upgrade, etc, and you'd be
wise not to plan on depending on kmail for a couple days afterward, in
case it takes you a bit to work out the kinks.

But the fact that you're doing the spam-filtering, etc, before kmail even
sees the mail, means your kmail filters are probably simpler than mine.
Which means that they'll be easier to setup again if you decide to switch
to something else. FWIW, there were claws scripts I was able to use to
automate the import of mail and vcf/contact data from kmail/kaddressbook
(tho I had to hack them up a bit to get them to work; they weren't
exactly current), but I had to setup the fifty-ish filters manually as
the way the two apps handled filtering was different enough that
automation would have been... complex (tho possible if I was determined
enough and had, say, several hundred filters instead of fifty).

Maybe kmail-2 will be multithreaded? I suspect that may well open a
configuration can of worms similar to my solution, which has been
several years in the genesis.

That's actually one of the features of the akonadi setup. It's WAAAYYY
different and will likely have you modifying your inotify setup quite a
bit as well. In general, kmail is now just a front-end for akonadi, with
akonadi-based accounts doing all the fetching, etc. Akonadi, meanwhile,
has backends for various resources including contacts (kaddressbook, vcf
files, etc), notes (knotes), organizer/scheduling (korganizer), maildir
and imap (kmail), etc. They have an nntp/news resource but hadn't yet
converted knode to use it, with kdepim 4.6 at least (I'm not sure about
4.7, but got the impression that would be more like 4.8), and have plans
to have an rss/atom news/feed resource and eventually convert akregator
as well, but that's further down the line I believe.

Meanwhile, akonadi uses a database (virtuoso, mysql, or postgres, mysql
is the older default, virtuoso the newer one, but you are probably on and
will stay on mysql unless you switch it manually) to track it all, altho
the individual resources remain in their native formats for backup and
compatibility purposes. So for kmail maildirs, it can simply copy and
index them (or perhaps indexes them in-place, I'm not sure as the auto-
migrator didn't work for me, as I mentioned above, but I /think/ it uses
a fresh location, just to be safe). For kmail mbox format, I think it
copies the messages to maildir in the new location and indexes that.

Either way, the existing data should remain safe since akonadi doesn't
actually convert it in-place at all, but rather copies it to the new
maildir location (at least for mbox, for maildir, as I said, I /think/ it
does, but it might just index-in-place) and indexes it.

But just to be sure, it's probably worth double-checking your backups
(and testing that you can actually restore from them) before you do the
upgrade, especially since you're forewarned AND already have the backup
procedure in place. It certainly can't hurt, and should give you
additional peace of mind knowing you have a backup of that 10-gigs of
precious data as you watch the upgrade process do its thing.

But... that ALSO means that you can expect another 10 gigs of database
index, too, in addition to the native format storage. =:^(

The heavy database bit was actually what finally drove me off. I don't
use enough bits of kdepim to get much benefit from the shared backend,
and just want mail to be mail, the feedreader to handle feeds, the news
client (which was pan not kdepim's knode before, here, so that didn't
change) to handle news and the lists thru gmane, etc. I had no use for
the scheduling/notes/organizer functionality at all, was already using
pan for news so have no use for that, and prefer my mail and feed readers
entirely separate so wasn't interested in that integration, either.

And the heavy database stuff was severely dragging down system
performance (much like many antivirus users on MS Windows, or those who
go without and perhaps get viruses that drag down the system as a result,
I didn't actually realize how badly until I got it off the system, it's
like I just boosted the CPU by a couple cores or 500 MHz!), all just so I
could run kmail. Plus it kept popping up reminders at every boot telling
me I had disabled nepomuk so part of akonadi's functionality wouldn't be
available, but enabling it would have meant even MORE resource usage on
the system (I know, I had it enabled for awhile).

Plus, while I planned for mail archive usage on my /home partition, I did
NOT plan for gigabytes of database indexes of the SAME data, or gigabytes
of nepomuk file index data either, for that matter, and ended up running
out of space a couple times with nepomuk/strigi indexing, until I turned
it off. (I have a dedicated media partition for the big stuff.) After
getting all the semantic desktop crap off the partition, I suddenly have
less than half of my 16-gig /home partition used, again, instead of
worrying about running out of space all the time! =:^)

And claws-mail really is as fast and resource-light as they claim,
certainly coming off the bloat of the akonadified kmail but I /think/
it's lighter/faster than kmail1 was, too. As for features, they seem
quite comparable, with customizable hotkeys for all the claws-mail
actions too, comparable filters plus the ability to invoke scripts and
external commands similar to (or arguably even more so than) kmail, etc.
The message-per-file MH format is similar in concept to maildir, tho
there are enough differences that a conversion script was quite useful
(even if I /did/ have to hack it a bit), etc. Claws-mail is gtk2-based,
so users with firefox or anything else gtk2-based on their system will
have very few additional dependencies (unlike a full gnome-based client,
for instance), and while the kde integration isn't /perfect/ with kde's
apply color-scheme to non-kde apps option checked, it's close enough.

FWIW, I ended up using a /second/ claws-mail instance with its rss plugin
installed in this one, to replace akregator as my feed-reader, too.
Claws-mail will only run a single instance by default, activating it
instead of a new instance (even if you feed it a different config file on
the command line), due to its use of a command-socket. However, setting
both $HOME and $TMPDIR to point to something other than the normal
defaults, in a wrapper-script for the feed instance, got it working. And
now I have BOTH the benefits of two separate apps (separate config,
separate instances, separate data, separate plugin, filters, and
notifications) AND the benfits of integration into the same app (same
custom hotkey config, same filter syntax tho separate filtersets,
familiar interface).

Plus as I said, I can't quite get over how much faster they load at kde
start, and how much more responsive the system seems, plus the gigs of
saved space on /home.

(Actually, system-startup-time-wise, I'm thinking about a third claws
instance now too, using its news plugin, to replace pan. Unfortunately
pan does NOT scale well with huge amounts of archived posts, since it
loads the threading dynamically, for each group, at startup. Right now,
~800 MB of mostly text posts, it's taking several minutes of disk
thrashing from cold-cache, and that's with quad-spindle RAID-1, it'd be
even worse if it were trying to load that from a single spindle. Once
everything's in the kernel's disk cache, it doesn't take long to restart,
and I start pan with kde, so it's not like I'm usually waiting on it, but
still. Now that I see how fast claws is with the same amount of mail, I
can't help but wonder if it'd be as fast with news too, at least for my
text instance, which is what I use most. I might keep pan around for
binaries, if I ever get back into 'em, anyway.)

I did look at claws once, quickly but not deeply, couldn't develop any
enthusiasm for it, but it was years ago. Things generally improve with
age. That saying about 'the older I get, the better I _was_' comes to
mind. ;-)

I looked at it years ago too... about the 9 + years ago that I've been
using kmail, in fact, as I looked at claws for mail and news when I first
switched to Linux when MS pushed me off with eXPrivacy. Back then, pan
was the clear winner for news, and kmail for mail, as compared to sylpheed
and claws, which back then was the experimental version of sylpheed. But
claws and sylpheed went their different ways, and I must say I've been
very impressed indeed with what claws has become.

Of course, that's not saying it's a suitable substitute for kmail in your
case, but it really was the perfect replacement for me, here.

The only worry I have at this point is that eventually, gtk2 will be
deprecated in favor of gtk3, and while I found the firefox bug on its
migration and thus know that they're planning on doing it and have some
idea of where they are in the process, I've not checked into that for
claws, yet. If claws doesn't port to gtk3, it'll probably eventually be
inviable for me, here. But development does still seem quite active, so
I expect they're looking at it. I just don't know the status.


Meanwhile, when/if you upgrade to kmail2, I'll be around if you run into
problems like I did, and can try to help with them, altho I don't have it
installed now.

And if you decide kmail2 isn't for you any longer and are interested in
trying claws but want help or hints with the conversion, ping me here. I
may join their list(s) but haven't yet, so don't know if I'll be there.
I will be here, but of course that's not kde, so we might take it
offlist. OTOH, it's possible there will be others trying to find a
replacement for akonadified kmail by then as well, in which case it might
fit right in with ongoing discussions. I guess we'll see what that
bridge looks like if/when we get to it.

But it could well be that with 10 gigs of mail, etc, akonadi will work
well for you, particularly if you want integrated well indexed full-text
search. (OTOH, the nepomuk/strigi indexer could provide that pointed at
claws' MH format dirs just as well, and be integrated with kde that way,
tho it'd not be mail app integrated as well. But since just as maildir,
mh-format is single-message-per-file, working with individual message
files is quite easy with either one, FAR easier than with file-per-mail-
folder mbox, for instance.)

But you know to be prepared now, which is good, given a 10-gig mail store
to upgrade. There's almost sure to be /some/ issues with /something/ in
that, so double-checking your backups and giving yourself a few days of
leeway that you don't need to depend on mail during, for the upgrade,
will certainly simplify things if there ARE any complications in the
upgrade.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

___________________________________________________
This message is from the kde mailing list.
Account management: https://mail.kde.org/mailman/listinfo/kde.
Archives: http://lists.kde.org/.
More info: http://www.kde.org/faq.html.



Relevant Pages

  • Re: [kde] Problem with kmail
    ... Other than that, I don't have a direct response, but there's some kmail ... Both kmail and kaddressbook ship as part of kdepim, ... from kde, which splits up the main kde software release into a number of ... It was thus that the database-backed akonadi middleware framework was ...
    (KDE)
  • [kde-linux] Re: KMail Distribution Groups
    ... About KMail shows KMail Version 1.13.6 Using KDE Development Platform ... The medium length version is that the kdepim folks have ... I'll be able to remove all of kdepim, akonadi, etc, from my system, and ...
    (KDE)
  • [kde] Re: KDEPIM 4.6 prob^Wimpressions
    ... discussion, so it's installed), tho I do use kmail and akregator... ... The problem with this was that folders opened in tabs did not get ... the Akonadi resource for itself, which it does every few minutes. ... So all this stuff is in Akonadi's mysql ...
    (KDE)
  • Re: [kde] Once again kmails folder creation fails
    ... The problem is that kdepim was stuck at the 4.4.x level for some ... relatively unstable for kmail, and I wouldn't be surprised to see this ... or switching to claws-mail or something else. ... expect that you may have some issues when you upgrade to kde 4.7 ...
    (KDE)
  • Re: [kde] Once again kmails folder creation fails
    ... Kmail help says 1.13.7 ... was a kdepim 4.6.0 and 4.6.1 release, but they came out much later than ... or switching to claws-mail or something else. ... expect that you may have some issues when you upgrade to kde 4.7 ...
    (KDE)