Re: POP3 mail fetcher that supports unreliable connections?

From: Vincent Lefevre (vincent_at_vinc17.org)
Date: 11/06/03

  • Next message: Andreas Janssen: "Re: Partitioning"
    Date: Thu, 6 Nov 2003 09:28:26 +0100
    To: debian-user@lists.debian.org
    
    

    On 2003-11-04 20:34:18 -0600, Jacob S. wrote:
    > All the Debian installations I've ever done asked me to choose one of
    > four choices for configuring the delivery system. And every fetchmail
    > installation I've done had the option to tell it which user to deliver
    > the mail to, making it nice and straight forward.

    Yes, this may be fine from Debian packages, and perhaps I could use
    fetchmail with SMTP reliably on Debian; I'm a bit doubtful, though:
    for instance, what happens if the local disk is full? I don't want
    a mailer-daemon to be sent to the sender of the message, as it is
    normally done with SMTP; instead, I want an error to be immediately
    reported so that the message isn't deleted from the POP3 server.

    But anyway, I want to use the same tools under different Unix
    distributions (unfortunately, Debian isn't everywhere): for instance,
    I currently use getmail to fetch my mail from POP3 mailboxes, mutt to
    read my mail and so on. If you want to know, when I lost my mail, it
    was under Solaris, but it was never said in the fetchmail documentation
    that fetchmail shouldn't be used under Solaris...

    > hmm... I always thought the Debian way was to test things before you
    > relied on them. I guess I've been doing too much work.

    I've now been warned for several years. However, one can't always
    easily test things (in particular the cases where the local disk is
    full, the cases where the memory is full and the OOM killer has to
    kill some random process, the cases where there are other failures,
    and so on).

    > > Completely wrong! I hope you don't think that for instance, a MUA
    > > should use the local delivery system for copying messages to a
    > > different mailbox!
    >
    > When you're copying mail between mailboxes you're not moving it to a
    > separate computer, downloading it off a server, or anything else.

    When using IMAP mailboxes, messages are downloaded from a server.
    So, with such mailboxes, I don't see the difference.

    > You're simply moving data between files or files between folders
    > (depending on what kind of mailbox you use). That's not a very good
    > example.

    Apart the above point about IMAP, I don't see why the fact that the
    mail comes from a different machine makes a difference. The important
    point IMHO is that the mail has already been delivered to the right
    person. (If you say that a single POP3 mailbox can be used by several
    people, this is not the primary use, and anyway this requires extended
    configuration.)

    > But on the other hand, sometimes you do use multiple programs chained
    > together for moving things around on your own system - that's why the
    > pipe | was invented.

    The pipe can't solve all problems and has its limitations. In particular
    this is more or less a one-way communication (with very limited error
    handling, something very important for data integrity). But I agree that
    for mail copying or for POP3, it can be OK, as long as error checking is
    properly done (is it for current versions of fetchmail? Old versions had
    problems even without any error -- see below). With a protocol with
    immediate delete for individual messages, this wouldn't be OK, as an
    error could be signaled too late.

    > > It is poorly designed as a POP3 mail fetcher tool, as it relies on
    > > special support of the local delivery configuration.
    >
    > I've never heard smtp called "special support" before.

    This requires the SMTP server to be configured to support fetchmail
    (perhaps the case with Debian, but not the case with every OS or
    configuration by system administrators).

    > In another post you claim getmail does it the "proper way". That's the
    > nice thing about Linux and open source; having multiple programs to do
    > the same job, and being able to choose between the two. But I certainly
    > wouldn't say that means fetchmail is defective by any means.

    Well, data/system privacy and data integrity are the most important
    points. Contrary to getmail, fetchmail doesn't ensure data integrity.

    On 2003-11-04 21:41:16 -0500, ScruLoose wrote:
    > Yup, and I think that what Monique was trying to say tactfully really
    > comes down to this: If you're stupid enough to connect fetchmail to a
    > misconfigured MTA, that's your problem.

    How could I know that it were misconfigured? (I *didn't* configure
    it, this was the job of the system administrators.)

    > If a non-privileged user wants to use fetchmail and the MTA is
    > misconfigured and won't play nice, there's always the -m option to
    > skip the MTA and pipe mail straight to procmail (or the MDA of
    > choice).

    It's a bit too late after losing mail. So, after losing mail, I used
    some option to send the messages directly to a file. This worked for
    some time, but after a fetchmail upgrade, fetchmail started to corrupt
    my mailbox by writing things such as "downloading mail..." messages to
    the mailbox instead of the terminal (or stderr), but this is another
    story... Then I switched to getmail and I have never had any lost mail
    with it.

    BTW, I installed fetchmail on another system (SuSE) later, and I again
    had lost mail problems (except that I didn't really lose mail since I
    had the good idea to keep the mail on the server this time): messages
    from one of the POP3 mailboxes weren't delivered (though they had been
    retrieved). So, I immediately switched to getmail, again (it was also
    faster than fetchmail). If you can read French:

    From: Vincent Lefevre <vincent+news@vinc17.org>
    Subject: [fetchmail] mails non recus
    Newsgroups: fr.comp.mail
    Date: 18 Aug 2001 23:21:51 GMT
    Message-ID: <20010818225650$2165@vinc17.org>

    > Lots of things that an unprivileged user will _use_ need root to
    > _configure_ before they work.

    But this shouldn't be the case for POP3 fetching.

    > I think that if you want to convince anyone that Monique, and fetchmail,
    > and its authors, and its many users (and the entire Unix philosophy?)
    > are "completely wrong"; you'll need a more solid argument than this
    > groping for a dubious metaphor.

    It seems that many other people have been convinced. From the getmail
    FAQ:

    Why did you write getmail? Why not just use fetchmail?

       I do not like some of the design choices which were made with
       fetchmail. getmail does things a little differently, and for my
       purposes, better. In addition, most people find getmail easier to
       configure and use than fetchmail. Perhaps most importantly, getmail
       goes to great lengths to ensure that mail is never lost, while
       fetchmail (in its default configuration) frequently loses mail, causes
       mail loops, bounces legitimate messages, and causes many other
       problems.

       In addition, fetchmail has a long history of security problems:
    [snip]

    > Besides which, I can easily see people having mutt use procmail to sort
    > and move mail to a different mailbox. Maybe I want to take advantage of
    > filter rules that weren't there when the mail was delivered the first
    > time...

    Of course, getmail can do that for instance (with a pipe).

    And as you talk about procmail, guess what is the default rule...
    store the mail to the user's mailbox! (Not send the mail to the
    local delivery agent.) So, why should this be different for a POP3
    fetcher?

    > Very much the same philosophy as with fetchmail passing to the MTA.
    > The MTA's job is to transport mail. It's generally good at it.

    No, not the same philosophy. First the MTA's configuration is at system
    level, though I want here a configuration at the user level (of course,
    the user could still choose to pipe the message to exim or whatever if
    he wishes to, but the configuration of what to do with the message is
    entirely his choice). Also a MTA has different kinds of rules, which
    are not suited to a simple copy (see the comments from the getmail FAQ
    for instance).

    > Fetchmail's job is to get mail from a remote mailbox. Why rewrite
    > MTA functionality?

    The goal is *not* to rewrite MTA functionality. The goal is to have
    different functionality (much simpler than a MTA).

    > Why not just hand off to the existing MTA and let it do its job? If
    > the MTA is broken/misconfigured that's hardly fetchmail's fault.

    It isn't necessarily broken or misconfigured; it may be configured
    for things other than mails forwarded by fetchmail. With antispam
    rules, I suppose that this is even more complex. Also, it is
    well-known that SMTP isn't a protocol as reliable as file copy.

    -- 
    Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
    validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
    des Jeux Mathématiques et Logiques, TETRHEX, etc.
    Work: CR INRIA - computer arithmetic / SPACES project at LORIA
    -- 
    To UNSUBSCRIBE, email to debian-user-request@lists.debian.org 
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
    

  • Next message: Andreas Janssen: "Re: Partitioning"

    Relevant Pages

    • Re: POP3 mail fetcher that supports unreliable connections?
      ... On the rare occasion I wasn't sure how to do something in fetchmail, ... > the local delivery. ... Everything I know about Linux is ... > special support of the local delivery configuration. ...
      (Debian-User)
    • Re: [SLE] POP Mail HOWTO?
      ... Mail are fetched from a remote server that handles incoming connections, and delivered locally to an IMAP server (cyrus) ... postfix, cyrus-imapd, fetchmail, fetchmailconf (GUI for configuration of fetchmail, not strictly needed) ... Once the email is in the spool file reception is done. ...
      (SuSE)
    • Re: POP3 mail fetcher that supports unreliable connections?
      ... If a non-privileged user wants to use fetchmail and the MTA is ... > the local delivery. ... > special support of the local delivery configuration. ...
      (Debian-User)
    • Re: fetchmail and spam assassin
      ... > it through spamassassin/an MTA junk mail filter? ... The typical configuration of fetchmail just feeds the mail back into ...
      (freebsd-questions)
    • Re: [SLE] mail trail question
      ... As each message is retrieved fetchmail normally delivers it via SMTP to ... then be delivered locally via your system's MDA (Mail Delivery Agent, ... mmdf, exim, or qmail). ...
      (SuSE)