Re: Delayed mails on debian-user [Now mostly fixed]

From: Pascal Hakim (pasc_at_redellipse.net)
Date: 08/19/04

  • Next message: Stefan O'Rear: "Re: Are Debian's 2.6 kernels stable?" 
    Date: Thu, 19 Aug 2004 08:52:53 -0500
To: debian-user@lists.debian.org

    'lo,

    I got some emails asking me what teergrubing was after this post which I
    managed to delete before replying to. Hopefully the people who were
    wondering about it will catch this message.

    On Thu, Aug 19, 2004 at 03:40:34AM +1000, Pascal Hakim wrote:
    > Hi,
    > <snip>
    > Delays were caused on this list by the high number of gmail
    > subscribers, which caused some gmail servers to start teergrubing
    > lists.debian.org.
    > <snip>

    Firstly, that sentence above is incorrect. It should have be: "Delays on
    this list were caused by the high number of gmail subscribers on this
    list, combined with the fairly high volume of mails that comes out of
    it".

    Basically, what's currently happening is that mails getting sent to
    gmail.com are taking a _long_ time to get processed. While we get an
    answer almost straight away on HELO, RCPT TO:, or Mail From, we have to
    wait until we get a reply at the end of the DATA section of the email.
    I've done some tests, and it seems to be taking at least 5 minutes to
    give us an OK code after the message has been sent.

    While this is not a problem if you are only sending a couple of emails,
    when you get to the levels that lists.debian.org has to deal with, it
    starts being a problem. We've setup our MTA so that it can be trying to
    deliver up 270 messages at any point in time. By holding a connection
    open for a long period of time, processes that would normaly be
    delivering mail just sit there waiting for a successfull delivery
    message. Since there weren't enough processes to deliver messages, the
    queues were getting longer and longer causing some people to have
    extreme delays in getting emails from lists. The listmaster team stuck a
    band-aid on the problem by having some processes set aside to be used
    uniquely by gmail. That way the problem now only affects people with an
    address at gmail and no others.

    If we look at some quick numbers, we realise quite quickly that we're
    still having quite a problem. In the last 90 minutes, 350 messages were
    accepted by gmail. Since there are 50 processes dedicated to delivering
    mail there, it's taking on average 12 minutes for a message to an
    individual gmail subscriber to go through.

    We still don't know what's causing the delays when we send mail to
    gmail. There's two main possibilities I can think of. (a) is that gmail
    is doing some content filtering after mail reception but before
    accepting a message. This way it can bounce messages it believes to be
    spam. (b) is that google is holding connections open to what it thinks
    are propbably spammers. This is what teergrubing refers to. The idea
    behind it is that most spammers do not use real SMTP clients, but
    rather, something that mostly works. By holding a connection open for a
    long time, a spammer is prevented from sending more spam out, as it has
    one less process and one less port it can send spam from. As we can see
    from what's happening to us, this is a very effective technique in
    stopping someone from delivering mail.

    There's a fairly interesting article about teergrubing at:
    http://www.iks-jena.de/mitarb/lutz/usenet/teergrube.en.html

    At the moment, I'm guessing that (b) happened. I'm still trying to get
    more information, so it's hard for me to make more than a guess at this
    point however.

            Cheers,

    Pasc 

    -- 
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
  • Next message: Stefan O'Rear: "Re: Are Debian's 2.6 kernels stable?"

    Relevant Pages

    • [ot] Gmail hit by zero-day
      ... Google's Gmail can be easily hacked, allowing any past and future emails to be ... the past two weeks, Petkov has publicly posted information about critical, ... inject a rogue filter into the user's filter list. ...
      (soc.culture.malaysia)
    • Re: Gmail ???
      ... On 12/18/06, Matthew Clarke wrote: ... I deal with so many emails that I've gotten hooked on the ... speed that only keyboard shortcuts can offer. ... but if gmail could use AJAX to do some drag&drop ...
      (Ubuntu)
    • Re: exchange with gmail
      ... ok i will sit down with the boss and we will write out a plan. ... gmail for personal emails. ... now gmail is also being used for work. ... anyways i had exchange setup so the domain email ...
      (microsoft.public.windows.server.sbs)
    • Re: When sending mail from gmail, blank emails are received
      ... I have Symantec SMTP gateway with exchange 2003 as my mail server. ... Problem is the emails came a couple of months ago and now it comes ... What's funny though sometimes we do get gmail mails, ...
      (microsoft.public.exchange.admin)