Re: Suggestions for custom application-layer protocol?

From: James Antill (james-netnews_at_and.org)
Date: 05/27/05 

Date: Fri, 27 May 2005 13:04:09 -0400

On Fri, 27 May 2005 14:59:49 +0000, Grant Edwards wrote:

> On 2005-05-27, James Antill <james-netnews@and.org> wrote:
>
>>> If you're using a text-based protocol, you don't nead a header
>>> with a payload count to tell you where the message ends. Use
>>> an end-of-message character or string. The cr/lf pair is used
>>> as the end-of-message delimiter by a lot of text-based
>>> runs-on-top of TCP protocols. That approach works wonderfully
>>> with line-oriented high-level text-handling libraries and
>>> utilties -- you can do all sorts of useful stuff using nothing
>>> but a few utilities like netcat, expect, and so on.
>>
>> It interacts well with telnet,
>
> If you handle IAC seqeunces -- at least well enough to ignore
> them.

 I don't know of any HTTP, SMTP, NNTP, POP3, IMAP, etc. server that does
anything with IAC sequences. Generally servers that care initiate by
sending IAC commands EOR, NAWS and TTYPE etc to the client.

>> which is about the best thing that can be said for it.
>
> And it's a pretty big thing, in my experience.

 It's "big" because it's there already so noone has to write a
simple client to test a service with. If SMTP had been written
using netstrings, then a telnetstr command would be available
everywhere and this "big" thing would be worth nothing.
 A similar thing is happening with HTTP, it's not "good"[1] but there are
a significant number of tools that understand a HTTP stream ... so people
hack it into places it shouldn't be so they can leverage those tools.

>> There are large problems with how you limit "too long" lines,
>
> What problems?

 Say I connect to an SMTP service and keep sending 'a', the other end has
to keep accepting data and parsing it upto the amount it limits itself to
accepting for a single line. With something like a netstring the remote
end can decide within 10 characters if it's going to just drop the
connection/message or handle it.

>> and what happens when you hit a CR or LF on it's own.
>
> Yes, that can be a problem if you have to be able to include arbitrary
> strings in the message body. I was under the impression that this
> wasn't the case for the OP's application. I could be wrong.

 It isn't just a problem of arbitrary data, but of different
clients/servers parsing the same data in different ways. This is
obviously a bigger problem the more implementations of clients and servers
you have, and how compatible you want to be ... but even with just one
client and one server it wouldn't be unique to have silent bugs where
someone typed \n\r or r\\n or just \n instead of \r\n at one point in the
code.
 At which point a third application implementing the protocol, or a
supposedly compatible change to either the server or client can bring out
bugs (often in the edge cases) ... something like netstrings is much
less likely to have this kind of problem.

[1] http://www.and.org/texts/server-http 

-- 
James Antill -- james@and.org
http://www.and.org/vstr/httpd

Relevant Pages

  • Re: breaking the model
    ... > The forms data then is in the Request object. ... HTTP Request; in this case, the form POST Request from the Page. ... client and server. ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: TLS/SSL in Outlook 2003 and Exchange 2003
    ... require TLS on my default virtual smtp server, running on port 25 and ALSO ... SMTP traffic from a client to the server. ... More specifically I want to make sure that the authentication part is ...
    (microsoft.public.exchange.admin)
  • Re: Banana Republic (was Re: OpenVMS Book Wins award)
    ... client ... No bollocks HTTP, SOAP, XML, Java, Garbage ... receive messages from any number of server processes who in turn could be ... Unlike TCP/IP and/or UDP Sockets with Java that have been around since ...
    (comp.os.vms)
  • Re: MDX Sample App broken by 2005
    ... You might find it useful as an alternative to http ... Windows authentication wouldn't work since client and server are ... > different non-trusted domains. ...
    (microsoft.public.sqlserver.olap)
  • Re: MSAS Licensing Part II
    ... wish to use http access then you must have Enterprise Edition. ... PTS looks at the server name. ... You will note that all of this is totally transparent to the client. ...
    (microsoft.public.sqlserver.olap)