binary protocols v. text protocols

does anyone have a good definition for what a binary protocol is? and what is a text protocol actually? how do these compare to each other in terms of bits sent on the wire?

here's what wikipedia says about binary protocols:

A binary protocol is a protocol which is intended or expected to be read by a machine rather than a human being (http://en.wikipedia.org/wiki/Binary_protocol)

oh come on!

to be more clear, if I have jpg file how would that be sent through a binary protocol and how through a text one? in terms of bits/bytes sent on the wire of course.

at the end of the day if you look at a string it is itself an array of bytes so the distinction between the 2 protocols should rest on what actual data is being sent on the wire. in other words, on how the initial data (jpg file) is encoded before being sent.

Binary protocol versus text protocol isn't really about how binary blobs are encoded. The difference is really whether the protocol is oriented around data structures or around text strings. Let me give an example: HTTP. HTTP is a text protocol, even though when it sends a jpeg image, it just sends the raw bytes, not a text encoding of them.

But what makes HTTP a text protocol is that the exchange to get the jpg looks like this:

Request:

GET /files/image.jpg HTTP/1.0
Connection: Keep-Alive
User-Agent: Mozilla/4.01 [en] (Win95; I)
Host: hal.etc.com.au
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Accept-Language: en
Accept-Charset: iso-8859-1,*,utf-8

Response:

HTTP/1.1 200 OK
Date: Mon, 19 Jan 1998 03:52:51 GMT
Server: Apache/1.2.4
Last-Modified: Wed, 08 Oct 1997 04:15:24 GMT
ETag: "61a85-17c3-343b08dc"
Content-Length: 60830
Accept-Ranges: bytes
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: image/jpeg

<binary data goes here>

Note that this could very easily have been packed much more tightly into a structure that would look (in C) something like

Request:

struct request {
  int requestType;
  int protocolVersion;
  char path[1024];
  char user_agent[1024];
  char host[1024];
  long int accept_bitmask;
  long int language_bitmask;
  long int charset_bitmask;
};

Response:

struct response {
  int responseType;
  int protocolVersion;
  time_t date;
  char host[1024];
  time_t modification_date;
  char etag[1024];
  size_t content_length;
  int keepalive_timeout;
  int keepalive_max;
  int connection_type;
  char content_type[1024];
  char data[];
};

Where the field names would not have to be transmitted at all, and where, for example, the responseType in the response structure is an int with the value 200 instead of three characters '2' '0' '0'. That's what a text based protocol is: one that is designed to be communicated as a flat stream of (usually human-readable) lines of text, rather than as structured data of many different types.

Here's a kind-of cop-out definition:

> You'll know it when you see it.

This is one of those cases where it is very hard to find a concise definition that covers all corner cases. But it is also one of those cases where the corner cases are completely irrelevant, because they simply do not occur in real life.

Pretty much all protocols that you will encounter in real life will either look like this:

> fg,m4wr76389b zhjsfg gsidf7t5e89wriuotu nbsdfgizs89567sfghlkf
>  b9er t8ß03q+459tw4t3490ß´5´3w459t srt üßodfasdfäasefsadfaüdfzjhzuk78987342
< mvclkdsfu93q45324äö53q4lötüpq34tasä#etr0 awe+s byf eart

[Imagine a ton of other non-printable crap there. One of the challenges in conveying the difference between text and binary is that you have to do the conveying in text :-)]

Or like this:

< HELLO server.example.com
> HELLO client.example.com
< GO
> GETFILE /foo.jpg
< Length: 3726
< Type: image/jpeg
< READY?
> GO
< ... server sends 3726 bytes of binary data ...
> ACK
> BYE

[I just made this up on the spot.]

There's simply not that much ambiguity there.

Another definition that I have sometimes heard is

> a text protocol is one that you can debug using telnet

Maybe I am showing my nerdiness here, but I have actually written and read e-mails via SMTP and POP3, read usenet articles via NNTP and viewed web pages via HTTP using telnet, for no other reason than to see whether it would actually work.

Actually, while writing this, I kinda caught the fever again:

bash-4.0$ telnet smtp.googlemail.com 25
Trying 74.125.77.16...
Connected to googlemail-smtp.l.google.com.
Escape character is '^]'.
< 220 googlemail-smtp.l.google.com ESMTP Thu, 15 Apr 2010 19:19:39 +0200
> HELO
< 501 Syntactically invalid HELO argument(s)
> HELO client.example.com
< 250 googlemail-smtp.l.google.com Hello client.example.com [666.666.666.666]
> RCPT TO:Me <Me@Example.Com>
< 503 sender not yet given
> SENDER:Me <Me@Example.Com>
< 500 unrecognized command
> RCPT FROM:Me <Me@Example.Com>
< 500 unrecognized command
> FROM:Me <Me@Example.Com>
< 500-unrecognized command
> HELP
< 214-Commands supported:
< 214 AUTH HELO EHLO MAIL RCPT DATA NOOP QUIT RSET HELP ETRN
> MAIL FROM:Me <Me@Example.Com>
< 250 OK
> RCPT TO:You <[email protected]>
< 250 Accepted
> DATA
< 354 Enter message, ending with "." on a line by itself
> From: Me <Me@Example.Com>
> To: You <[email protected]>
> Subject: Testmail
>
> This is a test.
> .
< 250 OK id=1O2Sjq-0000c4-Qv
> QUIT
< 221 googlemail-smtp.l.google.com closing connection
Connection closed by foreign host.

Damn, it's been quite a while since I've done this. Quite a few errors in there :-)

Examples of binary protocols: RTP, TCP, IP.

Examples of text protocols: SMTP, HTTP, SIP.

This should allow you to generalise to a reasonable definition of binary vs text protocols.

Hint: just skip to the example sections, or the diagrams. They serve to illustrate Tyler's rocking answer.

As most of you suggested we can't differentiate whether the protocol is Binary or text simply by looking at the content on the wire

AFIK

Binary protocol - Bits are boundary Order is very critical

Eg., RTP

First two bits are version Next bit is MarkUp bit

Text protocol - Delimiters specific to protocol Order of the fields is not important

Eg., SIP

One more is, in binary protocol, we can split a byte, i.e., a single bit might have a specific individual meaning; While in a text protocol minimum meaningful unit is BYTE. You can't split a byte.

Both uses different char set, the text one, use a reduced char set, the binary includes all it can, not only "letters" and "numbers", (that's why wikipedia says "human being")

>o be more clear, if I have jpg file how would that be sent through a binary protocol and how >through a text one? in terms of bits/bytes sent on the wire of course.

you should read this Base64

>any coments are apprecited, I am trying to get to the essence of things here.

I think the essence for narrowing the charset, is narrowing the complexity, and reach portability, compatibility. It's harder to arrange and agree with many to respect a Wide charset, (or a wide whatever). The Latin/Roman alphabet and the Arabic numerals are worldwide known. (There are of course other considerations to reduce the code, but that's a main one)

Let say in binary protocols the "contract" between the parts is about bits, first bit mean this, second that, etc.. or even bytes (but with the freedom of use the charset without thinking in portability) for example in privated closed system or (near hardware standars), however if you design a open system you have to take account how your codes will be represented in a wide set of situations, for example how it will be represented in a machine at other side of world?, so here comes the text protocols where the contract will be as standar as posible. I have designed both and that were the reasons, binary for very custom solutions and text for open or/and portable systems.

How can we send an image file in SOAP: Click here

This shows that binary data is attached as such [ATTACHMENT] and its reference is saved in SOAP message.

So, The protocol is text based and data[Image] is binary attachment whose encoding is not relevant

Thus, SOAP is text protocol due to the way we specify Soap headers and not actual data encoded in it.

If just looking at the concept can not get the target, let me give a practical example. When it comes to data exchange, structured data, unstructured data, and semi-structured data are always mentioned. The binary data you mentioned is actually unstructured data, and the data cannot describe itself; text data is typically JSON format data, also called semi-structured data, which can describe itself to a certain extent, at least know the data name and value.

The Web field usually uses JSON format to exchange data. GSON and FastJson can convert JSON formatted text into Java data objects. The Internet of Things field usually uses binary format. If you adopt a custom protocol binary, you can consider using FastProto (https://github.com/indunet/fastproto) to decode and encode.

I think you got it wrong. It's not the protocol that determines how data looks on the "wire", but it's the data type that determine which protocol to use to transmit it. Take tcp socket for instance, a jpeg file will be sent and received with a binary protocol 'cause it's binary data (not human readable, bytes that go among the 32-126 ascii range), but you can send / recv a text file with both protocols and you wouldn't notice the difference.

Text protocol can be self-explanatory and extensive. It's self-explanatory because the message includes the field names just in the message itself. You cannot understand which value means in the message of binary protocol if you don't refer to the protocol specification.

It's extensive means HTTP as a text protocol just make simple rules but you can extend the data structure by freely adding new headers or by changing the content type to transport different payloads. And the headers are the meta data and have the capability of negotiation and automatically adaption.