How Gmail Works

What the Devil Is Going On?

Before revealing just what’s happening, let’s recap. The DOM inspector inside Firefox will help you dissect the HTML, which will help you again. So, as before, open up Gmail in Firefox, and open the DOM inspector. You already know that the main document is made of two frames, the first made of many subframes and the second one with nothing but a huge chunk of JavaScript. Using the DOM inspector’s right-click menu Copy as XML function, you can grab the script’s text and copy it to a text editor. Ordinarily, I would include this code as a listing right here, but when I cut and pasted it into the manuscript of this book, it added another 120 pages in a single keystroke. This does not bode well, especially as Google has tried as hard as it can to format the JavaScript as tightly as possible. This saves bandwidth but doesn’t help anyone else read what Google is doing. We’ll reach that problem in a page or two.

Preloading the Interface

What is happening is this: Gmail loads its entire interface into one single HTML page. So when you move around the application, you’re not loading new pages but triggering the JavaScript to show you other parts of the page you already have in your browser’s memory. This is why it is so fast: There’s no network connection needed to bring up the Compose window or show the Settings page, as you’ve already loaded it. You can see this inside the DOM inspector. Further, inspection shows that d_comp holds the Compose window, and d_prefs hold the Settings window, and so on. This is very interesting, but it doesn’t show how the application works. If anything, it asks a difficult question: if the page never refreshes, how does it send or receive any messages? The answer to this is in JavaScript, and the use of one very clever function, XMLHttpRequest.

Introducing XMLHttpRequest

I like to think of this as a pretty romantic story. JavaScript, you see, has had a bad rap over the years: it’s commonly misconceived as a scrappy language for dodgy website effects circa 1999 and up there with the tag as something to be avoided by the genuinely righteous web developer. This is, of course, utter rot: Modern JavaScript is a rich and powerful language and is rapidly regaining momentum. Perhaps since IE5 was launched, and indeed since Mozilla and Safari became mainstream, the majority of browsers have been capable of doing some very clever things in JavaScript.

One such function is XMLHttpRequest. Invented by Microsoft and now universally implemented, it allows a JavaScript program to communicate with a server in the background without refreshing the page. This is very key for Gmail. It means that the JavaScript code can, upon a button push or any other trigger, send a tiny request to the Gmail server, parse the response, and throw it onto the screen, entirely without refreshing the page or causing any more traffic than is necessary. It’s blazingly fast, especially if you have a server optimized for just such a thing. Google, naturally, does.

Using XMLHttpRequest Yourself

To get an idea of just what is going on, it’s a good idea to use XMLHttpRequest yourself. In this section, you’ll use it to create a little application of your own. Of course, you can skip this section if you’re not interested in a deep understanding, but it’s pretty cool stuff to play with anyway.

First, open up a directory on a website. You’ll need to access it via a proper domain, you see. Create the guide, and make sure your browser can see it. In that directory, place a text file, called Listing.txt, and put the exclamation “Horrible!” inside the file. Bear with me.

Finding XMLHttpRequest within the Gmail code

Don’t take the presence of XMLHttpRequest within Gmail on trust. You can see this in action in Gmail’s code. Go back to the DOM inspector and open the second frameset — the one with all of the JavaScript in it. Please copy the entire script into a text editor and save it, as you will refer to it a lot in this section. Once you’ve done that, search for the string xmlhttp.

Sniffing the Network Traffic

So now that you understand how XMLHttpRequest works, you’re led to some further questions: What is being sent and received using the XMLHttpRequest functions, and what are the URLs? Once you know the answers to these questions, you can write your code to spoof these requests and then interface directly with the Gmail system. The rest of the book relies on this idea. To find out what Gmail is saying to the browser, use a new tool: the packet sniffer. This is a generic term for a range of applications that can listen to raw network traffic, display it on the screen, log in, analyze it, etc. You’re interested in watching what your browser is doing in the background: what it is sending, where it is sending it to, and then the replies it is getting.

Firing up TCP flow

Install TCP flow, and set it running inside a terminal window, monitoring port 80. On my machine, that means typing the following:

sudo cp flow -c port 80

As you can see from the figure and your screen, Tcpflow captures all of the traffic flowing backward and forward across Port 80 — all your web traffic, in other words. It shows the requests and the answers: headers, content, and all. So Tcpflow is perfect for the job. But there’s a snag. Open up Gmail, and let it sit there for a while.


You have seen this sort of URL before: Look back again at Listing A-3, after the second excised block of encrypted code. You can also guess that something happens to the cookie you set on the first page — it is being checked for something. Considering that those cookies do not contain anything but the time they were set, I am guessing that this step ensures that the connection is current and not the result of caching from someone’s browser. Instead, it’s to ensure a good, fresh session with Gmail on the browser application and the user himself. Or so I would guess.


Check Also

Football Betting Terminology Rookies Must Know

Soccer betting terminology is one of the important factors that helps players grasp the match …

Leave a Reply

Your email address will not be published. Required fields are marked *