Apache sometimes sends an HTTP header “X-Pad: avoid browser bug” for reasons that aren’t entirely clear. There isn’t a lot of searchable information about this header, so my coworker Matt dug into the source code and found out what’s going on.
The header was added to deal with a bug Netscape versions 2 through 4.0b2. The bug was that Netscape would hang if a linefeed character was sent as the 256 or 257 byte. Apache checks to see if it’s about to hit the bug and, if so, sends this header to pad the output.
The fix is from April of 1997. If they had waited a few months to report it, they might have earned $1,000 with Netscape’s Bug Bounty.
Hopefully this blog post will make the X-Pad header show up in Google and save future people from wondering what it is.
The code change that provided the fix can be viewed on Apache’s site. I’ve reproduced the comments from the code below for anyone curious.
Navigator versions 2.x, 3.x and 4.0 betas up to and including 4.0b2 have a header parsing bug. If the terminating rn occur starting at the 256th or 257th byte of output then it will not properly parse the headers. Curiously it doesn’t exhibit this problem at 512, 513. We are guessing that this is because their initial read of a new request uses a 256 byte buffer, and subsequent reads use a
larger buffer. So the problem might exist at different offsets as well.
This should also work on keepalive connections assuming they use the same small buffer for the first read of each new request.
At any rate, we check the bytes written so far and, if we are about to tickle the bug, we instead insert a bogus padding header. Since the bug manifests as a broken image in Navigator, users blame the
server. 🙁 It is more expensive to check the User-Agent than it is to just add the bytes, so we haven’t used the BrowserMatch feature here.