[Project_owners] XMLHttpRequest characterset weirdness

Sudhakar Chandra thaths at openscroll.org
Tue Oct 5 13:02:43 EDT 2004


Hi,

I am the creator of the News Aggregator for Mozilla called Habari 
Xenu[habarixenu.mozdev.org].  My application downloads RSS/RDF/Atom news 
feed using XMLHttpRequest.  I've noticed a certain oddness with some of 
the content downloaded this way. I'd appreciate any help in fixing this 
problem.

When I download the same content with wget, I see extra characters in 
the downloaded file which are missing from the one downloaded through 
XMLHttpRequest.  The problem is that I am unable to parse the locally 
downloaded and stored copy because it contains xml parsing errors. When 
I open the locally downloaded and stored copy in the browser it 
complains of unparseable XML.

Here is a short example of the difference:

feed downloaded with wget
-------------------------
..overblown claims about copyrightâ~@~Ys power to protect Canadian

feed downloaded through XMLHttpRequest
--------------------------------------
...overblown claims about copyright^Ys power to protect Canadian

I am suspecting that this is a problem with either XMLHttpRequest or 
(less likely) JSLib's File object.

Here is the snippet of code that I use to download the feed and store a 
copy in a local file.

     xmlhttp = new XMLHttpRequest();

     // On the following events run the named functions
     xmlhttp.onload = xmlhttpLoaded;
...
     // Initiate the request
     try {
         xmlhttp.open("GET", url, true); // NOTE: Calling synchronously
         xmlhttp.setRequestHeader("User-Agent", USER_AGENT);
	// Even tried test/xml
         xmlhttp.overrideMimeType("application/xml");
     } catch (e) {
         dump(e);
         xmlhttpGetResult(RESULT_ERROR_FAILURE);
     }

     try {
         xmlhttp.send(null);
     } catch (e) {
         dump(e);
         xmlhttpGetResult(RESULT_ERROR_FAILURE);
     }

...

function xmlhttpLoaded(e) {
     var localFile;
     localFile = new File(localPath);

     if(localFile.open('w') == 0) {
         localFile.write("" + xmlhttp.responseText);
     }
     localFile.close();
     xmlhttpGetResult(RESULT_OK);
}

S.
-- 
Grandpa: "I'm an old man, no one listens to me."
Lisa:    "I'm a young girl, no one listens to me."
Homer:   "I'm a white male aged 18 to 49, everyone listens to me, no
          matter how dumb my suggestions are."
Slacker Without Borders                        http://openscroll.org/
Key fingerprint = 8A 84 2E 67 10 9A 64 03  24 38 B6 AB 1B 6E 8C E4


More information about the Project_owners mailing list