[Greasemonkey] Javascript character encoding problem

[Lenny Domnitser]

On 4/7/06, Jonas Lundberg <my.name.is.hans at gmail.com> wrote:
> The problem is the following:
> I read some contents (including the html for the form)  from my own
> server for web2-services. I then inject that into any web page I
> visit, through the greasemonkey script. The form contents are  sent
> back to my web-2 server.

So let's say you're on a page served as iso-8859-1. You pull utf-8
data from your server via GM_xmlhttpRequest. You insert the data into
the DOM. Then I'm not sure what is happening. The user modifies the
data, and you take it out of the DOM. You send it back to your server,
and it turns out to be iso-8859-1.

> Also, different pages will have different encodings, and this should
> not affect my form.
> So, it is good that the text encoding in the Ajax call does not vary.
> But it is BAD that XMLHttprequest does not use utf-8.

XMLHttpRequest does look at encoding of data it pulls in, and no
matter what encoding it is it will be converted to Unicode, because
that's what the Javascript string type is. I'm not sure when this
happens, but the unicode string has to be converted to bytes of a
particular charset before it is sent over the wire, and it seems that
it takes the charset from the page.

If the page charset includes the international characters you need,
then you can specify

headers: { 'Content-type': 'multipart/form-data; boundary=shapecms;
charset=' + document.characterSet }

and assuming that your web service looks at Content-Type, it will know
to decode from the right charset.

At least I think it should work that way.

If the page charset simply cannot encode the characters you need, then
I have no idea what could work, besides the form. If this is the case,
then maybe put the form in a hidden iframe and submit that.