[Project_owners] Open unicode files with jslib ?

Karsten Düsterloh mnenhy at tprac.de
Sat Nov 8 15:32:53 EST 2003


Erwan Loisant aber hob zu reden an und schrieb:
> I am playing with jslib file.js module, and I want to read an UTF8 file. 
> It contains both Japanese and French, so unicode is mandatory.
> 
> When I simply open this text file in Firebird (just entering location in 
> location bar) it is correctly displayed. However, when I load it using f 
> = new File(); and read it, Japanese character and French special accents 
> are broken. It seems like jslib read it as a western europe encoded file.

Have you tried the scriptable Unicode converter?
(http://lxr.mozilla.org/mozilla/source/intl/uconv/idl/nsIScriptableUConv.idl)

I had the same problem with Mnenhy's junk stats:

const gksCID_UniConv  = "@mozilla.org/intl/scriptableunicodeconverter";
const gksUTF8         = "UTF-8";
  var oUTFConv     =
Components.classes[gksCID_UniConv].createInstance(Components.interfaces.nsIScriptableUnicodeConverter);
  oUTFConv.charset = gksUTF8;
...
    // read token data (add space to avoid problems with keywords)
    iRefCount = is.read32();
    iTokenLen = is.read32();
    var oToken = {sToken: is.readBytes(iTokenLen)+" ",
                  iGood:  0,
                  iEvil:  0,
                  iProb:  0};
    // most tokens are in UTF-8 (but not all, so ConvertToUnicode may
vomit :( )
    try{
      oToken.sToken = oUTFConv.ConvertToUnicode(oToken.sToken);
    }catch(ignore){}
...

(Where 'is' is a binary input stream)


HTH,

Karsten



More information about the Project_owners mailing list