[nem-en] Source files encoding auto detection and coding tag

[Snaury]

On 7/15/06, Kamil Skalski <kamil.skalski at gmail.com> wrote:
> Well, from the beginning we chose to only allow utf-8 files, as it is
> the best format to handle multi-language file encodings. Any other
> approach encourages people to use old formats, which causes problems.
> Your patch brings much complexity to the lexing process, I'm also a
> little bit concerned about performance.

Ah, ok, that's fair. I just didn't know that and thought of it as a
lack of feature. :)

> Maybe forcing people to utf-8 is not the most friendly thing we could
> do... but IMHO this is the best we can do to spread this standard and
> it has the increasing value of interoperability between operating
> systems. What if you saved the sources on Windows using cp-1251(or
> something similar) and then tried to read it in editor on Linux? I
> doubt you can easily find one showing the language marks correctly.

Actually, that's pretty simple and doesn't pose much problems. Both
vim and emacs allow recoding of files so opening file in some
particular encoding is not hard at all. :)

> Maybe we should just add the command like switch to specify encoding?

If the former decision is to educate people and promote utf-8 then
there's really no need for such an option...

> BTW. I use XEmacs/Emacs and it handles utf-8, though I had to play
> with it a little bit (installed xemacs-mule, specified some special
> settings, etc.)

Since I discovered emacs I also found that I can seem to solve all
problems related to encodings and national input (something I
surprisingly couldn't do earlier, didn't pay enough time for that). :)
I just used FAR Manager for pretty much everything before recently,
and sadly its internal editor doesn't understand unicode (yet)...