Unicode strings: Difference between revisions
m If there is a "Task" heading like that then it ends up on the TOC eventually and looks weird |
{{header|ZX Spectrum Basic}} |
||
Line 23: | Line 23: | ||
PicoLisp can directly handle _only_ Unicode (UTF-8) strings. So the problem is rather how to handle non-Unicode strings: They must be pre- or post-processed by external tools, typically with pipes during I/O. For example, to read a line from a file in 8859 encoding: |
PicoLisp can directly handle _only_ Unicode (UTF-8) strings. So the problem is rather how to handle non-Unicode strings: They must be pre- or post-processed by external tools, typically with pipes during I/O. For example, to read a line from a file in 8859 encoding: |
||
<lang PicoLisp>(in '(iconv "-f" "ISO-8859-15" "file.txt") (line))</lang> |
<lang PicoLisp>(in '(iconv "-f" "ISO-8859-15" "file.txt") (line))</lang> |
||
=={{header|ZX Spectrum Basic}}== |
|||
The ZX Spectrum does not have native Unicode support. However, it does support user defined graphics, which makes it is possible to create custom characters in the UDG area. However, there is only 48k of memory available on a traditional rubber key ZX Spectrum, (or 128k on some of the plus versions), and 510k would be needed to store the Unicode characters for display, so Unicode is not really viable on this platform. |
Revision as of 17:57, 30 June 2011
Demonstrate how one is expected to handle Unicode strings. Some example considerations: can a Unicode string be directly written in the source code? How does one do IO with unicode strings? Can these strings be manipulated easily? Can non-ASCII characters be used for keywords/identifiers/etc? What encodings (UTF-8, UTF-16, etc) can your language accept without much trouble?
Perl
In Perl, "Unicode" means "UTF-8". If you want to include utf8 characters in your source file, unless you have set PERL_UNICODE
environment correctly, you should do<lang Perl>use utf8;</lang> or you rick the parser treating the file as raw bytes.
Inside the script, utf8 characters can be used both as identifiers and literal strings, and built-in string functions will respect it:<lang Perl>$四十二 = "voilà"; print "$四十二"; # voilà print uc($四十二); # VOILÀ</lang> or you can specify unicode characters by name or ordinal:<lang Perl>use charnames qw(greek); $x = "\N{sigma} \U\N{sigma}"; $y = "\x{2708}"; print scalar reverse("$x $y"); # ✈ Σ σ</lang>
Regular expressions also have support for unicode based on properties, for example, finding characters that's normally written from right to left:<lang Perl>print "Say עִבְרִית" =~ /(\p{BidiClass:R})/g; # עברית</lang>
When it comes to IO, one should specify whether a file is to be opened in utf8 or raw byte mode:<lang Perl>open IN, "<:utf8", "file_utf";
open OUT, ">:raw", "file_byte";</lang>
The default of IO behavior can also be set in PERL_UNICODE
.
However, when your program interacts with the environment, you may still run into tricky spots if you have imcompatible locale settings or your OS is not using unicode; that's not what Perl has control over, unfortunately.
PicoLisp
PicoLisp can directly handle _only_ Unicode (UTF-8) strings. So the problem is rather how to handle non-Unicode strings: They must be pre- or post-processed by external tools, typically with pipes during I/O. For example, to read a line from a file in 8859 encoding: <lang PicoLisp>(in '(iconv "-f" "ISO-8859-15" "file.txt") (line))</lang>
ZX Spectrum Basic
The ZX Spectrum does not have native Unicode support. However, it does support user defined graphics, which makes it is possible to create custom characters in the UDG area. However, there is only 48k of memory available on a traditional rubber key ZX Spectrum, (or 128k on some of the plus versions), and 510k would be needed to store the Unicode characters for display, so Unicode is not really viable on this platform.