Perl's JSON::XS not encoding UTF8 correctly? -

this simple code segment shows issue having json::xs encoding in perl:

#!/usr/bin/perl use strict; use warnings; use json::xs;  use utf8; binmode stdout, ":encoding(utf8)";  (%data);  $data{code} = "gewürztraminer"; print "data{code} = " . $data{code} . "\n";  $json_text = encode_json \%data; print $json_text . "\n"; 

the output yields is:

johnnyb@boogie:~/projects/repos > ./jsontest.pl  data{code} = gewürztraminer {"code":"gewürztraminer"} 

now if comment out binmode line above get:

johnnyb@boogie:~/projects/repos > ./jsontest.pl  data{code} = gew�rztraminer {"code":"gewürztraminer"} 

what happening here? note trying fix behavior in perl cgi script in binmode can not used "ü" characters above returned in json stream. how debug this? missing?

encode_json (short json::xs->new->utf8->encode) encodes using utf-8, re-encoding printing stdout you've added encoding layer. effectively, doing encode_utf8(encode_utf8($uncoded_json)).

solution 1

use open ':std', ':encoding(utf8)';  # defaults binmode stdout;                      # override defaults print encode_json(\%data); 

solution 2

use open ':std', ':encoding(utf8)';    # defaults print json::xs->new->encode(\%data);   # or to_json json.pm 

solution 3

the following works encoding on stdout using \u escapes non-ascii:

print json::xs->new->ascii->encode(\%data); 

---

in comments, mention it's cgi script.

#!/usr/bin/perl use strict; use warnings;  use utf8;                      # encoding of source code. use open ':encoding(utf-8)';   # default encoding of file handles. begin {    binmode stdin;                       # nothing on non-windows.    binmode stdout;                      # nothing on non-windows.    binmode stderr, ':encoding(utf-8)';  # text sent log file. }  use cgi      qw( -utf8 ); use json::xs qw( );   {    $cgi = cgi->new();    $data = { code => "gewürztraminer" };    print $cgi->header('application/json');    print encode_json($data); }