Skip to content

Supplying :|utf-16be| external encoding for OPEN causes stack overflow, but not equivalent :UCS-2BE

Supplying :|utf-16be|, :|utf-16le|, :|utf-32be|, or :|utf-32le| external encoding for OPEN causes stack overflow, but not equivalent :UCS-2BE, :UCS-2LE, :UCS-4BE, or :UCS-4LE.

Note that external format :UTF-8 does not cause an error.

Here is the expected behavior, shown by using :UCS-2BE, etc. rectangular.csv-utf-foo are files which are encoded with different Unicode formats.

> (open "rectangular.csv-utf-8" :external-format :utf-8)
#<input stream "rectangular.csv-utf-8">
> (open "rectangular.csv-utf-16be" :external-format :ucs-2be)
#<input stream "rectangular.csv-utf-16be">
> (open "rectangular.csv-utf-16le" :external-format :ucs-2le)
#<input stream "rectangular.csv-utf-16le">
> (open "rectangular.csv-utf-32be" :external-format :ucs-4be)
#<input stream "rectangular.csv-utf-32be">
> (open "rectangular.csv-utf-32le" :external-format :ucs-4le)
#<input stream "rectangular.csv-utf-32le">
> 

Here is the incorrect behavior, shown by using :|utf-16be| etc.

robert@freekbox:~/maxima/maxima-code/share/numericalio$ ecl
ECL (Embeddable Common-Lisp) 16.1.3 (git:UNKNOWN)
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
Copyright (C) 2016 Daniel Kochmanski
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.  
Top level in: #<process TOP-LEVEL>.
> (open "rectangular.csv-utf-16be" :external-format :|utf-16be|)

Condition of type: STACK-OVERFLOW
C-STACK overflow at size 524288. Stack can probably be resized.
Proceed with caution.

Available restarts:

1. (CONTINUE) Extend stack size
2. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: (EXT:OPTIONAL-ANNOTATION 'EXT::UTF-16BE 'EXT:LOCATION '(DEFPARAMETER EXT::UTF-16BE) '(#P"SYS:ENCODINGS;UTF-16BE.NEWEST" . 0))] In: #<process TOP-LEVEL>.
 File: #P"SYS:ENCODINGS;UTF-16BE.NEWEST" (Position #0)
>> 
robert@freekbox:~/maxima/maxima-code/share/numericalio$ ecl
ECL (Embeddable Common-Lisp) 16.1.3 (git:UNKNOWN)
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
Copyright (C) 2016 Daniel Kochmanski
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.  
Top level in: #<process TOP-LEVEL>.
> (open "rectangular.csv-utf-16le" :external-format :|utf-16le|)

Condition of type: STACK-OVERFLOW
C-STACK overflow at size 524288. Stack can probably be resized.
Proceed with caution.

Available restarts:

1. (CONTINUE) Extend stack size
2. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: (EXT:OPTIONAL-ANNOTATION 'EXT::UTF-16LE 'EXT:LOCATION '(DEFPARAMETER EXT::UTF-16LE) '(#P"SYS:ENCODINGS;UTF-16LE.NEWEST" . 0))] In: #<process TOP-LEVEL>.
 File: #P"SYS:ENCODINGS;UTF-16LE.NEWEST" (Position #0)
>> 
robert@freekbox:~/maxima/maxima-code/share/numericalio$ ecl
ECL (Embeddable Common-Lisp) 16.1.3 (git:UNKNOWN)
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
Copyright (C) 2016 Daniel Kochmanski
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.  
Top level in: #<process TOP-LEVEL>.
> (open "rectangular.csv-utf-32be" :external-format :|utf-32be|)

Condition of type: STACK-OVERFLOW
C-STACK overflow at size 524288. Stack can probably be resized.
Proceed with caution.

Available restarts:

1. (CONTINUE) Extend stack size
2. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: (EXT:OPTIONAL-ANNOTATION 'EXT::UTF-32BE 'EXT:LOCATION '(DEFPARAMETER EXT::UTF-32BE) '(#P"SYS:ENCODINGS;UTF-32BE.NEWEST" . 0))] In: #<process TOP-LEVEL>.
 File: #P"SYS:ENCODINGS;UTF-32BE.NEWEST" (Position #0)
>> 
robert@freekbox:~/maxima/maxima-code/share/numericalio$ ecl
ECL (Embeddable Common-Lisp) 16.1.3 (git:UNKNOWN)
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
Copyright (C) 2016 Daniel Kochmanski
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.  
Top level in: #<process TOP-LEVEL>.
> (open "rectangular.csv-utf-32le" :external-format :|utf-32le|)

Condition of type: STACK-OVERFLOW
C-STACK overflow at size 524288. Stack can probably be resized.
Proceed with caution.

Available restarts:

1. (CONTINUE) Extend stack size
2. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: (EXT:OPTIONAL-ANNOTATION 'EXT::UTF-32LE 'EXT:LOCATION '(DEFPARAMETER EXT::UTF-32LE) '(#P"SYS:ENCODINGS;UTF-32LE.NEWEST" . 0))] In: #<process TOP-LEVEL>.
 File: #P"SYS:ENCODINGS;UTF-32LE.NEWEST" (Position #0)
>> 

uname -a reports: Linux freekbox 3.13.0-44-generic #73-Ubuntu SMP Tue Dec 16 00:23:46 UTC 2014 i686 i686 i686 GNU/Linux

ECL version info:

> (lisp-implementation-version)
"16.1.3"
> (ext:lisp-implementation-vcs-id)`
"UNKNOWN"
> (software-type)
(SOFTWARE-TYPE)
> (software-version)
"3.13.0-44-generic"
> (machine-type)
"i686"
> *features*
(:ECL-BYTECMP :WALKER :CDR-1 :CDR-5 :LINUX :FORMATTER :CDR-7 :ECL-WEAK-HASH
 :LITTLE-ENDIAN :ECL-READ-WRITE-LOCK :LONG-LONG :UINT64-T :UINT32-T :UINT16-T
 :RELATIVE-PACKAGE-NAMES :LONG-FLOAT :UNICODE :DFFI :CLOS-STREAMS :CMU-FORMAT
 :UNIX :ECL-PDE :DLOPEN :CLOS :THREADS :BOEHM-GC :ANSI-CL :COMMON-LISP
 :IEEE-FLOATING-POINT :CDR-14 :PREFIXED-API :FFI :I686 :COMMON :ECL)
>