The End Of The Line
The End Of The Line
I have been writing terminal / TUI / text windowing type projects for almost 20 years. I started with colorized log outputs, then wrote a curses-based terminal emulator with many bells and whistles, then a mouse-supporting text windowing system, and finally did something truly new: I added sixel image support to that text windowing system. The story with pictures: https://jexer.sourceforge.io/evolution.html
These projects represent about two hundred thousand lines of code (including comments), and are available at the following links:
-
http://qodem.sourceforge.net - Qodem, a curses-bases terminal emulator. Runs on Linux, Mac, and Windows. Emulates ANSI.SYS, Avatar, VT52, VT100/102, VT220, Linux console, a subset of Xterm, and some others. Connects via many methods. Supports file transfers with Xmodem, Ymodem, Zmodem, and Kermit. Written in C, Public Domain. Feature complete.
-
https://gitlab.com/klamonte/lcxterm - LCXterm, a curses-based terminal emulator. Only runs on Linux and other POSIX-type command line environments. Provides a subset of Qodem's features that can be used to supplement another terminal. Written in C, Public Domain. Feature complete.
-
https://jexer.sourceforge.io - Jexer, a text windowing system that resembles Turbo Vision. The first and only text windowing system that supports mixed text and images to Xterm via sixel. Runs on Linux, Mac, and Windows. Written in Java. Feature complete.
-
https://tjide.sourceforge.io - Tranquil Java IDE, a text-based development environment for Java. 80% feature complete.
-
https://gitlab.com/klamonte/jermit - Jermit, the serial file transfer protocols Xmodem, Ymodem, Zmodem, and Kermit. Qodem's file transfer protocols, rewritten in Java. 80% feature complete.
Late last year (2019) I got Jexer to feature complete. As part of that effort I spent several months looking around the "terminals" ecosystem and sharing what I have learned with other terminal emulator authors, and reached a decision to end this hobby. I did not bother tagging and releasing the 1.0.0, instead choosing to mark all of my projects archived or abandoned. I am working to find new things to do with my free time.
The rest of this document is historical notes and my personal reasoning that the era of VT100/Xterm-type terminals is nearing its end. By analogy: this is 1990, terminal emulators are the pre-Internet bulletin board systems, and while they are rounding out the feature set that could lead to RIPscript and Prodigy, someone out there is working on a web browser that will soon wash it all away.
The Command Line Versus The Terminal
Writing a minimally-working terminal emulator appears to be a common right of passage. They are cute to see, quick to get running, and after the initial hurdle of getting a basic curses application looking right one feels quite accomplished. However, it turns out that this is merely the first 20% of the effort to make a terminal good enough for real work, and maybe 2% of the work to make something truly new.
Being one of those people who has actually created something truly new, what I learned from the experience is that users really need the "command line", but not so much the "terminal". What do I mean by that?
-
By "command line", I mean pipe-able (composable) programs that read from stdin, write to stdout, and do a sequence of command / response / command / response. Just like the "read-eval-print-loop" (REPL) used by Lisp, Python, and many other programming languages.
-
In contrast we have the "terminal", meaning a grid of fixed cells comprising a screen of X columns by Y rows. There are many kinds of terminals such as mainframe, DOS, and now most commonly VT100 or "xterm" (really ANSI X3.64 / ECMA-48 type terminals). Nowadays every major desktop operating system has a good or getting-better terminal: the new Windows Terminal for Windows 10, Terminal.app for Mac, and dozens of fantastic open-sources choices for Windows/Mac/Linux/BSD/other (xterm itself, rxvt-unicode, gnome-terminal, konsole, mintty, iTerm2).
The evidence to me shows that people, in general, do not like the "terminal" part of the "command line". Not enough to replicate the DOS-era full-screen applications like Turbo Vision in the 27 years since its initial release, or in the 22 years since Borland abandoned it. Not enough to create a new Turbo Pascal or Turbo C++ IDE experience for other languages. Not enough to request that the most popular terminals pass vttest. Not enough to look around at other terminals and applications to make theirs notably better.
Looking back to the rise of the WYSIWYG word processors and GUI desktops of the 90's, it is clear that people tolerated fixed-width interfaces only when they had no other option. As soon as they had an honest choice between fixed-width text and variable-width text-and-pictures they have taken the latter path every time. I have come to believe that text-mode terminals are an unnatural means for people to work, and their existence is a very temporary state of affairs. There are people like me who grew up on DOS and actually liked it, or got into Unix/Linux later on and enjoyed the quickness of mouse-less typing around, but we are a tiny minority in a very large sea of people. Most people have a much easier time recognizing pictures than reading words, much less scanning a screen full of words to find what they need.
At the moment there is a push for making it easier to automate the management of thousands of cloud VM instances, and for this the command line is well suited. Every language is picking up the basics of an Xterm widget that can run the existing commands in a terminal, and some of them are even going a little further down the road and making that terminal pass (most of) vttest. But these efforts aren't any attempt to popularize the "terminal", they exist solely to gain access to existing "command line" monitoring and management tools. These tools need their stdin/stdout pipes, but they do not need the terminal's fixed-width display.
What Do People Really Need?
I was the first to demonstrate several features. As such, I have learned that a feature is only interesting to others if it is something that they would like to use once they have seen it. If you have a feature, and people know about it, and it doesn't appear anywhere else after several years, then the market has spoken: that feature is not actually needed. So what has the market said to me about what people need in a terminal?
-
What they do not need are: vttest compliance, custom keybindings, Zmodem/Kermit file transfers, scripting, screen dump, scrollback dump, session capture, sixel graphics, and text-based windowing systems.
-
What they do need are: full-width glyph support (CJK), emojis, 24-bit RGB, tabbed windows, reflowable text, clickable URLs, and background transparency.
When you compare these lists of features, it looks an awful lot like what people actually need from their terminal are the same underlying features of SMS/iMessage mobile apps: language support, colors, emojis, and pictures. iMessage is in fact a "command line", just with another person (presumably) on the other end rather than a computer. Again: "command line", not "terminal".
So let us imagine tomorrow's "command line":
-
Start with an interface where the commands and responses are in alternating bubbles on the left/right side, just like a smartphone's text messaging app.
-
Let it use proportional fonts rather than fixed-width cells, using all of the multinational language support that smartphones and browsers now have.
-
Make the pipes between command line programs object-oriented like Powershell, but with a less verbose syntax (e.g. txt speak).
-
Create content handlers for the final responses shown to the user: pull up tabular data in a live spreadsheet widget, put free-form text in a browser-like reflowable textarea with live links for URLs, display images and animations in full (and fast) truecolor, and so on.
-
Add gestures for copy/paste, command history, and searchable scrollback.
You now have "iMessage meets computer", an interface that can fully supplant the "terminal". It will most likely be written in a dialect of JavaScript, written mostly by Windows desktop users, and run smoothly on server, desktop, and mobile.
The End Of The Xterm Terminals
I think something like "iMessage meets computer" will come in the next few years, and it will wipe out the "full-screen terminal" for future generations of developers. It won't be instant, but experiments like Powershell and Nu Shell are already underway. Once it is here and running well on Windows and Mac, a noticeable chasm will form between the (generally older) users of "Xterm-terminals" and (generally younger) users of "iMessage-terminals". After about 5-15 years there will be a majority user base for the "iMessage-terminals", and though millions of people will use the "Xterm-terminals" they will be at the back of the line for support on the latest-and-greatest desktops, tablets, and whatever else is in use.
In the meantime, as you look around the terminals ecosystem, you see an eery echo of the BBS era's circa 1993. In that time the BBS developers were looking into incorporating GUI elements (NAPLPS, RIPscript) in order to stay relevant against the rapidly growing AOL and World Wide Web, but these were heavyweight features that most of their existing userbase did not need or want.
Today I see terminal authors digging deeply into the corners of various ECMA and ITU standards (that have never been fully implemented by anyone) to find escape sequences to support features for hypothetical future applications in spaces that often have only one or two applications already there and apparently zero demand for more. Look in the Debian archive, and you see: two terminal multiplexers, one terminal-based word processor, one terminal-based spreadsheet. By contrast there are lots of terminal-based programming and system admin tools. I expect that the terminal will become the niche of professional computer programmers and admins, just as the RS232 serial port has become the niche of embedded/industrial developers.
If You Want To Write Terminal Applications
I would never tell anyone not to write code. If you want to write terminal applications, go for it.
I would only caution you not to do this with any hope or expectation that you will find peers to talk to along the way or at the end. Keep your expectations very low, because unless your project is tapped by a much larger player (e.g. included in an IDE or shipped with a commercial package), you are most likely to get no feedback. Your victories will be cool, and other terminal authors may understand some of the tricks you figured out, but no one else will.
If you are writing an actual terminal, get just enough to support curses applications and then step back. Only get to vttest if you really find yourself needing it.
If you are writing a terminal application, decide if it can be a pipe-able command line application first. If so, just do that and don't build a curses-like interface. If users show up who want pretty output, put together something that takes that command line output and puts up a colorized screen.
If you want your own full-screen text-based user interface (TUI), try using termbox first. It has bindings to a bunch of languages, and looks cool. Only use curses if you really need to support non-xterms like real hardware terminals.
If you really need a TUI that looks and behaves like the best of the DOS-era applications, you can give Jexer a try. It is the easiest to get running and looks great, but it has very few users so bugs are lying in wait and you will be doing your own maintenance. If you hate Java with a passion, try urwid (Python) or gowid (Go).
Where I'm going with all of this is: the "terminals" mountain is much higher than you think, and it will very likely not be worth the time you invest in it.